WP6120

Daniel, Keita, Jim, Dave:

We installed a spare Dolphin card into the h1psl0 front end computer. The sequence was:

1. change h1ioppsl0 model to add the pciRfm=1 flag
2. edit h1psliss model to add two dolphin receivers and process this data
3. edit h1lsc model to pickoff TR_[X,Y] and send as two new dolphin channels
4. edit dolphin manager's node configuration and restart on h1boot
5. h1psl0: kill models, power down, install Dolphin card, connect to switch, power up.
6. On power up of h1psl0 the MSR Dolphin fabric glitched (as we suspected) which stopped all SUS, SEI and ISC models in the corner station.
7. h1ioppsl0 IRIG-B drifted negative, we waited until it became good before we could check the new Dolphin IPC was functioning correctly.
8. we restarted the sender (h1lsc0), but the dolphin was not working. After scratching my head a bit, I realised I had not loaded the new h1lsc model. Once I did this, all was working correctly.
9. restarted all models on all MSR front ends which are on dolphin
10. restarted the DAQ to resync to the new h1psliss model

We found that the PSL-OPC shutter still closed, investigating this offline.

I checked that the received data on the two new dolphin channels were correct, and they are.

I noticed that the channels are called TR_[X,Y]_NORM but they are picked off before the NORM filter modules in the LSC model, so perhaps not normalized or incorrectly named.

Running a Worden experiment:

-CP4 at 88% full
-Set LLCV to nominal in manual mode (41% open)
-Increase LLCV by 5% (43% open) and leave it at this value
-Monitor increase in fill
-Allow fill to exceed 100% and monitor exhaust flow meter (expect a sharp increase in flow when liquid starts to hit warm exhaust)
-After this sharp increase, reduce LLCV by 10% to below nominal (<41%)
-Hope that no LN2 reaches the flow meter this time (when flow approaches 100% I will monitor while sitting next to exhaust to be ready to open bypass exhaust valve and remove flow meter with LN2 gloves on)

**THIS WILL GENERATE CP4 ALARM ON PUMP LEVEL**

DIAG_MAIN was reporting that IM2 P and Y were out of normal range, and after trending them I saw that they were WAY out of range (Normal for H1:SUS_IM2_M1_DAMP_P_INMON is 603 and -205 for Y). The values now are at P = 0 and Y=-19. The trend shows a sudden move at around 11:15UTC (4:15PST), when no one was here and the IFO was unlocked. The most recent lockloss before that was at 8:03UTC. The suspension has been aligned with no trips since the 18th, and HAM2 has been Isolated since the 19th, no trips.

The other IM's also seem to show signs of movenment then, but nothing as extreme as IM2. 1&3 seemed to get noisy after 11:15. (see attachment)

What could have caused this move?

Chandra, Kyle

Attached are scans following the most recent bake of the Vertex RGA. The pressure in the Vertex at the time of these scans was 3.4 x 10-9 torr as indicated by the nearest CC gauge.  The bake out of this RGA was resumed following these scans.

[Jenne, Terra]

We've been pretty frustratingly plagued by the CSOFT / dPdTheta instability today.  Earlier today with Sheila, we were able to use the offsets from yesterday, but then later in the evening those offsets make things unstable when we go up in power.  I'm starting to wonder if an initial alignment needs to be done, since the green arm powers (measured at LockingArmsGreen, or anything early in the sequence) have been decreasing with each lock.  Maybe that will help? 

Sheila migrated the Soft offsets into the trans QPDs, so the IFO now aligns to that placewhen the soft loops come on.  Also, the POPA offsets are engaged during ASC_Part2, so the IFO is aligned to yesterday's place.  However, with this alignment at 2W, we cannot engage the roll mode damping.  The last few hours we've been skipping over EngageRollModeDamping, and I've commented out the final roll mode gain setting that used to happen in DC readout.  We don't seem to usually have much of a problem just leaving the damping off, but we can turn it on once we're at at least a semi-high power (maybe 20+ W?). 

Since we kept being troubled by the CSOFT instability, we went back to trying new offsets.  We think there's a little more that could be done, but we have a place where the recycling gain stays fairly constant as we power up.  The recycling gain looks terrible at 2W (30ish), but then only decays to 28ish.  This is in comparison with the usual decay to 22ish.  Also with this alignment, when we're at high power, the green arm transmissions are both high.  This seems like something that we want, since we think that the green and red beams are pretty well co-aligned after the Soft loops come on, so hopefully this means that we're closer to finding a place that maintains the alignment of the IFO from 2W to 50W. 

In the SDF screenshot, the POP_A offsets in the setpoint are what Sheila and Terra found last night, while the current values are the place that we like best so far tonight.  The setpoint offsets for the soft loops aren't meaningful, but the current values are the ones that we like from tonight, which are on top of last night's offsets (which have already been put into the QPDs and accepted in SDF). 

Attached also is a screenshot showing two striptools (and a bunch of other stuff that's basically ignorable).  Right around -100 minutes is the inital power-up and corresponding power recycling gain drop.  You can see that as we move offsets around, we're obviously changing the red and green arm powers, as well as the recycling gain.  Note though the time around -15 min where both green arm transmissions (blue and teal) are high, and the PRC gain is high.  Xarm green seems most strongly affected by Pop_A_pit, while Yarm green seems most strongly affected by DSOFT. 

To help maintain these locks, we were increasing the ISS 3rd loop gain from -1 to -10.  Also, we increased the SOFT Yaw gains from 3 to 7, and the Soft Pit gains from 0.5 to 0.7.  This seemed to ameliorate most instabilities, although we still are sometimes struggling to hold the lock - I think it's perhaps an indicator of needing an initial alignment.

H1 Commissioning

Jenne, Terra, & Sheila were at the helm for H1 higher power work.

MEDM work

Made a corrections to the sitemap (IFO Common Mode Servo on pull-down), and to the Digital Video Overview (added name for POP Air camera at the CAM30 postion).

Stranger Things Happening Around Gate Tonight

Happened to catch a vehicle rolling on site around 9:45 (it turned toward the LSB, turned around, & lingered before leaving).  The other oddity here was this car illuminated another vehicle as it approached.  This vehicle was at the gate with lights off.  Monitored the care for about 30min, called out to the Gate phone, and then three of us went to investigate.  There were about 4-6people, and they were looking at the stars.  They said they had talked to someone on the phone about coming out to watch the sky. 

Upon returning to the OSB, we checked the external front door lock to make sure it hadn't been unlocked (luckily it was locked).  The group eventually left (so they were here for atleast an hour.  

Per FAMIS #7068, Saw that two ISIs needed their L4C WD counters to be cleared.  

• HAM5 was at 16
• ETMy was at 104

Jeff K, Darkhan T,

Last Tuesday we updated the infrastructure for injecting calibration lines (see LHO alogs 29245, 29249). Below is the table of currently active calibration lines:

Channel Names                 _FREQ (Hz)  _SINGAIN (ct)
H1:CAL-PCALY_PCALOSC1_OSC        36.7            125       O1-scheme kappa_TST / kappa_PU
H1:CAL-PCALY_PCALOSC2_OSC       331.9           2900       O1-scheme kappa_C / f_C

H1:CAL-PCALY_PCALOSC3_OSC      1083.7          15000       high-frequency calibration check ("bonus" line)

H1:CAL-PCALX_PCALOSC1_OSC      3501.3          39322       high-frequency sensing function characterization ("mobile" line)

                              _FREQ (Hz)  _CLKGAIN (ct)
H1:CAL-CS_TDEP_DARM_LINE1_DEMOD  37.3              0.1     O1-scheme kappa_PU
H1:SUS-ETMY_L3_CAL_LINE          35.9              0.11    O1-scheme kappa_TST / kappa_PU

H1:SUS-ETMY_L1_CAL_LINE          33.7             11       O2-scheme synched oscillator for kappa_U
H1:SUS-ETMY_L2_CAL_LINE          34.7              1.1     O2-scheme synched oscillator for kappa_P
H1:SUS-ETMY_L3_CAL2_LINE         35.3              0.11    O2-scheme synched oscillator for kappa_T

We plan to adjust the three O2-scheme line frequencies and amplitudes, and cancel them out with PCALY (they will not appear in the reconstructed DARM spectrum), following synchronized oscillators will be utilized for this purpose H1:CAL-PCALY_PCALOSC{4-6}_OSC.

We were locked last night at 50 W for ~2.5 hours and lost lock from an ITMX 15522 Hz PI. This is a known PI seen months ago; I had purposefully left damping settings off to see if it rang up. See first attachment showing lockloss. 

This afternoon we were locked at 50 W for ~2 hours and I let the mode ring up so I could demonstrate successful damping. See second attachment showing damping. 

All PIs that were previously observed have now been seen and damped post OMC vent. We have 5 PIs at 50 W  (at least up to ~3 hour locks): ITMX 15520 Hz; ETMX 15541 Hz; ETMY 15542 Hz, 15009 Hz, 18041 Hz (aliased from 47495 Hz).  All are successfully damped via the guardian and have had their damping phase and gain optimized. 

Last week, Keith posted the results of a study of folded magnetometer channel data (alog 29166) aimed at understanding the results of recent changes to the timing system (primarily LED reprogramming and power supply switching). This is a follow-up, looking at the spectra of the same channels, and tracking the behavior of the two combs which the timing system interventions were intended to mitigate.

Detailed plots

Overview table (daily spectra, selected dates)
Full data set (daily spectra)
Full data set (cumulative spectra since Jul 1 2016, covering date ranges where Fscan SFTs were available)
These plots were generated from Fscans + spec_avg_long + my own plotting tools.

Timeline

July 14-21 comparison: before and after initial updates to timing slave card firmware (blinking LEDs turned off in many places, but not on timing fanouts)
July 21-Aug 6 comparison: firmware updated for EX fanout; CPS timing fanout power supply changed
Aug 6 - Aug 18 comparison: firmware updated for CER, MSR, EY fanouts

Notable features

This afternoon I ramped CP4's LLCV in 5% increments every 2 minutes, from 39% to 88% open, from 88% full to 100% full, for more data from exhaust flow meter. 

Fill level SP has been reset to 92% and level is slowly coming back down at 20% open on LLCV.

Kudos to Patrick for writing an effective PI-code. It works very well for overfill scenarios! 

Gerardo, Chandra

On Tuesday, Aug. 23rd, we adjusted potentiometer on PT-140a (pirani) again - this time CCW 11 turns. Since Gerardo terminated cables for AIPs, gauge voltage has changed again and needs to be adjusted again so CC does not keep tripping due to set point interlock.

TITLE: 08/24 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Corey
SHIFT SUMMARY: Commissiong work continues.
LOG:

• 15:43 Kyle to LVEA to check on RGA
• 15:54 Kyle out
• 16:05 Betsy, Travis to LVEA looking for parts.
• 16:16 Betsy, Travis out
• 17:04 Fil, Ed M to MY
• 17:57 Fred, Hannah, and guest driving down the arms
• 19:20 Travis, Betsy to MX
• 14:13 Kyle to LVEA RGA
• 14:17 Kyle out
• 15:09 Handford Fire at the gate and then driving down X arm.

1425 - 1440 hrs. local -> To and from Y-mid 

Opened exhaust check-valve bypass-valve, opened LLCV bypass valve 1/2 turn -> LN2 @ exhaust in 1 minute 20 seconds -> Restored valves to as found configuration.  

Next CP3 overfill to be Friday, August 26th.

I was asked to summarize the SWWD (software watchdog) timing sequence as a reminder.

t=0: SUS IOP detects top OSEM RMS exceeds trip level, starts its 1st countdown (5 mins)

t=5mins: SUS IOP 1st countdown expired, its IPC output goes to BAD and it starts its 2nd countdown (15 mins). SEI IOP receives the BAD IPC, and starts its 1st countdown (4 mins)

t=9mins: SEI IOP 1st countdown expired, its IPC output goes to BAD and it starts its 60 second 2nd countdown. SEI user models get the 60 second warning IPC so they can cleanly shutdown before the DACs are killed

t=10mins: SEI IOP 2nd countdown expired, DAC cards associated with the chamber the SUS is located in are killed

t=20mins: SUS IOP 2nd countdown expired, SUS DAC cards are killed

For the hardware watchdog (HWWD) the times are doubled. The power to the ISI Coil Driver chassis is removed after 20 mins of continuous SUS shaking.

S. Dwyer, J. Kissel, C. Gray

After successfully recovering the IMC's VCO and recovering the IMC (29264), we were able to get up through LOCKING_ARMS_GREEN in the lock acquisition sequence. However, we found that ALS COMM failed caused lock losses during the next step (LOCKING_ALS), when the input for IMC length control in its Common Mode Chassis was switched from the IMC's PDH output to the ALS COMM PLL output. The ALS COMM PLL output is connected to IN2 of the IMC chassis that had a new daughter board installed in the star-crossed ISC rack H1-ISC-R1 today (LHO aLOG 29250). After fighting through MEDM screen confusion* at the racks, we found that OUT2 (an analog pickoff pick-off just after the input gain circuit) indicated a ~2.5 [V] offset, even with IN2 terminated with 50 [Ohms]. 

Suspecting that this symptom was indicative that the input gain circuit (circled in red in MEDM screen capture) was yet another causality of the unfortunate rack power mishap today (LHO aLOG 29253), we've replaced the entire chassis (which lives in U14 of H1-ISC-R1) with a spare we found in the EE shop -- S/N S1102627 (or Board S/N S1102627MC). 

Notably, this spare does not have one of the new daughter boards on which Chris has worked so hard.

We're not suggesting this swap be permanent, but we make the swap for tonight at least, so we can hopefully make forward progress. We suggest that IN2 and/or the input gain stage of SN S1102626 be fixed tomorrow, and the chassis restored so we can employ the new daughter board.

Other Details:
- Before removing the chassis, we powered down the entire rack using the voltage sequencer around the back at the top of the rack. 
- After installing the rack, we were sure to have all cables connected appropriately before turning the rack power on again (via the sequencer again).
- We added a few labels to the IMC's PDH output and the ALS COMM PLL output cables such that they're easier to follow and reconnect in the future.

*MEDM Screen Confusion -- whether IN1 or IN2 is fed into OUT2 of all common mode chassis is selectable on their MEDM screens. For the IMC's common mode board (at least for SN S1102626), the MEDM screen's indication of the status of that switch is exactly backwards. When the screen indicates that IN1 is feeding OUT2, IN2 is feeding OUT2, and vice versa. #facepalm

Summary:
Repeating the Pcal timing signals measurements made at LHO (aLOG 28942) and LLO (aLOG 27207) with more test point channels in the 65k IOP model, we now have a more complete picture of the Pcal timing signals and where there are time delays.

Bottom line: 61 usec delay from user model (16 kHz) to IOP model (65 kHz); no delay from IOP model to user model; 7.5 usec zero-order-hold delay in the DAC; and 61 usec delay in the DAC or the ADC or a combination of the two. Unfortunately, we cannot determine from these measurements on which of the ADC or DAC has the delay.

Details:
I turned off the nominal high frequency Pcal x-arm excitation and the CW injections for the duration of this measurement. I injected a 960 Hz sine wave, 5000 counts amplitude in H1:CAL-PCALX_SWEPT_SINE_EXC. Then I made transfer function measurements from H1:IOP-ISC_EX_ADC_DT_OUT to H1:CAL-PCALX_DAC_FILT_DTONE_IN1, H1:IOP-ISC_EX_MADC0_TP_CH30 to H1:CAL-PCALX_DAC_NONFILT_DTONE_IN1, and H1:CAL-PCALX_SWEPT_SINE_OUT to H1:CAL-PCALX_TX_PD_VOLTS_IN1, as well as points in between (see attached diagram, and plots)

The measurements match the expectation, except there is one confusing point: the transfer function H1:IOP-ISC_EX_MADC0_TP_CH30 to H1:CAL-PCALX_DAC_NONFILT_DTONE_IN1 does not see the 7.5 usec zero-order-hold DAC delay. Why?

There is a 61 usec delay from just after the digital AI and just before the digital AA (after accounting for the known phase loss by the DAC zero-order-hold, and the analog AI and AA filters). From these measurements, we cannot determine if the delay is in the ADC or DAC or a combination of both. For now, we have timing documentation such as LIGO-G-1501195 to suggest that there are 3 IOP clock cycles delay in the DAC and 1 IOP clock cycle delay at the ADC.

It is important to note that there is no delay in the channels measured in the user model acquired by the ADC. In addition, the measurements show that there is a 61 usec delay when going from the user model to the IOP model.

All this being said, I'm still a little confused from various other timing measurements. See, for example, LLO aLOG 22227 and LHO aLOG 22117. I'll need a little time to digest this and try to reconcile the different results.