Recovered after the 7.0 earthquake off of the coast of CA. Recovery was basically automatic, the only thing I did was untrip ISIs and SUS watch dogs when I could, and tell guardian when to start initial alignment.

Initially we went into Observing but Francisco realized that his his from earlier today (alog81630) didn't make it in. We updated lscparams.py with the new value and the Observe.snap. The safe.snap was saved at a value of 1.0, so we must be relying on ISC_LOCK to make the change.

One other SDF was found from the pico H delay. This was unmonitored.

Camilla, Sheila, help from CDS team

We've been having some locklosses during the ESD transitions, some of them seem to be related to a large glitch that happens at the end of the 10 second output ramp time when DARM1 FM1 (a boost) is turned off in preparation to switch back to ETMX control where the boosts are in the pum rather than the DARM bank. 

Erik and Jonathan told us that since the filter ramp is linear, it's not suprising that there is a discontinuity when the ramp time ends.  They have opened a ticket after a discussion about ways to avoid this: 667

We could make this better by increasing the ramp time, but we want to keep this state short because of a slow instability we can get if we wait here: 81430. 

For now we've removed the boost which was being turned on in the state DARM offset before the ASC gets engaged, during the power up. The attachments compare the ESD drives with and without this filter being ramped off, the glitch is gone.

At Sheila's suggestion, I've turned the IMC gain redistribution back on in LASER_NOISE_SUPPRESSION, ISC_LOCK state number 575. Just a reminder, this is the code (lines 5793-5806 now):

       if self.counter ==5 and self.timer['wait']:
            if self.gain_increase_counter <7:  #icrease the imc fast gain by this many dB
                #redistribute gain in IMC servo so that we don't saturate splitmon in earth quakes, JW DS SED
                #setting this to not change the gain sliders October 9, because Camilla saw that the IMC locklosses that we think are due to laser glitches started happening on the day we did this gain redistribution. alog 80561
                ezca['IMC-REFL_SERVO_IN1GAIN'] -= 1
                ezca['IMC-REFL_SERVO_IN2GAIN'] -= 1
                ezca['IMC-REFL_SERVO_FASTGAIN'] += 1
                time.sleep(0.1)
                self.gain_increase_counter +=1
            else:
                self.counter +=1
        
        if self.counter >= 6 and self.timer['wait']:
            return True

This can be turned off by setting changing gain_increase_counter <0, on line 5794. If we are losing lock during this state we should turn it off again, or reduce the gain_increase_counter to <5 or something.

FRS9371

The CDS alarms system status is now shown on the CDS Overview MEDM. The attached snapshots show the system when there are no alarms, and when two test alarms were raised.

Clicking on the "CDS ALARM" button on the overview opens the alarm MEDM.

Details:

The locklossalert EPICS IOC was modified to provide, as EPICS PVs, the ALARMS GPS time, the number of active alarms, and the channels which are in alarm.

locklossalert and alarms both run on cdslogin, and so a temporary file is used for the interprocess communication. Alarms updates the file on the minute and LLA reads the file at the 30 second mark.  If the temporary file stops being updated, the oveview button will turn magenta to indicate a stopped service.

With the IFO down due to an earthquake, went out to the LVEA to verify installation and location of the Picomotor Driver E. This controller was slotted for Optical Lever but never implemented.

Control and readback cables are connected to the Corner 2 Slow Controls Chassis (D1100680) in the CER. Cables labeled H1:IO_305 and H1:IO_306 are connected in slot 5 on the rear of the chassis. Cables are routed to the HAM6 ISC racks. No driver is installed in the LVEA. Cables are disconnected and coiled up in the cable tray.

7.0 off the coast of N. California. Tripped all ISIs and many suspensions.

Thu Dec 05 10:07:59 2024 INFO: Fill completed in 7min 56secs

Jordan confirmed a good fill curbside.

FranciscoL, SheilaD

Changed H1:SUS-ETMX_L3_DRIVEALIGN_L2L_GAIN by 3.63 % using KappaToDrivealign.py at gps 1417450784.

Script output:

Average H1:CAL-CS_TDEP_KAPPA_TST_OUTPUT is -3.6267% from 1.
Accept changes of    
H1:SUS-ETMX_L3_DRIVEALIGN_L2L_GAIN from 191.711517 to 198.664299
Proceed? [yes/no]
yes
Changing
H1:SUS-ETMX_L3_DRIVEALIGN_L2L_GAIN
H1:SUS-ETMX_L3_DRIVEALIGN_L2L_GAIN => 198.6643

First figure (kappa2drivealign_ndscope) is an ndscope of the relevant channels. The the second plot (top to bottom) is CAL-CS_TDEP_KAPPA_TST_OUT with a verticar marker aligned at a time where the uncertainty from CAL-CS_TDEP_PCAL_LINE3_UNCERTAINTY (third plot)  increased. This increase in uncertainty is expected from turning off lines before calibration measurement.

KAPPA_TST_OUT is around 1 from minute 5 to the marker, which is the objective of changing the drive gain.

Following our usual instructions.

Simulines start:

PST: 2024-12-05 08:35:51.659267 PST
UTC: 2024-12-05 16:35:51.659267 UTC
GPS: 1417451769.659267
 

Simulines end:
PST: 2024-12-05 08:59:32.899615 PST
UTC: 2024-12-05 16:59:32.899615 UTC
GPS: 1417453190.899615
 

2024-12-05 16:59:32,822 | INFO | File written out to: /ligo/groups/cal/H1/measurements/DARMOLG_SS/DARMOLG_SS_20241
205T163552Z.hdf5
2024-12-05 16:59:32,830 | INFO | File written out to: /ligo/groups/cal/H1/measurements/PCALY2DARM_SS/PCALY2DARM_SS
_20241205T163552Z.hdf5
2024-12-05 16:59:32,834 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L1_SS/SUSETMX_L1_SS
_20241205T163552Z.hdf5
2024-12-05 16:59:32,838 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L2_SS/SUSETMX_L2_SS
_20241205T163552Z.hdf5
2024-12-05 16:59:32,843 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L3_SS/SUSETMX_L3_SS
_20241205T163552Z.hdf5
ICE default IO error handler doing an exit(), pid = 911686, errno = 32

We should return to Observing at 20 UTC.

TITLE: 12/05 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 161Mpc
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 3mph Gusts, 1mph 3min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.17 μm/s
QUICK SUMMARY: Locked for 36 hours! Looks like we just made it through a 5.6 from Alaska, as well as some smaller quakes in CA. Otherwise a calm environment and no alarms.

TITLE: 12/05 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 161Mpc
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Very quiet shift with H1 observing throughout; current lock stretch is now over 26 hours. Only thing to report is that the SQZ_PMC Guardian has occasionally been saying the PMC PZT volts are low; currently around 6V and has been low for a few hours (see attached trend), but hasn't unlocked.

FAMIS 28455, last checked in alog81106

Only things of note on these trends are that the ITMX spherical power has been dropping for the past week and a half, which Camilla agrees looks strange, and the ITMY SLED power has reached the lower limit of 1, so it will need replacing soon. Everything else looks normal compared to last check.

TITLE: 12/04 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 158Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 4mph Gusts, 2mph 3min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.17 μm/s
QUICK SUMMARY: H1 is running smoothly, has been observing for 20+ hours.

• Wind and secondary microseism low
• All other systems nominal

TITLE: 12/04 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 158Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY: Locked for 20 hours. Very quiet shift with nothing to report.
LOG:                                                                                        

FAMIS 21691

pH of PSL chiller water was measured to be just above 10.0 according to the color of the test strip.

Ansel reported that a peak in DARM that interfered with the sensitivity of the Crab pulsar followed a similar time frequency path as a peak in the beam splitter microphone signal. I found that this was also the case on a shorter time scale and took advantage of the long down times last weekend to use  a movable microphone to find the source of the peak. Microphone signals don’t usually show coherence with DARM even when they are causing noise, probably because the coherence length of the sound is smaller than the spacing between the coupling sites and the microphones, hence the importance of precise time-frequency paths.

Figure 1 shows DARM and the problematic peak in microphone signals. The second page of Figure 1 shows the portable microphone signal at a location by the staging building and a location near the TCS chillers. I used accelerometers to confirm the microphone identification of the TCS chillers, and to distinguish between the two chillers (Figure 2).

I was surprised that the acoustic signal was so strong that I could see it at the staging building - when I found the signal outside, I assumed it was coming from some external HVAC component and spent quite a bit of time searching outside. I think that this may be because the suspended mezzanine (see photos on second page of Figure 2) acts as a sort of soundboard, helping couple the chiller vibrations to the air. 

Any direct vibrational coupling can be solved by vibrationally isolating the chillers. This may even help with acoustic coupling if the soundboard theory is correct. We might try this first. However, the safest solution is to either try to change the load to move the peaks to a different frequency, or put the chillers on vibration isolation in the hallway of the cinder-block HVAC housing so that the stiff room blocks the low-frequency sound. 

Reducing the coupling is another mitigation route. Vibrational coupling has apparently increased, so I think we should check jitter coupling at the DCPDs in case recent damage has made them more sensitive to beam spot position.

For next generation detectors, it might be a good idea to make the mechanical room of cinder blocks or equivalent to reduce acoustic coupling of the low frequency sources.