Rahul, Camilla, Jonathan, Erik, Dave:

At 07:33 PST during a measurement this morning the ETMX test mass was set into motion which exceeded the user-model, SWWD and HWWD RMS trigger levels. This was very similar to the 02 Dec 2023 event which eventually led to the tripping of the ETMX HWWD.

The 02 Dec event details can be found in T2300428

Following that event, it was decided to reduce the time the SUS SWWD takes to issue a local SUS DACKILL from 20 minutes to 15 minutes. It was this change which prevented the ETMX HWWD from tripping today.

The attached time plot shows the details of today's watchdog events.

The top plot (green) is h1susetmx user-model's M0 watchdog input RMS channels, and the trigger level (black) of 25000

The second plot (blue) is h1susetmx user-model's R0 watchdog input RMS channels, and the trigger level (black) of 25000

The lower plot shows the HWWD countdown minutes (black), the SUS SWWD state (red) and the SEI SWWD state (blue)

The timeline is:

07:33 ETMX is rang up, M0 watchdog exceeds its trigger level and trips, R0 watchdog almost reaches its trigger level, but does not trip.

At this point we have a driven R0 and undriven M0, which was also the case on 02 Dec which keeps ETMX rung up above the SWWD and HWWD trigger levels

The HWWD starts its 20 minute countdown

The SWWD starts its 5/15 minute countdown

+5min: SEI SWWD starts its 5 minute countdown

+10min: SEI SWWD issues DACKILL, no change to motion

+15min: SUS SWWD issues DACKILL, R0 drive is removed which resolves the motion

HWWD stops its count down with almost 5 minutes to spare.

We have opened a workpermit to reduce the sus quad models' RO trigger level to hopefully always have M0 and R0 trip together which will prevent this is the short term. Longer term solution requires a model change to alter the DACKILL logic.

WP 11566

A thermocouple was installed on the floor between BSC2 and BSC3 to monitor temperature deviations from the LVEA floor and LVEA ambient air. Thermocouple connected to LX vacuum slow controls chassis, Terminal M21, Channel 4 (Connector G, pins 29 & 30 on side of rack).

F. Clara, G. Moreno

Using the Fluke 62 Mini Thermometer (IR) I checked the status of the input and output air on all of the Kepco power supplies at EX, EY, and CER Mezzanine per WP11586

EX and CER tested normal.  Air flow is good, all supplies humming away, no odd vibrations.

At EY, ISC-YC1-TCS -24V supply looked to have a stuck fan.  The supply is part of the ISC-YC1-TCS +/- 24V pair located in slot 18 of the C2 Power rack on the right hand side (RHS).

Ambient air at EY is 65F, air measured at the fan braket at 110F, and air measured output air at 80F.  This supply shows -24V at 0A, low current is the reason it has not tripped already.

This pair of supplies powers pair of RF Oscillators and RF Amplifiers at EY.  This is part of the Low Noise Power system thus its current draw on the +/- 24V is very low, as it is a reference.

Last checkup of supplies was September 19th, linked here ALOG72968 all supplies checked out normal.  No issues were reported between then and now.

We replaced the supply with an upgraded spare with new ball bearing fan installed per WP11588 and we can hope for smooth sailing through the Christmas break.

Kepco with Failed Fan SN = S1300290

Kepco with Ball Bearing Fan SN = S1201931

M. Pirello, F. Clara

Lights and Mega CR light off, paging system unplugged, WAP off.

As TJ found last week, Robert's shaker is still connected to near HAM2, it is plugged in but powered off.

All else looked good, followed T1500386

Maintenance day ran a bit long due to an issue with the HEPI pump station at EX and an emergency power supply swap at EY, but all activities have now finished and H1 has started initial alignment.

Seismon and several other IOCs, including lveatemps, picket fence, and external alerts, were restarted.

Camilla, Ansel

At 17:46UTC, Camilla changed ITMX and ITMY HWS to 'sem 3', which is external trigger mode. This seemed to work, and stopped the code taking photos.

It appears that the HWS-associated combs are gone in the magnetometer witness channel after this change. Pre/post 1-hour spectra attached. (All known HWS-associated combs overlaid, just to check-- the near-7Hz is the one actually present in the "pre" spectrum which corresponds to current HWS sync frequency settings.)

Sheila, Naoki, Vicky - SQZ OMC mode scans with cold OM2

• 76mW seed power launched on SQZT0, and OPO = LOCKED_SEED_DITHER + FC = MISALIGNED:
  • To start, we measured opo ir seed trans power on sqzt7: OPO_IR_PD_LF_OUTPUT = 0.876mW, and OPO_IR_PD_DC_POWERMON = 0.888 mW. PD calibration LHO:72761, Sept 2023.
    • After seed dither locking at 1387039858, opo ir trans increased ~1% in the first 5 min and settled.
• Open SQZ beam diverter, align SRM, misalign SR2, open ham6 fast shutter.
  • immediately,  ASC-AS_A_DC_NSUM_OUT16 = 19,  ASC-AS_B_DC_NSUM_OUT16 = 51-52.  ASC-AS_C_NSUM_OUT16 = 680e-6.
• Engaged DC{3/4} centering loops - just worked. Will leave DC3/4 on for mode-scans.
  • This aligned ASC-AS_{A/B}_DC_NSUM_OUT16 to 51 counts both. On OMC QPD's, ASC-OMC_{A,B}_NSUM_OUT16 are each ~630e-6.
• OMC ASC master gain "one click up" to 0.025.
• Lock OMC on SQZ beam. (scan OMC PZT2, find max, set trigger value to 0.1, manual OMC guardian: DOWN, ASC_QPD_ON, PREP_LOCK_OMC, OMC_LSC_ON, OMC_LOCKED. Freeze OMC ASC. Manual optimize alignment using OM3/OMC sliders.)
• Take 1 minute "LOCKED" time.
• Turn off OMC LSC servo (input,output). Move OMC PZT off-resonance, take 1 minute "UNLOCKED" time.
• Take OMC mode scan (run DTT template).

Taking SQZ-OMC mode scans, using DTT template saved in $(userapps)/sqz/h1/Templates/dtt/OMC_SCANS/Dec19_2023_PSAMS_OMC_scan_coldOM2.xml

PSAMS 200/200, cold OM2

• Locked: 1387045702 + 60 seconds.  OMC_DCPD_SUM_OUT_DQ~0.476.
• Unlocked: 1387045917 + 60 seconds.
• Mode-scan (ref 30) cyan. 19/12/2023 18:32:56 UTC (1387045994) + 200 seconds.

PSAMS 120/120, cold OM2

• Locked: 1387047076 + 60 seconds.  OMC_DCPD_SUM_OUT_DQ~0.465.
• Unlocked: 1387047200 + 60 seconds.
• Mode-scan (ref 31) pink. 19/12/2023 18:55:33 UTC (1387047351)+ 200 seconds.

Dark

• Dark noise: Turned off DC centering, fast shutter, sqz beam diverter.
• Dark: 1387048209 + 60 seconds.

Tue Dec 19 10:06:51 2023 INFO: Fill completed in 6min 47secs

Over the past few weeks we've occasionally been seing the notice on DIAG_MAIN that the IMC WFS need to be centered, so I did that during the maintenance window this morning, The process was as follows:

1. Offload IMC WFS by taking IMC_LOCK to 'MCWFS_OFFLOADED'
2. Take IMC_LOCK to 'OFFLINE'
3. Using the MCWFS centering scope and picomotors on the IMC overview screen, center the signal on each WFS
4. Accept IMC PZT offset SDFs from the previous offloading

I'm leaving the IMC offline for SQZ/OMC scans.

TITLE: 12/19 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 12mph Gusts, 8mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.28 μm/s
QUICK SUMMARY:

After a test this morning caused H1 to lose lock at 15:33 (see alog74887), maintenance activities have begun. Camilla is in the process of untripping SEI WDs.

Camilla, Louis

We dropped out of Observing at 7am PST test the new DARM loop config: 

- the new DARM loop config, navigated to by going to NLN_ETMY -> NEW_DARM, seems to be stable. We were able to sit there for several minutes without issue.
- Camilla took a quick look at MICH FF settings while in 'NEW_DARM'. She will follow up with more info on that. 
- We lost lock again while trying to cal measurements in the NEW_DARM state. Still trying to figure out what's happening there; clearly reducing the amplitudes by 50% (LHO:74883) wasn't nearly enough. It seems we kicked the IFO hard enough for ISI_ETMX watchdogs to trip.

due to the lockloss we did not get to test the DARM_RECOVER guardian state.

Summary of strain coherencce with all channels: https://ldas-jobs.ligo-wa.caltech.edu/~gabriele.vajente/bruco_1386992886_STRAIN/ 

It looks like there is no coherence with MICH nor SRCL

At low frequency, the usual coherence with DHARD_Y

The only noticeable difference now with respect to hot OM2 is the incresed coherence with beam jitter

TITLE: 12/18 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Austin
SHIFT SUMMARY:

IFO is LOCKING and at CARM_TO_ANALOG

16:09 UTC - Lockloss from NLN - alog 74864

16:48 UTC - Lockloss at Max_Power - due to 6.0 EQ in China

1. Staying down after 3rd ALS/IR lockloss due to EQ microseism - it doesn’t look terrible so will try again in 15 minutes.
2. Resumed at 17:14 UTC

17:00(ish) UTC - Power cycled Nuc35 due to the screen crashing and not responding.

17:12 UTC - Fire alarm panel went off. Tyler called a few seconds after saying it was a false alarm/to be expected due to work he was doing. Alarm stopped shortly after

18:00 UTC Reached OMC_Whitening but violins are quite high - NLN reached at 19:07 UTC. It took 1 hour and 7 mins for violins to be low enough to go through whitening (Tagging SUS).

19:07 UTC - Reached OBSERVING

22:01 UTC - COMMISSIONING - went into planned commissioning. Expected to continue until around 00:00 UTC.

22:35 - 22:39 UTC - Took a broadband calibration measurement. (screenshot)

23:46 UTC - Lockloss during commissioning during Simulines calibration - alog 74874

Other: MX/Woodshop Access Door Issue - people with access can’t get in. Fil is working on it and seems to have fixed it.

LOG:

As Ryan did last week 74741. I have taken PEM_MAG_INJ and SUS_CHARGE from WAITING to DOWN so that they do not run tomorrow. Instead, tomorrow Louis will try the risky DARM loop swaps and calibration. To re-enable the automated measurements, the nodes should be requested to INJECTIONS_COMPLETE before next Tuesday.