Lockloss at 06/17 01:37 UTC - looks like it may have been due to a jump in the wind speed?

Link to report.

• The duty factor for the week was 15.43%, much lower than previous weeks. Locking problems persisted for the second half of the week (see, e.g., alog 78308). It was better in the first couple of days, at 51.91% and 38.43%, but dropped substantially thereafter. No observing data was taken from Friday through Sunday.
• The week started with a wind advisory in the region with wind gusts in excess of 25mph some times. Winds were partially responsible for lockloss and caused difficulties in maintaining lock. The wind speed was around 10 m/s or less on June 2 (UTC) but increased to as much as 17.5 m/s around 15:45hrs UTC on June 3. June 4 also remained windy; while it ebbed in the night hours, it climbed back up in the day hours to as high as 18.5 m/s by 18:00 UTC. The wind finally slowed down to below 10m/s around 4:00 UTC on June 5 and did not rise above that for the rest of the week.
• June 5 onward the duty factor remained below 10%, while several commissioning / troubleshooting exercises attempted to get the detector to maintain lock / relock.
• "Whistle" glitches in SQG FC WFC were associated with range drops in a couple of instances, as has been observed in May too -- in this alog 78263. A drop in the relative vacuum in H1:SQZ DC PhotoDiode sum/null ratio BLRMS was also found to have some correlation with range drops.
• Low-frequency (10-50Hz) noise in the IMC spectrogram appears to be correlated with high SNR glitches in Omicron (especially, on June 3).
• Hveto: 12 channels appeared in Hveto this week of which 8 were not seen in the previous week.
• FScan: Plots from June 3 seem to be missing. All plots have appeared for June 4 – 6, for which the line counts are 656, 267 and 291, respectively.

TITLE: 06/16 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Oli
SHIFT SUMMARY:

IFO was in NLN and OBSERVING as of 06:05 UTC (21hr 37 min lock) but is NOW in CORRECTIVE_MAINTENANCE while we briefly restart the ADS Camera.

Like 5 minutes before my shift ended, the ADS Pitch1 Inmon and Yaw1 Inmon are stuck. It seems that they keep trying to turn on, but can't get past TURN_ON_CAMERA_FIXED_OFFSET. This has happened before alog 77499 and it is likely that the cameras just need to be restarted. We did not lose lock. Oli (incoming Op) has called Dave and they are working on it.
 

LOG:

None

TITLE: 06/16 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 13mph Gusts, 11mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.07 μm/s
QUICK SUMMARY:

Observing and locked for 21.5 hours.

IFO is in NLN and OBSERVING (Now 18hr 35 min lock!)

• LDAS seems to have come back online in the last few hours.
• We survived a 6.0 from Peru

Nothing else of note.

Sun Jun 16 10:10:34 2024 INFO: Fill completed in 10min 30secs

Note TCs did not reach -200C because lower outside temps this morning (15C, 59F).

TITLE: 06/16 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 156Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 9mph Gusts, 4mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.09 μm/s
QUICK SUMMARY:

IFO is in NLN and OBSERVING since 06:06 UTC (12hr 45 min lock)

NUC27 Glitch screen is giving a warning: "Cluster is down, glitchgram is not updated", which I haven't seen before.

TITLE: 06/16 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Observing at 158Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY:  We are Observing at 158 Mpc and have been locked for 6 hours now. The only issues I had today was the OPO ISS maxing out at 54uW and so not being able to catch at 80uW and so needing to adjust the setpoint, and then relocking after the 06/16 00:01 UTC lockloss , I had some issues with getting PRMI and DRMI to lock, but it makes sense since the wind was still a bit high at the time. After that the night has been quiet.
LOG:

23:00 Detector relocking and at DARM_TO_RF

23:39 NOMINAL_LOW_NOISE
- SQZ OPO ISS pump having trouble locking. The OPO transmission couldn't go higher than 54.6uW
    - Adjusted OPO temp, but the current temp was the best so I put it back. Reloaded the OPO guardian (So I had changed nothing!). OPO was able to get up to 72 after this, but still not to 80.
    - Naoki came on teamspeak and lowered the threshold to 70, and it caught very soon after that.
00:01 While dealing with the OPO, we lost lock. We had been locked for 22 minutes

00:29 Lockloss from ACQUIRE_DRMI
00:30 Started an initial alignment
00:51 Initial alignment done, relocking
01:04 Lockloss from ACQUIRE_DRMI
01:52 NOMINAL_LOW_NOISE
01:56 Observing
02:01 Our range was low so I took us out of Observing and ran the sqz tuning guardian states
02:10 Back to Observing, with a 7Mpc increase in range

05:25 Left Observing and started calibration measurements
05:53 Calibration measurements done, running sqz alignment (new optic offsets accepted)

06:05 Back into Observing

Calibration was run between 06/16 05:25 and 05:53 UTC

Calibration monitor screenshot

Broadband (2024/06/16 05:25 - 05:30 UTC)

File: /ligo/groups/cal/H1/measurements/PCALY2DARM_BB/PCALY2DARM_BB_20240616T052532Z.xml

Simulines (2024/06/16 05:32 - 05:53 UTC)

Files:

/ligo/groups/cal/H1/measurements/DARMOLG_SS/DARMOLG_SS_20240616T053211Z.hdf5
/ligo/groups/cal/H1/measurements/PCALY2DARM_SS/PCALY2DARM_SS_20240616T053211Z.hdf5
/ligo/groups/cal/H1/measurements/SUSETMX_L1_SS/SUSETMX_L1_SS_20240616T053211Z.hdf5
/ligo/groups/cal/H1/measurements/SUSETMX_L2_SS/SUSETMX_L2_SS_20240616T053211Z.hdf5
/ligo/groups/cal/H1/measurements/SUSETMX_L3_SS/SUSETMX_L3_SS_20240616T053211Z.hdf5

 

06/16 05:24 UTC I took us out of Observing to run a calibration sweep that we weren't able to run earlier.

Currently Observing at 152 Mpc and have been Locked for 55 mins. We had a lockloss at 06/16 00:01 UTC, 22 minutes into NOMINAL_LOW_NOISE, and relocking required me helping with PRMI and DRMI, but we were able to get up eventually. The wind is slowly going down and is around 25mph now.

Lockloss @ 06/16 00:01UTC (LDAS currently down so no link). We had only been locked for 20 minutes and I was trying to get the SQZ OPO to lock, so we had not been in Observing. There was a small jump up in wind at that time, but not sure if that would have caused the LL

TITLE: 06/15 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Wind
INCOMING OPERATOR: Oli
SHIFT SUMMARY:

IFO is in and is locking at LOWNOISE_COIL_DRIVERS

Wind gusts have been high since the last lockloss and per usual, I would put H1 in down when the wind would go higher than 30mph and wait for oppurtunistic low-wind periods to lock ALS - here's a timeline since the last LL. The only sdf revert I had to accept was in my midshift alog 78457

All times are UTC

19:31 - NLN/Observing Lockloss

19:38 - ALS Lockloss

19:39 - DOWN due to 40-50mph winds

21:43 - Attempt to re-lock due to wind speeds below 30mph

22:08 - Two failures at MICH_FRINGES and no ability to lock PRMI let alone DRMI - decied to run initial alignment

22:25 - Initial Alignment done fully auto, attempt to relock - winds picking up about to surpass 30mph

22:41 - DOWN again after 2 ALS locklosses and 3 IR Locklosses, with winds around 35mph.

22:54 - Saw small oppurtunistic sub 30mph calmness (that only lasted 10 mins) and decided to try locking again - it worked!

23:02 - DRMI locked!

23:30 - Shift ended while in LOWNOISE_COIL_DRIVERS

LOG:

None

TITLE: 06/15 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Wind
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 28mph Gusts, 21mph 5min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.08 μm/s
QUICK SUMMARY: Relocking and at DARM_TO_RF

IFO is in NLN and OBSERVING as of 18:29 UTC

We lost lock 2 hours after reaching observing at 16:56 UTC due to unknown reasons. The range was definitely lower than usual and while I was attempting to investigate this using Sheila and TJ's new wiki low range checks page, we had our initial lockloss. SDF Table of acceptances is attached.

After PRMI failed, I ran an initial alignment and then we locked swiftly despite winds in the high 20's and low 30's (mph). The range is still quite low now but I will wait the recommended hour or two (per Sheila's instruction) until IFO is thermalized enough to run the low range check dtt.

Wind forecast show gusts are expected to continue picking up until around 4PM so here's hoping we don't need to lock ALS during that time!

In the observing stretch that started Monday around 8 pm, there was a drop in optical gain (by 4%), and power at AS port PDs (4% drop in AS_A sum, AS_B sum, AS_C sum, and OMC REFL).  There not at that time any change in circulating power in the arms, PRG, coupled cavity pole, or alignment of suspensions (shown are SR2, SRM, OM1+2, but we have also looked at many other suspension, ISI and HEPI related channels for this time 77382 ).  This time is shown by the first vertical cursor in the attachment. 

We were unable to relock the interferometer after that Monday evening lock (a new lockloss type appeared early Tuesday morning, 77359), and after the maintence window we were uable to lock with difficulties powering up (77363) and an AS camera imagine that had lobes whenever the beam was well centered on AS_C.  We locked that evening by adding large offsets to AS_C's set point, (77368).  This is the cause of the large alignment shifts seen in the screenshot on Tuesday evening to Wed morning, which allowed us to lock the IFO but created a large scatter shelf and resulted in a lower coupled cavity pole, and lower optical gain, and we did not have much squeezing that night. 

We do not think that Tuesday maintence activities are the cause of our problems, although some of the Tuesday work has been reverted (77350 77369).

Yesterday we spent much of the day with SRY locked, single bounce or using the squeezer beam reflected off SRM to investigate our AS port alignment.  77392, We are able to recover the same throughput of a single bounce beam to HAM6 by moving the alignment of SR2 + SR3 by a huge amount 77388, which also produced a round looking beam on the AS camera.  We haven't since tried to explore this aperture, to see if we could have also recovered this thoughput with a pitch move, for example.  We also saw that the squeezer beam does not arrive in HAM6 with a good transmission when injected with the alignment used previously, but that we could recover good transmission to HAM6 by moving ZM5 by 150-200 urad, in pitch or yaw.   With this shift in SRC axis, and squeezer alignment we were able to relock and not have the large scattering issues we had Tuesday night.

Today's time was spent recovering from a seemingly unrelated problem in the SQZ racks, and some commissoning aimed at recovering our previous (165Mpc) sensitivity with this new alignment.  This will continue during tomorow's commissoning time.

additional related information:

• (no SQZ, HWS, OFI voltmons)  77374
• Move of SQZ beam in lock to get squeezing: 77400
• OFI transfer functions: 77391
• other suspension health checks: 77383
• dark offset script: 77362
• SRM and SR2 beam positions after the shift: 77443
• Further look at powers on HAM6 PDs: 77441
• AS port fringe wrapping worse than Summer 2023: 77452