Lockloss @ 00:37 UTC - no obvious cause. Lockloss tool analysis failed, but I don't see anything abnormal on those plots.

LSC and ASC trends attached.

TITLE: 09/28 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 151Mpc
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 29mph Gusts, 25mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.21 μm/s
QUICK SUMMARY: H1 has been locked for over 18 hours.

• All systems look good, wind is elevated but not extreme

TITLE: 09/28 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY: Much more productive day, we stayed locked the whole shift and ran a calibration suite. Locktime of 18:08 as of 23:00UTC

• While LLO was down from an EQ I took a calibration measurement, we dropped out of observing from 19:29UTC to 19:59UTC for it.
• 21:58UTC Superevent S230928cb, again no verbal alarm for this event.
• We survived several earthquakes today

LOG:

I ran a calibration suite today,

/ligo/groups/cal/H1/measurements/PCALY2DARM_BB/PCALY2DARM_BB_20230928T193035Z.xml

Simulines:
GPS start: 1379964982.031822

GPS stop: 1379966313.087435

2023-09-28 19:58:14,885 | INFO | File written out to: /ligo/groups/cal/H1/measurements/DARMOLG_SS/DARMOLG_SS_20230928T193609Z.hdf5
2023-09-28 19:58:14,928 | INFO | File written out to: /ligo/groups/cal/H1/measurements/PCALY2DARM_SS/PCALY2DARM_SS_20230928T193609Z.hdf5
2023-09-28 19:58:14,946 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L1_SS/SUSETMX_L1_SS_20230928T193609Z.hdf5
2023-09-28 19:58:14,963 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L2_SS/SUSETMX_L2_SS_20230928T193609Z.hdf5
2023-09-28 19:58:14,982 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L3_SS/SUSETMX_L3_SS_20230928T193609Z.hdf5

 

Full report available here.

Summary:

• Daily observing duty cycle: 87%, 32.3%, 66.2%, 43.8%, 71.3%, 73.9%, 91.9%. Week average: 66.6%
• Observed noise lines in h(t) occurring intermittently, particularly at 15, 29, and 32 Hz, throughout the week
• Lock-losses mostly caused by hardware/power issues, earthquakes, in some cases it looked like an earthquake caused a lock-loss but I could not find an obvious candidate in USGS
• High microseism throughout the beginning of the week, range floated in the 140-145 Mpc range from Mon-Wed, began to improve once microseism had calmed down in the latter half of the week
  • Mon had both high microseism and high winds, however the detector managed to ride out most of it
• Not much observing time on Tues due to maintenance
• On Thurs the detector lost observing time due to a hardware problem + alignment difficulties when attempting re-lock
• On Fri we noticed a regurly occurring ground motion in the corner station, which was picked up by multiple sensors and was weakly coupling to strain, as reported in this alog
  • Feature had appeared before, but not with the same frequency/regularity
  • Persisted throughout Fri/Sat, on Sun it changed and is now occurring less often, and more sporadically
  • Similar feature sometimes seen at LLO
  • DetChar suggested to follow-up by upconverting microphone data to audible range to help identify a cause
• Better duty cycle and range during the weekend, Sun in particular was very quiet (almost to 160 Mpc on Sun)
• Re-appearance of comb associated with OM2 heater (alog), later seemed to change frequency spacing
• Few PyCBC triggers with chi2-weighted SNR above 10
• H1:SUS-ETMX_L3_OPLEV_PIT_OUT_DQ and H1:ASC-DHARD_Y_OUT_DQ were frequently in the top two Hveto round winners for the week

We've been locked for 14hours, we've rode through a few earthquakes this morning. The airhandler alarm for MX has lost connection. The wind is also a lot more managable this morning/afternoon.

Thu Sep 28 10:22:26 2023 INFO: Fill completed in 22min 21secs

Gerardo confirmed a good fill curbside

TITLE: 09/28 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 156Mpc
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 3mph Gusts, 2mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.24 μm/s
QUICK SUMMARY:

• MX temperature alarm, low temp
• CDS/VAC/DUST/SUS looks good, wind has calmed way down since yesterday
• Incoming 6.0 EQ from Vanuatu as I'm posting this
• We've been locked for 10:16, in observing since 05:04UTC

TITLE: 09/28 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 144Mpc
INCOMING OPERATOR: Austin
SHIFT SUMMARY: Started the shift waiting for strong winds to calm down, but was able to run an alignment and lock easily. One quick lockloss midway through, but was able to relock and observe shortly after.

• Waiting for high winds to calm before locking
• 01:20 Starting alignment
• 01:55 Alignment complete, starting lock acquisition
• 02:44 Reached NLN, fully automated acquisition
• 03:02 Started observing
• 03:49 Lockloss - alog
• 04:49 Reached NLN, fully automated acquisition
• 05:05 Started observing

LOG:

No log for this shift.

Lockloss @ 03:49 UTC - no obvious cause as of yet.

After waiting for high winds to calm down, H1 has just locked and started observing as of 03:02 UTC.

FAMIS 26059

There is no measurement this week for ETMX as the IFO lost lock before it was finished.

The V_eff values for ETMY have quickly changed direction since the last analysis and are now trending away from zero. All other test masses look okay.

This afternoon, the ITMX ISI tripped, was continuously glitching and couldn't be recovered. Even when I put the ISI and HEPI in non-actuating states the ISI continued to get glitches that almost saturated the seismometers. Fil went out to the rack and while I watched the ISI he turned off the coil drivers one at a time. It wasn't until he turned off the third coil driver that the glitching stopped and the ISI started behaving normally. This is exactly the same as the event in July, where I found the St2 H3 sensor seemed to be mis-behaving. Looking at the current and voltage mons, it seems the St2 H3 coil was again misbehaving, attached trend shows the st2 coil voltage and current mons. The only outliers are the St2 H3 channels, this window should be during a time when the ISI masterswitch was off, the drives should be ~0.

We should pull the corner 3 coil drive at the next opportunity. I think we have a spare in the EE shop.

TITLE: 09/27 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Wind
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 27mph Gusts, 16mph 5min avg
    Primary useism: 0.08 μm/s
    Secondary useism: 0.34 μm/s
QUICK SUMMARY:

• ITMX ISI issues appear to have been fixed
• H1 still down waiting out high winds (forecast says likely won't calm down for another couple of hours at least)

Oli, Camilla

ISC_LOCK takes longer to recognize and move to the LOCKLOSS state when a lockloss occurs than we want it to, sometimes leading to confusion regarding lockloss causes. Example: 71659

The part of ISC_LOCK that determines whether we have lost lock is in the is_locked function in ISC_library.py(attachment1). The current parameters for determining a lockloss rely on the H1:LSC-TR_X_NORM_INMON channel - once that channel's value falls below 500, ISC_LOCK will recognize that we have lost lock and will take us to the LOCKLOSS state.

Currently, when we are locked LSC-TR_X_NORM_INMON sits right around 1540, and when a lockloss happens, it takes ~0.55-0.65s after the lockloss start (based on ASC-AS_A_DC_NSUM_OUT) to drop below 500, and then takes another ~0.07-0.15s for ISC_LOCK to register this and take us to LOCKLOSS. To minimize this latency, since LSC-TR_X_NORM_INMON never gets below 1450 once we reach NOMINAL_LOW_NOISE, Camilla and I think it would be useful for the LSC-TR_X_NORM_INMON is_locked threshold value to be raised significantly to allow ISC_LOCK to register locklosses faster than it is currently. By raising the threshold to 1300 for example(attachment2), the time between the lockloss start and ISC_LOCK changing states would be reduced by a factor of 2-3.