When Ibrahim relocked the IFO, the fast shutter guardian was stuck in the state "Check Shutter"
This was because it apparently got hung up getting the data using cdsutils getdata.
The first two attachments show a time when the shutter triggered and shows up in the HAM6 GS13s, and the guardian passes. The next time shows our most recent high power lockloss, where the shutter also triggered and shows up at a similar level in the GS13s, but the guardian doesn't move on.
The guardian log screenshot shows both of these times, it seems that it was still waiting for data and so the test neither passed nor failed.
To get around this and go to observing, we manualed to HIGH_ARM_POWER.
Vicky points out that TJ has sovled this problem for other guardians using his timeout utils, this guardian may need that added.
Another thing to do is make things so that we would notice that the test hasn't passed before we power up.
I looked at nds1's logs for this data request, the request appears to come it at the time it had timed out on h1guardian1.
From Shela's guardlog (paraphrasing somewhat):
2024-10-18_19:36:22Z timer
which is
2024-10-18T12:36:22 in "log PDT" format
NDS1 logs show (h1guardian1 is 10.101.0.249):
2024-10-18T12:36:27-07:00 h1daqnds1.cds.ligo-wa.caltech.edu daqd[1267959]: [Fri Oct 18 12:36:27 2024] connection on port 38943 from 10.101.0.249; fd=75
2024-10-18T12:36:27-07:00 h1daqnds1.cds.ligo-wa.caltech.edu daqd[1267959]: [Fri Oct 18 12:36:27 2024] ->42: version
2024-10-18T12:36:27-07:00 h1daqnds1.cds.ligo-wa.caltech.edu daqd[1267959]: [Fri Oct 18 12:36:27 2024] ->42: revision
2024-10-18T12:36:27-07:00 h1daqnds1.cds.ligo-wa.caltech.edu daqd[1267959]: [Fri Oct 18 12:36:27 2024] ->42: status channels 3 {"H1:ISI-HAM6_BLND_GS13Z_IN1_DQ"}
2024-10-18T12:36:27-07:00 h1daqnds1.cds.ligo-wa.caltech.edu daqd[1267959]: [Fri Oct 18 12:36:27 2024] ->42: status channels 3 {"H1:SYS-MOTION_C_SHUTTER_G_TRIGGER_VOLTS"}
So on first look it appears nds1 didn't get the request until after the 5 second timeout had expired.
Here is the request line (line 53) of isc/h1/guardian/LOCKLOSS_SHUTTER_CHECK.py
gs13data = cdu.getdata(['H1:ISI-HAM6_BLND_GS13Z_IN1_DQ','H1:SYS-MOTION_C_SHUTTER_G_TRIGGER_VOLTS'],12,self.timenow-10)
This request is for 12 seconds of data with a start time 10 seconds in the past, meaning it cannot complete until +2 seconds have elapsed. Does this mean the 5 second timeout is effectively a 3 second timeout?