IFO is LOCKING but losing lock at TRANSITION_FROM_ETMX (probably due to high wind/microseism)
MX VEA Temperature Investigation:
- Alarm shows that H0:MX_VEA_202D_DEGF temperature has gone too low, triggering an alarm
- Further investigation by Dave and I, trending all 4 sensors shows that two of the thermometers increased in temperature, while the other 2 decreased (one of these gave the alarm)
- The fluctuation of the temperature seems to be erratic and only started 18 hours ago, around the same time the wind picked up
- Further trending with the MX wind sensor shows that the fluctuations followed the erratic wind pattern
- This indicates that there is a draft making its way through the room with the 4 sensors, increasing the temp on one side and decreasing it on the other side.
- This may mean that a door is open and needs to be closed.
- The wind is dying down and the alarm is no longer active, but this should be physically investigated.
- The wind went back up past 25mph and then the alarm came on again - this is further evidence for the “something is open/ajar” guess.
- Screenshot below
GraceDB Query Failures
- GraceDB Query Failures flashing 5 or 6 times since the start of the shift
- Checked in with LLO and they are experiencing the same thing, implying that this is a server problem and not an H1 problem.
Transition from EX Failures - The Lockloss causing state (alog 75215)
- After losing lock at EX, managed to make it back to the same place (alog 75214)
- Jenne came online to investigate what would be causing these locklosses
- We are pausing at LOWNOISE_ASC (18:37 UTC) until we have an idea of what’s happening. We were able to stay here for a while.
- Another lockloss happened when Jenne was trying to troubleshoot the faulty guardian state. We lost lock the moment we tried running the EX transition part of the code (alog 75218)
- Still troubleshooting but not coming up with any reason other than this state being more susceptible to high microseism and wind.
- Jenne’s Idea: Perhaps the Low Noise coil driver is putting us in a more delicate state so what if we skip it temporarily? We tried this and it caused another Lockloss (alog 75222)
- I will attempt to get through this state again but if the issue keeps happening, I will put H1 in DOWN.
- This is because the wind is forecasted to go up even more until the evening when it is supposed to die down (by EVE shift)
Minor:
- Nuc24 Reboot by Dave (Was working in the control room but not externally)
- Nuc33 FCIR Trans B Cameras Bluescreened - Collectively agreed (Jenne, Dave, Ibrahim) that they can be fixed at a later date (not high priority)
- OpenEye Crashed and had to be restarted