TITLE: 07/13 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
SHIFT SUMMARY:  H1 has been down since the lockloss at 16:44 UTC earlier today.

Since midshift, Sheila was working with me on diagnosing locking issues. After locking DRMI, we turned on the DRMI ASC loops one-by-one until we found that turning on the SRC1_P&Y loops caused the buildups and camera image on AS AIR to get worse, so we've commented out using those loops in the ISC_DRMI Guardian. Locking DRMI after that and running DRMI ASC all at once made DRMI look much better and we didn't see the oscillations and saturations that I was seeing earlier. We also commented out the use of SRC1 in ENGAGE_AS_FOR_FULL_IFO since Sheila and Keita saw earlier in the day that it was not minimizing POP90 well at that point.

Once we made it past DRMI, we then encountered our next issue at CHECK_AS_SHUTTERS. The FAST_SHUTTER Guardian ran through its shutter test, then jumped to SHUTTER_FAILURE where it reported "Fast shutter failed tests! Do not power up!" and the AS port protection screen shows the error messages "Protection in fault" and "Power interlock is on". Unsure of what these errors meant, and having been unsuccessful in running the fast shutter test again manually, I called Fil for support. He had me power cycle the fast shutter driver chassis in the ISC rack by HAM6, which showed gibberish on the LCD screen (a previously seen issue), and the screen looked better and showed 255V DC charge. This didn't seem to fix the issue of the test failing, so I also power cycled the shutter logic chassis at the bottom of the rack, but this also didn't solve it.

Later, Keita discovered that the shutter test was failing because when the shutter closes, the signal on ASC-AS_B_DC_NSUM was not low enough, so Guardian didn't think it was closed even though it was. By just pressing the open and close buttons manually on the fast shutter screen, the shutter was able to run through its test and the Beckhoff errors cleared. Nothing was fundamentally changed here to fix this, aside from adding some log statements to the FAST_SHUTTER Guardian, but at least it sounds less like an electronics issue at this point. Keita commented that this perhaps isn't an alignment issue, unless some alignment in HAM6 changed recently.

Even though the issue with the fast shutter seems to be dealt with (but untested during a lock acquisition), there are still IFO locking issues that Sheila and I were unable to fully address this evening due to the fast shutter issues, including the 0.5Hz oscillation and lockloss when going to full power and the strange image on the AS AIR camera, which will need to be addressed in the morning. I'm leaving H1 DOWN for the night until more thorough investigations can be done.