TITLE: 06/01 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
SHIFT SUMMARY:

1. Lockloss at 12:44 UTC due to EX ISI movement it seems. The PUM, ISI Stage 1 and 2 WDs all tripped about 10 seconds prior 
   1. ISI WDs untripped
   2. PUM WD is not resetting and is showing a big “Rocker Switch Death” sign
2. Lock (Attempt)  2 - “ROCKER SWITCH” Edition: 
   1. Put IFO in down until the Rocker Switch issue is fixed.
   2. Texted LHO Ctrl Rm and Jenne came on and helped get the ISI WDs untripped and discuss the not-so-normal rocker switch issue.
   3. There were no earthquakes, wind or other potential explanations for both the ISI and the PUM WD trip - Per Jenne’s discussion, this suggests an electronics/hardware issue:
      1. No Signals from OSEMs
      2. Big red ROCKER SWITCH DEATH seems to be an “and” condition of the OSEM inputs not working (per Jenne’s help finding the right alog)
      3. PUM Coil WD RESET button not working
      4. Wiki’s helpfulness was limited 
   4. Richard McCarthy is going to EX to investigate the coil drivers.
      1. As Richard is investigating at EX, I see the Rocker Switch Flickering much more frequently.
      2. Secondary ISI WD trip due to Richard arriving at EX - will keep tripped until investigation is over.
   5. Upon Richard checking, it seems this was a bad/blown coil driver in the EX PUM. It will need replacement. Putting ops mode in maintenance.
      1. Richard reports a 15V rail

EX PUM Coil Driver Failure Story Summary and Findings

See above for more detailed LOG of events.

At 12:44 UTC the ISI and PUM Watch Dogs tripped seemingly out-of-nowhere.

From the above details, we can safely conclude that the PUM Coil Driver failure is what caused both ISI and PUM watchdogs to trip. It also explains why the PUM WD button “doesn’t work” - it’s that the WD is just being re-tripped as the driver continues to fail and flail. Richard called Fil and he’s on the way to do the needed maintenance. IFO is in down until and OPS mode is in “corrective maintenance” until the issue is resolved.

LOG: