Displaying report 1-1 of 1.
Reports until 22:47, Tuesday 04 August 2015
H1 ISC (CAL, CDS, IOO, ISC, SEI, SUS)
jeffrey.kissel@LIGO.ORG - posted 22:47, Tuesday 04 August 2015 (20241)
IFO Maintenance Recovery Complete at 05:20 UTC (22:17 PDT)
J. Kissel, K. Izumi, P. Thomas, B. Weaver, for the whole team.

It was another rough Tuesday. Thanks all for your help and patience.

The biggest issues (time delays in recovery) were (all times PDT):
09:00 - 10:27 (~1.5 hrs) Debugging the Guardian code core before the blanket upgrade LHO aLOG 20210
12:26 - 13:10 (~0.5 hrs) Finding that HEPI foton files all needed a "foton -c" cleansing from bad auto-quacking over the years LHO aLOG 20212
12:50 - 13:25 (~0.5 hrs) Finding that ETMX ESD driver had been railed negative since the night before LHO aLOG 20203
14:15 - 15:40 (~1.5 hrs) End-station Beckhoff finally crashing killing ETMX ESD driver *again* during recovery and it going unnoticed LHO aLOG 20219
16:10 - 16:40 (~0.5 hrs) LSC_CONFIG guardian becomes undead, is killed, which requires a DAQ restart, which fails miserably LHO aLOG 20228
18:05 - 19:15 (~1.5 hrs) PSL rotation stage roulette, which eventually requires restarting the corner station Beckhoff slow-controls chassis LHO aLOG 20235
19:34 - 21:40 (~2.0 hrs) OMC_LOCK guardian has logic in it that needs two channels specific data rate, and those channel's data rate was changed today during the OMC / LSC science frame channel cleanup LHO aLOG 20237

I attach a picture of the full-whiteboard, recovery diary. 
Non-image files attached to this report
Displaying report 1-1 of 1.