Displaying report 1-1 of 1.
Reports until 16:00, Tuesday 20 January 2015
H1 ISC (IOO, SEI, SUS, SYS)
jeffrey.kissel@LIGO.ORG - posted 16:00, Tuesday 20 January 2015 (16165)
Maintenance Day Recovery
J. Kissel, K. Izumi, A. Staley, S. Dwyer

ETMX, ETMY, and HAMs 2 and 3 were involved in maintenance day, and recovery was little-to-no different than any other Tuesday. Chasing alignment in the IMC and ARMs have been the most time consuming parts, though my impression is that it has been a little bit shorter / better today and the SUS are not nearly as far off as they have been. Indeed, in order to hard-reboot the SEI HAM23 front end and IO chassis, we needed to take down the IMC, all its associated suspensions (and the IM and PR suspensions), and the HAM 2-3 ISIs and HEPIs -- and this DID NOT shake the MC REFL path enough to cause any misalignment in pitch as we've seen in the past (see Integration Issue 854. Rather than going through my usual detail play-by-play, I've decided that its more productive to list what we should change as a result of today's experiences in order to keep getting better at the recovery.

Lessons (at least *I*) learned from today, in no particular order:
- We need to have guardian transition the BSC ISI Stage 2 Gains from HI to LO when a SEI manager requests to go from FULLY_ISOLATED to OFFLINE for ALL BSCs, because requesting to go from WATCHDOG TRIPPED (afer computer reboot) to OFFLINE to FULLY_ISOLATED after does *not* transition the gains to LO before getting started.
- We need to have the HAM ISI guardians *ensure* the GS13 gains are HI.
- We need to have an IMC "down" state that is actually DOWN, (or perhaps call it "OFFLINE") -- specifically the IMC WFS are turned off so no angular control is fed to the MC suspensions, and (less importantly) no longitudinal control is fed to MC2.
- The new(ish) IMC ASC DOF4 / MC WFS center servo really can pull the MC REFL pointing off into the weeds, so these should be the first thing turned off when ramping down the IMC.
- The transition from ISOLATED to OFFLINE on the HAM's watchdog trips the ISI halfway through the turn off process more often than not, and its likely because the GS13 gains remain in high-gain mode indefinitely.
- We need a guardian to manage the HSSS (today it was the IMs, but another day it was the RMs, etc.).
- We need to calculated / measure / install / save alignment offset calibrations for the HSSS. 
- We need a standardized, well-ordered check-list procedure for taking chambers / parts of the offline -- and then bringing them BACK online.
- We need to either resurrect / commission / trust the SUS DRIFT_MONITOR, or find another tool that we can quickly assess how all SUS alignments have changed from prior to the start of a maintenance day to after finishing.
- We need a tool that takes the output of the above (i.e. the assessment of the alignment), and *fixes* any SUS that are out of alignment.
- When the global "initial" alignment changes -- even by a few microradians here or there -- the optical gain of the IFO changes, which means we need to re-adjust ISC parameters.
- We need to include turning ON and OFF the ESD Bias into the SUS ETM guardians, such that it is ON in every state but SAFE. Further, we should pick a standard location to turn ON and OFF the ESD BIAS path.

We're not sure if we're *fully* recovered at this point, but we have gotten as far as a short DRMI locks with ARMs off-resonance, and we push onward.

Regarding the Guardian Logs: 
SEI_ETMX Manager Guardian was requested to go to OFFLINE from FULLY_ISOLATED between
2015-01-20 18:15:24.753 UTC
2015-01-20 18:16:43.581 UTC 
Non-image files attached to this report
Displaying report 1-1 of 1.