Reports until 01:59, Friday 07 December 2018
H1 ISC (ISC)
craig.cahillane@LIGO.ORG - posted 01:59, Friday 07 December 2018 - last comment - 06:08, Friday 07 December 2018(45751)
The Return of the Fast Lockloss
Hang, Dan, Craig

While powering up from 20 W to 30 W tonight, we've been losing lock after about 10 minutes.  It seems to coincide with a certain thermal state of the interferometer (i.e. when POP18 hits around ~50 cts, we always lose lock), see an upcoming alog from Dan.
In one lockloss, we saw a ~17 Hz oscillation ring up in all the LSC error signals.  
In another two, we saw the DARM error signal float away to infinity, causing saturations in all LSC PDs/error signals.

I took a quick DARM OLGTF while in ADJUST_POWER with 20 watts input.  Our problem could be the changed DARM phase from when the reference in this plot was taken, which was apparently September 12, 2018.  Jeff K seems to understand our DARM OLG to better than 5 percent, and our measured DARM OLGs match his.

Our phase crossover is now at 17.25 Hz, which is super close to the L2 actuation stage calibration line.  I've turned off the L2/L3 calibration lines for now just to see if this makes a difference in the next lock, probably isn't an issue.
Images attached to this report
Comments related to this report
craig.cahillane@LIGO.ORG - 02:22, Friday 07 December 2018 (45752)
During one of our locklosses at 1228209616, the fast shutter did not close.

This could have been because the LOCKLOSS_SHUTTER_CHECK guardian was in the SHUTTER_FAIL state, but it was most likely because of the second point.
Dan had a look and saw that the OMC DCPDs only saw a max of 26 mA during the lockloss, and this was not enough to trigger the fast shutter. 
Plot of the OMC_DCPD_SUM during the lockloss in question attached.

We should fix the LOCKLOSS_SHUTTER_CHECK guardian so that we don't go to high power when in the SHUTTER_FAIL state.
Images attached to this comment
jenne.driggers@LIGO.ORG - 06:08, Friday 07 December 2018 (45754)

I think that the shutter did close. In the attached, I have the same lockloss time, and I see that the AS_A sum (which is after the shutter) drops immediately, as I would expect.

Also, the fast shutter itself is independent of the guardian state.  The guardian should, however, prevent us from going past DRMI until either the auto-check passed or a person confirms that it is okay.  I'll look to ensure that this is still coded in there properly.

Images attached to this comment