Sheila, Evan, Carl Terra
At the end of the night we had several locklosses when trying to engage the soft loops. It seems like the problem could be turning on the 10 dB more offloading gain in the top mass, which we have been doing without a problem for several weeks now. In the last lockloss like this, L2 rms watchdogs tripped on all 4 test masses (not all coils). Evan manually reset the Y arm optics, but we couldn't reset the X arm optics. For ETMX, Terra and I went to the end station and tried power cycling the coil driver, but this didn't allow us to reset the watchdogs. Carl and Evan power cycled the ITMY coil driver, and were able to reset the RMS WDs but the software WD cannot be reset.
Also, we have had several EQs tonight (both large and small) and tried out turning sensor correction on and off.
Here is a time series of L2 master outs durring the lockloss where the RMS WDs tripped. This lockloss happened durring a 15 second sleep in the guardian, which is why down was not run for so long after the lock was lost. We've eliminated this particular long sleep but there are other places where we have similar sleeps that need to be fixed. We went through the guardian states up to DC_READOUT this afternoon, rewritting in places that have long sleeps. We've tested this up to ENGAGE_SOFT_LOOPS.
The version of ISC_LOCK committed just before starting these edits is 13570, the version after is 13576.