Plot of our 51 locklosses from the last 20 hours from Dan.

The lockloss tool stopped working 4 hours ago.

Problems today, in addition to 45998:

NOMINAL_LOW_NOISE has 8 locklosses, but it's not clear why we are so unstable here.  I mention blaming ASC last night, so I wrote a cross-correlation tool to look at which LSC/ASC error signal fluctuations were most correlated with the dips in PR Gain, POPAIR 18 and POPAIR 90.  It seemed that SRC1 and PRC1 were the most correlated with POPAIR, along with CSOFT, so we opened the SRC loops and found offsets for SRC1 and SRC2 which minimized POPAIR 90 fluctuations.  We were in the process of walking PRC1 and PRC2 yaw when we lost lock in CHARD_BLEND.  

H1:ASC-SRC1_P_OFFSET           -0.4054
H1:ASC-SRC1_Y_OFFSET           -0.0767
H1:ASC-SRC2_P_OFFSET           0.1109
H1:ASC-SRC2_Y_OFFSET           -0.0083

Still unclear if our NOMINAL_LOW_NOISE locklosses are actually ASC, but our fluctuations in POPAIR 18 and 90 can be as large as 20% and are slow, with dips lasting about 2-3 seconds.  If these fluctuations exceed 20% we tend to lose lock. See Pic 3.

ENGAGE_DRMI_ASC locklosses, turning off the ASC MICH FM1 (a -20dB gain) seems to be happening a bit early and kicking MICH in pitch pretty hard.
EDIT: I changed the ramp time of FM1 for MICH_P and MICH_Y from 4 seconds to 10 seconds.

TR_CARM locklosses from glitching from the LSC IN2GAIN lowering.

ALS XARM is glitching pretty badly sometimes, causing several ALS locklosses.