Camilla was having a few locklosses at TURN_ON_BS_STAGE2, so she asked me to check. The only issue that I found was that the ST1-2 sensor correction was on, which I had changed the SEI_CONF nominal state to have off a couple of months ago, as part of this alog, where I also made some tweaks to the ISO filters. This probably got turned on by accident with the cornerstation crash a couple days ago, and for some reason wasn't recovered properly. This was probably caused by a the gains for the sensor correction being set on in the safe.snap. Because this is controlled by a guardian, it isn't monitored in SDF, and the guardians don't raise an alert. I've fixed the safe.snap to have the senscor off, so this shouldn't happen in the future, but several issues with the seismic configuration have come up over the last several days because there isn't a clear procedure for making sure these systems are fully recovered after ISI restarts. This control path is kind of obscure, so wouldn't be easy to find.

The attached plot shows the BS ST2 drives, BS guardian state number and POPAIR from the end of the last lock (with the st1-2 senscor on) until now (with this senscor off). The blue and orange are the ISI ST2 drives and the drives are definitely larger with the sensor correction on, which is probably fine for steady state, but seems to be a problem for trying to turn the loops on with MICH locked. Probably we could turn this sensor correction on after the loops were on and settled, but getting all of the necesary guardians wrangled would be complicated, and the difference probably isn't worth it.

I've talked to Niko about adding a flag to the lock loss page for lock losses that coincide with this ISC_LOCK state, and SEI_BS being in transition. Should help us track these kinds of locklosses over time.