TJ investigated why the LOCKLOSS_SHUTTER_CHECK guardian was sometimes mistakenly identifying locklosses when there had not been any lockloss while ISC_LOCK was in the DRMI_ON_POP state.
As a reminder, the only purpose of LOCKLOSS_SHUTTER_CHECK is to check that the shutter triggers after locklosses in which we had more than 25kW circulating power in the arms. The lockloss checking for this guardian is independent of any other guardian.
The problem TJ found was a consequence of the work described in 31980 . Since that work when we switch the whitening gain on the transmon QPDs, there is a large spike in the arm transmission channels which the LOCKLOSS_SHUTTER_CHECK guardian recognizes as a lockloss (TJ will attach a plot).
We edited the ISC_LOCK guardian to hold the output of the TR_A,B_NSUM filter before making the switch, and turn the hold off after the switch is done. We loaded this when we were knocked out of observe by TCS. This is a simple change but if operators have any trouble with DRMI_ON_POP tonight you can call TJ or I.
Here are some plots with the TR_A,B_NSUM channels and the numeric states for ISC_LOCK. The Lockloss Shutter Check node would think that the power in the arms was above its 25kW threshold where it would then move to it's High Power state. This state would check that the arm power didn't drop below its low threshold, thinking it was a lockloss, and then jump to the Check Shutter state. Here it takes the last 10sec of data and tests for a kick in the HAM6 GS13s. This test would fail since there was no lockloss. We were not even in high power at the time.