The SR3 cage servo has gone beyond its limit, and is saturating the output of the SR3 M2 coils.
Back in November (aLog 23131) I put a check into the SR3_CAGE_SERVO guardian to let the operator know when it's close to the limit, and that it needed to be offloaded to the M1 actuators. However, I only put in a check for if the servo is getting close to the positive limit. As of 12 Dec around 18:00 UTC, we've been railed at the negative limit. Ooops.
Right now, SR3 is about 13 urad from the usual value in pitch (currently the OSEMs read 935, rather than the 922 that is the cage servo's setpoint), but the ASC seems to be fine with it.
At the next opportunity, we'll offload SR3's alignment, and fix the guardian code so that it checks for both the positive and negative limits.
In the attached figure, you can see that SR3 is no longer at its nominal pointing, and that the cage servo's setting is no longer within +/-15,000 counts (which corresponds to +/-130k counts at the DAC).
Trying to fix this broke the lock :(
Even though Corey and I typed a number (-16,000) into the SR3_M2_TEST_P_OFFSET (to get the cage servo closer to zero) that should have left the coil actuators railed, 2 of the coils seem to have flipped sign. This broke the lock. I'm still looking into why typing -16,000 didn't do what I expected.
I have been looking into the lockloss that I caused earlier when Corey and I were offloading the SR3 cage servo, because I don't think that it should have caused a lockloss.
I think that I have found a mysterious sign flip in the DAC output, which I'm hoping that someone will have an idea about. Somehow, the LR and UR DAC outputs switched sign a few days ago, even though the MASTER_OUTs did not flip sign. See first attachment for a 3 day trend showing this.
When Corey typed -16,000 into the SR3_M2_TEST_OFFSET, the signs of the LR and UR DAC outputs flipped to correct themselves. See second attachment for a 20 minute trend showing this. The COILOUTFs are the same as the MASTEROUTs, and they stay whatever sign they started (LL and LR positive, UL and UR negative). However, the DAC_OUTPUTs 6 and 7 (6=UR, 7=LR) flip sign. This undoes the mysterious sign flip that happened a few days ago.
Why is the DAC output flipping sign, when the digital signals going to them are not?!?!
