patrick.thomas@LIGO.ORG - posted 23:27, Wednesday 04 November 2015 - last comment - 01:00, Thursday 05 November 2015(23128)
Out of Observing
Jenne, Patrick Happened to notice that a bit on the ODC mini overview on nuc1 was red. Traced it to a DAC overflow on SR3 M2. Jenne trended it back. It looks like it started about 2 hours ago. The signal originates from the guardian cage servo. We have taken the IFO out of commissioning to manually offload it to M1.
The SR3 cage servo hit its limit, and started going crazy. This wouldn't have caused (and didn't cause) an immediate lockloss, but probably would have caused one eventually, after SR3 drifted far enough that the other ASC loops ran out of range.
As a reminder, the "cage servo" puts an offset into the M2 stage of SR2, such that the OSEM witness sensors on the M3 stage are kept at a constant pitch value. There is no offloading of this offset to M1, and we just ran out of range on the M2 actuators.
In the attached plot, the offset that we send to the M2 actuator is the "TEST_P_OUTPUT", which is multiplied by 8.3 and then sent to the M2 coils. This factor of 8.3 in the Euler-to-OSEM matrix means that if the Test_P output is above 15,600, we'll saturate the DAC output. You can see that about when the output hits 15,000 the SR3_M3_WIT starts to drift away from the 922 setpoint, and the Test_P_OUTPUT starts going crazy since it thinks it needs to keep pushing harder, since the witness is below 922.
While I don't think the data is compromised, we did take the IFO out of Observing mode while I manually offloaded the pitch actuation to the M1 stage. I moved the M1_OPTICALIGN_P_OFFSET by about 3 urad, and that brought the cage servo offset back to near zero. The SRC1 and SRC2 ASC loops reacted to this, but I did the offloading slowly enough that we didn't have any problems.
I have added a notification to the SR3 cage servo guardian to put up a message if the TEST_P_OUTPUT gets above 10,000 counts, so there's plenty of time (days) to offload the SR3 alignment before we run into this problem again.