While running an Initial Alignment tonight, I was stuck at SRC alignment for quite a bit of time (I tried to address issue with various cycles of running down script, centering SR2 on AS_C, but had no luck).
Jenne tracked down the issue with guardian not liking where one of the SUS guardians was at (basically it thought one of the SUS guardians was actively changing). Sure enough this was tracked down to ITMx (If I remember correctly).
SYMPTOM:
SOLUTION:
After Jenne tracked down our sticky slider, she moved ITMx pit by 0.1 one way and back the other way, and as she did this (from Japan!) Guardian swung back into action and began to run through SRC alignment.
ACTION:
Notify TJ, Dave B, or Jaime about this issue since it has been a known issue to cause DOWN Time.
We've been seeing this issue for a while, although this might be one of the first times that I can remember it causing down time versus just general confusion (which is also not so good). Usually we see the yellow boxes stuck on after some alignment is offloaded via guardian in the main acquisition sequence, but they haven't seemed to stop the guardian from going forward.
As Corey reports, the ITMX pitch opticalign slider had it's is_ramping value stuck at True. The is_ramping value is what makes the boxes yellow when we ramp offsets or gains. This was causing the ITMX guardian to be stuck in MISALIGNING, and not getting all the way to MISALIGNED.
The ALIGN_IFO guardian is checking ITMX to ensure that it is fully misaligned before trying to lock the SRY cavity. Since ITMX was stuck in MISALIGNING and wasn't getting to MISALIGNED, the ALIGN_IFO guardian was stuck in PREP_SRY. To clear the is_ramping value, I clicked the ITMX pitch slider up one, and then back.
FRS filed here: https://services.ligo-la.caltech.edu/FRS/show_bug.cgi?id=13563