jenne.driggers@LIGO.ORG - posted 07:21, Thursday 18 May 2023 - last comment - 16:42, Thursday 18 May 2023(69719)
Sqz guardian was stuck in Sqz_ready_ifo
This morning, H1 was not in Observing because the squeezer was not injecting squeezing. I requested the Sqz_manager to DOWN, then FREQ_DEP_SQZ. That worked, and the IFO automatically flipped to Observing.
Since we increased the SHG power on Tuesday (69671) the FC launch power almost doubled, 3mW to 6mW see attached. I have increased the sqzparams.fcgs_trans_lock_threshold from 50 to 110uW (was recently reduced from 80uW to allow FC to lock with low SHG) and reloaded SQZ_FC to stop this happening again.
We should think about if we want 6mW into FC launch and if we've always had this much power in higher order modes in the FC. There hasn't been any ZM2 PSAMS changes since the end of April.
TJ noted the SQZ_LO_LR guardian was unhappy at the time the FC unlocked, this is because the OPO unlocked and SQZ_MANAGER jumped to LOCK_OPO_AND_FC but didn't run though the DOWN state so BDiv stayed open and LO locking, I added a turn_off_sqz() line to LOCK_OPO_AND_FC. We should check there aren't more jump states that need this added.
Like Sheila and Camilla found, the first hours-long squeezer lockloss was due to FC locking on a LG01 donut mode. Their solution to increase the green lock threshold was great! Daniel and I have also updated the nominal FC GREEN TRANS value in slowcontrols, H1:SQZ-FC_TRANS_C_DC_NOMINAL.
Additionally though, FC guardian should have brought itself down b/c the beam clearly failed the TEM00 mode-checker decorator. This function looks at the FC TRANS GREEN camera's gaussian widths of the beam waist, H1:VID-FC_TRANS_C_{WX,WY} to check TEM00. Daniel and I also fixed this secondary bug, so now we should be doing check_TEM00() in the SQZ_FC guardian again. See the bottom trends: FC green trans was above its lock threshold (SQZ_FC_TRANS_C_LF_OUTPUT), but the beam's fitted waists were about 2x bigger than normal -- however, this didn't trigger FC to relock because of a stray "else" statement error in the guardian code logic. This should now be fixed, and we will monitor.
The second sqz lockloss could have happened because the OPO PZT voltage bottomed out, see the top trend here when SQZ_MANAGER went down. Not sure this is related to lab temp changes of even 0.5C. We've previously addressed this same issue for the SHG PZT in 69366. What I've done to address this: to deal with OPO PZT bottoming out in the future, I've edited SQZ_MANAGER to relock the OPO if its voltage is too low, and we're not squeezing. Specifically, I've upgraded "@SHG_PZT_checker" to a general "@PZT_checker" which now checks both SHG + OPO PZTs in the SQZ_READY_IFO state. And, in the "LOCK_OPO_AND_CLF" state, if not OPO_PZT_OK() (b/c its too low), the opo will re-lock.
From TJ and Camilla's investigations above, I checked and I don't think there are more jump states that need to "turn_off_sqz()"; in fact only DOWN and LOCK_TTFSS are goto=True states, and not LOCK_OPO_AND_FC. In turn_off_sqz(), I added a 1 second sleep timer to let the beam diverter close before moving on to relock the squeezer.