We were just kicked out of OBSERVE by an error in the squeezer guardian. There are ECATC1PLC4 errors (first screen shot) and the SQZ_MANAGER is complaining about an error, but it looks like another typo in a state name. The log has this as the last few lines before the first error:
2019-04-09_04:36:00.627746Z SQZ_MANAGER MODE: MANAGED
2019-04-09_04:36:00.630705Z SQZ_MANAGER MANAGER: assigned: ISC_LOCK
2019-04-09_08:26:01.821216Z SQZ_MANAGER [SQUEEZING.run] Unstalling SQZ_SHG
2019-04-09_08:26:02.444636Z SQZ_MANAGER [SQUEEZING.run] Unstalling SQZ_CLF_LR
2019-04-09_08:26:04.655168Z SQZ_MANAGER [SQUEEZING.run] Unstalling SQZ_OPO_LR
2019-04-09_08:26:04.775252Z SQZ_MANAGER [SQUEEZING.run] OPO unlocked, turning off SQZ ASC
2019-04-09_08:26:04.776589Z SQZ_MANAGER [SQUEEZING.run] ezca: H1:SQZ-ASC_WFS_SWTCH => 0
2019-04-09_08:26:04.777483Z SQZ_MANAGER [SQUEEZING.run] ezca: H1:SYS-MOTION_C_BDIV_C_CLOSE => 1
2019-04-09_08:26:04.778570Z SQZ_MANAGER [SQUEEZING.run] ezca: H1:GRD-SQZ_LO_LR_REQUEST => DOWN
2019-04-09_08:26:04.779165Z SQZ_MANAGER W: ERROR: state returned unknown jump state: LOCK_OPO_DUAL_RESONANCE
2019-04-09_08:26:04.831804Z SQZ_MANAGER ERROR in state SQUEEZING: see log for more info (LOAD to reset)
In the guardian there is no state called LOCK_OPO_DUAL_RESONANCE, but there is a state called LOCKING_OPO_DUAL_RESONANCE. Looks like a typo on line 88.
After fixing that, SQZ_MANAGER seems happy again, but there is still an SDF diff LO_SERVO_COMBOOST, setpoint is zero, epics value is 2, but the range seems to have recovered? I'm accepting to get back to observing.
Thanks Jim.
The LO SERVO board should have 2 boosts on, so I'm not sure why the setpoint in SDF would have been set to not have boosts on.
The error you got was a typo, thanks for fixing it. Yesterday Nutsinee and I made some changes to the way the squeezer guardians check for locklosses and respond to them, this typo was left over from that. It looks like the squeezer lost lock again, probably related to the mode hopping problems we've been having since Friday.