Dan, Sheila, Evan, Jeff, Alexa
Tonight locked many times (on RF) but didn't stay locked verry long. We found two new lockloss causes that were relatively easy to solve:
- The ALS_DIFF guardian was going into the DOWN state (and setting the DARM gain to 0) even though it was stalled. We have a temporary solution (which is awkward and can sometimes cause the DIFF guardian to get stuck in the DIFF_LOCKLOSS state) and Jamie is looking into why this happened.
- The COMM (fast) Tidal was not happening. The offset was off for END X, and the TIDAL_HACK guardian was also not running. We updated the safe.snaps for h1alsex and h1alsey.
- By the time that we understood and fixed these two problems we got another large earthquake.
During one of our short locks early in he evening , we made an attempt to align the green to the arm cavities after the DHARD WFS had come on. For the X arm the only thing we needed to do to get a good build up was move the TMS, we saw the spot move on the ITMX green camera and updated the spot positions for the green WFS (248.3 for PITCH, and 354.8 for YAW). For the Y arm we saw that moving TMS was not good enough, we needed to also adjust QPD offsets. We lost the lock in the middle of this process. Note: the recycling gain tonight was low, around 25, but we think that having the green co-alinged even for this low build up would be better than our original situation.
During the earth quake we have spent some time on the new COMM IR search state that evan wrote. We modified it, it is now 100% successful, but it can take a couple of minutes (generally under 5). This is still faster on average than having a human do it, and we can think about something else while this is working now.
Here are the plots that tell the tidal story. The offset to the {X,Y}_COMM_ERR filter banks is supposed to be updated by the TIDAL_HACK Guardian; this provides the error signal which is controlled through the SUS and ultimately offloaded to HEPI. This offset was turned off for the X-end on Tuesday, which is why the X_COMM_ERR output is zero. At the same time the HACK Guardian, which takes the DC component of IMC_F and writes this to the COMM_ERR offsets, was turned off, which is why the output of Y_COMM_ERR is a nonzero constant. The Y_COMM_CTRL filter bank was happily integrating this constant off to infinity whenever the arms were locked. HEPI was still getting some input from the SUS drives but it was, as you might expect, not helpful.
First plot is today, we lost lock because IMC_F (lower left corner) was railing at the limit of the VCO, something like >2000 counts (calibrated to kHz, so that's the 2MHz VCO range).
Second plot is the 2-hour lock from last week, when the IMC_F output was small and steady. The low-frequency signal was being offloaded to HEPI, the COMM_ERR outputs were nonzero.
It looks like the TIDAL_HACK guardian module does not define an initial request, meaning that it will come up with REQUEST = NONE, and the system won't leave the INIT state. This is why the TIDAL_HACK node wasn't doing anything after the recent guardian restarts.
I suggest setting:
request = 'COPY_AND_PASTE'
in that module, so that when the node restarts it always goes back into the appropriate state.