This afternoon we spent about 3 hours struggling to lock ALS.
At 20:45 UTC we lost lock while transitioning to Earthquake mode (though the ground motion wasn't really that high)
While reacquiring we kept losing lock while trying to lock ALS. We saw a few different problems:
In conclusion: for some reason we don't know, we couldn't lock ALS for a couple of hours this afternoon. Then for some reason we don't know, we could lock it again.
very similar symptoms now, July 8th 2019 around 23:00 UTC
The problem yesterday evening and probably at the earlier time as well was that the COMM beatnote strength was too low (less than -10dBm). We fixed this by moving a pico in HAM3. For now I've added a check to the ALS_COMM guardian so that it will give a notification and not try to lock if the comm beatnote is less than -9dBm.
During O1+O2 operators used to adjust the alignment of PR3 when the X arm was locked in green to increase the beatnote strength for COMM, we stopped doing this because we thought that PR3 is actually more stable when left alone, and that it might be better to just move the pico when the beatnote gets misaligned, which is what is done at LLO.
Corresponds to FRS Ticket 13231, which has now been closed as per above course action.