Camilla, Sheila, Erik
Erik points out that we've lost lock 54 times since November in the guardian states 557 558 transition from ETMX or low noise ESD ETMX.
We thought that part of the problem with this state was a glitch caused when the boost filter in DARM1 FM1 is turned off, which motivated Erik's change to the filter ramping on Tuesday 82263, which was later reverted after two locklosses that happened 12 seconds after the filter ramp, 82284.
Today we added 5 seconds to the pause after the filter is ramped off (previously the filter ramp time and the pause were both 10 seconds long, now the filter ramp time is still 10 seconds but the pause is 15 seconds). We hope this will allow us to better tell if the filter ramp is the problem or something that happens immediately after.
In the last two lock acquisitions, we've had the fast DARM and ETMY L2 glitch 10seconds after DARM1 FM1 was turned off. Plots attached from Jan 26th and zoom, and Jan 27th and zoom. Expect this means this fast glitch is from the FM1 turning off, but we've seen this glitch come and go in the past, e.g. 81638 where we though we fixed the glitch by never turning on DARM_FM1, but we still were turning FM1 on, just later in the lock sequence.
In the lock losses we saw on Jan 14th 82277 after the OMC change (plot), I don't see the fast glitch but there is a larger slower glitch that causes the lockloss. One thing to note different between that date and recently is that the counts of the SUS are double the size. We always have the large slow glitch, but when the ground is moving more we struggle to survive it? Did the 82263 h1omc change fix the fast glitch from FM1 turning off (that seems to come and go) and we were just unlucky with the slower glitch and high ground motion the day of the change?
Can see from the attached microseism plot that it was much worse around Jan 14th than now.
Around 2025-01-21 22:29:23 UTC (gps 1421533781) there was a lock-loss in the ISC_LOCK state 557 that happened before FM1 was turned off.
It appears to have happend about 15 seconds after executing the code block where
self.counter == 2. This is about half way through the 31 second wait period before executing the self.counter == 3,4 blocks.See attached graph.