We've now had two lock losses from the Transition from ETMX state or immediately after while trying to reacquire. Unlike back on Nov 22 (alog81430), SRCL seems fine. For the first one, the locklost tool 1417133960 shows the IMC-SERVO_SPLITMON channel is saturating ~5sec before lock loss, then there is some odd LSC signals 40msecs before the tool tagged the lock loss (attachment 1). This might just be the lock loss itself though. The second one (locklost tool 1417136668) hasn't tagged anything yet, but ETMY has a glitch 2.5 sec before lock loss and ETMX seems to move more from that point (attachment 2).
Another one while I was looking at the code for the Transition_From_ETMX state. We were there for a few minutes before I noticed CHARD & DHARD inputs ringing up. Unsure how to save it, I just requested it to move on but it lead to a lock loss.
I ended up changing the SUS-ITMX_L3_LOCK_BIAS_TRAMP to 25 from its 30 to hopefully move to a safer place sooner. Since it was already changed from 60 a week ago, I didn't want to go too short. Worked this one time.
Sheila, Camilla
We had another lockloss with an ETMX_L2 glitch beforehand this morning, plot, and it seems that even a successful transition this morning had a glitch too but it was smaller plot. We looked at the ISC_LOCK.py code and it's not yet obvious what's causing this glitch. The successful transition also had a DARM wobble up to 2e6 plot but when we have the locklosses, DARM goes to ~10e6.
While looking at all the filters, we found that ETMX_L2_LOCK_L ramp time in 20s screenshot although we only wait 10s in ISC_LOCK. We will edit this tomorrow when we are not observing. We don't think this will effect the glitch as there is no input/output to this filter at the time of the glitch.
The only thing that seems like it could cause the glitch is the DARM1 FM1 being turned off, we don't yet understand how and had similar issues we thought we solved in 77640
This morning I edited ETMX_L2_LOCK_L FM1 ramp time down to 10s, reloaded coefficients.