E. Hall, J. Kissel, T. Sadecki, H. Radkins, J. Warner

The IFO had lost lock again from the same problem about ~20 minutes later. We noticed LLO was down, so we tried pushing buttons hoping to rectify the problem. 

We tried several things, all on hunches we'd had from all signals we saw oscillating at ~30-40 [mHz], which included HEPI / ST1 ISI / PRC1 Yaw and TIDAL -- so this is NOT just an "ISI ETMX Loop goes unstable" problem, its a nasty, slow, cross-coupled interaction of which the ISI is only one of the players.
Things we "know" that informed our hunches and button mashing:
- This only shows up when the ISIs are on 45 [mHz] blends.
- Symptoms point us toward ETMX, because it stays rung-up after lock-loss
- The above mentioned channels are visible wobbling at these 30-40 [mHz] frequencies
- There seems to be a 6-8 minute period envelope to the oscillations

Our hunches include 
(1) Bad IMC_F-to-UIM or UIM-to-HEPI tidal cross-over when ISIs are in 45 [mHz] blends, i.e. the tidal offload is at a higher frequency than the ISI's inertial blend frequency, so the ISI's isolation loops are fighting actuation from tidal
(2) PRC1 Yaw ASC loop (which keeps PRM aligned to the PRC's cavity axis) unstable, which modulates the power into the arms. This modulates the classical radiation pressure on the ETMs in common, which excites CARM / Tidal. This then puts excess motion into the Tidal feedback and offload. 
(3) Excess motion on HEPI from tidal is causing excess tilt, because HEPI tilts when you request it to drive in longitudinal. With the ISI ST1 in 45 [mHz] blends, and the RX&RY blend still in the 250 [mHz] blend configuration, the excess tilt from HEPI is coupling into the ST1 CPS, tilting the ISI, and fooling the X&Y DOFs into thinking there's excess X&Y motion, and drives excess X&Y. This pushes the test mass in X&Y, exciting some DARM / CARM action, which then spills back to HEPI. 
Note that these are all sort of half-coupling mechanisms that, if I waved my hands hard enough, you might believe. But, we have no measurements (i.e. Sys ID, Loop Transfer Function measurements) to prove any of these statements.

Based on these hunches, we twiddled the following knobs:
- Reducing UIM to HEPI offload UGF (H1:LSC-${X,Y}_TIDAL_CTRL_UGF) from 0.002 to 0.001 [Hz] --> no noticable effect, but we were watching a 10 [minute] strip tool, so out patience may not have been high enough. The Oscillation was present, but not ringing up.
- Reducing UIM to HEPI offload UGF by half again to 0.0005 [Hz] --> same result
- Returning UIM to HEPI offload UGF to 0.001 [Hz], and reducing the IMC-F to UIM offload UGF (H1:LSC-${X,Y}_COMM_CTRL_UGF) --> We might have started to see a reduction in amplitude.
- Reducing the PRC1 loop gain (via the POP A DC to PRC1 ASC input matrix H1:ASC-INMATRIX_Y_3_21) from 1.0 to 0.5 --> This seems to reduce the oscillations in all signals.

While Evan and Jim coninued to twiddle knobs, I and LLO was down, I suggested we try to characterize some of these low frequency loops. However, a third of the way through my initial characterization of the UIM to HEPI tidal offload (which had already had a bad start, see LHO aLOG 23680), the oscillations had come back to a point of no return. Unclear whether they were because we just didn't reduce the original oscillation fully, or whether the bad start to the excition kicked everything enough to restart a bigger oscillation, or if it was my measurement itself slowly driving things around that re-triggered the oscillation enough to eventually saturate Tidal / the UIM. However, it was apparent that it was the 6-8 minute envelope that brought things over the edge, not the 30-40 [mHz].

After that lock-loss, Jim had found IR in the arms to be pretty bad, so he has since gone into initial alignment.

Stay tuned...

I attach some visual aides that I'd used to discuss the hunches with Hugh / Evan / Travis in case it helps anyone else.

(THC = Tilt Horizontal Coupling).