We have had several incidents where MC1+MC3 trip, and sometimes also cause HAM2 ISI to trip. Kiwamu found that there were large signals coming through DOF4 (16128), although there was some other underlying problem which was causing MC2 to get large length signals.
We Had DOF 4 off for most of the day saturday, once we found and fixed the PSL noise eater oscillation we turned it back on. This morning we dropped the mode cleaner lock when we were attempting to lock the X arm in IR, which also tripped MC1, MC3, and HAM2 ISI (this lockloss is the kind of thing that is expected to happen once in a while, and should not cause a cascade of trips). We have again held the offsets on DOF 4, because somehow this loop makes the sysem more fragile. Since holding the outputs we have had several mode cleaner lock losses without any trips.
Perhaps it is worth while going through the WFS alignment process once. The most likely scenario is that the WFS_B_DC_PIT and YAW offsets were turned on while the spots were centered. I am sure the following steps are obvious but let me put them down here any way for future reference:
Procedure to center the direct reflection from the IMC on the WFS sensors:
1) Start with a good alignment of the IMC input beam. Off load any offsets from the suspensions. Maximise IMC output by hand if necessary.
2) Unlock the mode cleaner by invoking the "Down" state in the IMC Guardian
3) Using the MC2 Alignment sliders misalign the MC2 by a milliradian or so in pitch till the IMC stops flashing. (Use StripTool)
4) Open the WFS DC screens and inside that open the WFS_A and WFS B_PIT and YAW filter banks. Switch off any offsets that may be present.
5) Using the IMC WFS picomotors center the direct reflection spots on the WFS A and WFS B.
6) Switch off the DOF4 inputs in YAW and PIT. Make sure that the history is cleared and that all servo outputs are zeroed.
(At this stage we have ensured that the gaussian sidebands (24MHz) are centered on the WFS sensors)
Now we start setting up the DOF4 servos:
7) Bring the MC2 Back into alignment. Note that there is a wait time of several minutes before the top stage of the MC2 moves into place. There is a long time constant low pass filter on the alignments sliders
8) Due to some hysteresis somewhere, you may need to search for good high intensity flashes. They will be within a few tens of microrads from the previous setting. (StripTool)
9) Bring the IMC back to "LOCKED" state in IMC Guardian. Keep the DOF4 loops off at this stage.
10) As the mode cleaner alignment loops (DOF1,2,3) work the reflected field is extinguished and some wierd pattern develops in the IMC REFL camera. This will take about 3 to 5 mins or so.
11) This wierd shape causes a spurious offset to appear on the WFS_A and WFS_B DC signals.
12) Zero these signals using the offsets in the WFS_DC PIT and YAW filter banks.
13) If everything is stable switch on the DOF4 filter bank inputs.
14) Keep track of the DOF4 outputs. They should not be monotonously building up to values larger than a few tens. If they cross 100 then something is misaligned.
15) Currently DOF4 uses WFS_B. WFS_A serves as an out of loop sensor. Trend the WFS_A PIT and YAW after a day and see if there is any drastic change. It indicates a drifting alignment causing an evolving IMC REFL pattern. Not great news but not a deal breaker either. As long as it is not monotonously running away to values outside [-0.25, +0.25] it should be okay.