Jennie found that HAM3 ISI loops wouldn't turn on. This morning I have been looking at why the loops are unstable. I'm pretty sure that the locked HEPI on that chamber is creating a notch in the ISI plant at 15 hz, where the loop gains and phase margin are both pretty low. The easiest way to fix this would be to unlock HEPI, but I don't think vac is ready for that. I could also tweak the boost filter to get some phase margin back, but that will reduce the loop gain below 10hz. I think that's probably the best choice at this point.
Attached plot shows some open loop gain measurements I've done for this chamber. Red trace is the unboosted loop, the cursor is at 15 hz, which is where HAM3 was ringing up when Jennie tried to turn the ISI on. This notch is not visible in the green trace, which is the unboosted loop gain with HEPI unlocked.
The boost filter is a pair of poles at .2hz and normally a pair of zeros at 9 hz. I've moved the zeros down to 7hz, which has substantially increased the phase margin shown in the attach plot. Red is the current, modified loop. I will test if this is stable. The other vertical loops don't seem to be affected by this.
Things were still kind of ringing even with the modified loop gain, so I asked Travis if he was done climbing around HAM3. He said the only thing was an annulus pump that could be accessed from the cover on the pier, so I've unlocked HEPI on HAM3. The ISI is much happier now. I haven't tried reverting the filter change yet, I will wait for a bit before changing anything, in case HAM3 HEPI needs locked and we need to try to run the ISI.