Finally got around to looking more closely at the ITMX coil driver oscillations that caused a lockloss on July 9th . We have some channels that report the voltage and current used by each actuator, and looking at those time series it's pretty obvious that the ST2 H3 coil was the culprit. First attached plot are asds of the St2 current (top plot) and voltage (bottom plot) monitor channels. This was taken right after the ISI watchdog trip when the front end wasn't requesting any drive, but for some reason this particular coil was spewing a lot of noise. Both H3 current and voltage are many times the other 6 st2 coils. There was enough drive coming out of this one actuator that it was shaking St1 as well.
Second attached plot are time series for a st 1 actuator and the st2 h3 leading up to the trip. I think the St2 H3 actuator is yellow on both plots. About 40 seconds before the trip the voltage for the st2 h3 actuator starts behaving strange, gets worse until the ISI finally trips. The current for this actuator doesn't show this behavior until after the ISI trips, I don't really understand that.
We have an ECR to remove these DQ channels to try to free up some mb/s, and just keep some equivalent epics monitors. I don't think this presents an argument against removing them, the epics timeseries are just as clear as the full data channels in showing there was a problem. I would like to add the epics mons to our ISI coil driver overview screens, that would make finding this issue easier in the future. Could also think about a DIAG_MAIN test, or something, but only if this starts becoming a problem. I think we have seen this twice since aLIGO install started.