I think I've fixed the issue with the bad blends we found on Friday, so we should have any more slow drifts in RZ CPS on the HAMs. First attached screen shot shows the zeros and poles as read by foton for the new filter (top row, installed as SUPERSENS5 on all the table) and the old bad blend (bottom row, SUPERSENS4). The issue with the bad blend were 3 poles at "0 hz" and 3 zeros at "0 hz" in the high pass, the low passes appear to be identical. I missed a step doing a "minimal realization" of the filters in my design script, and it wasn't caught by some of the checks I usually do, so I'm going to look at adding some alarms for that. These erroneous poles and zeros don't show up in any normal bode plot or step response (second image).
For now, I've installed the fixed filters on all chambers in the SUPERSENS5 path, set and tested the blend guardians to use the new filter and accepted the settings in SDF for all of the chambers. I've left the old filters in, but they aren't being used, I'll remove them in a couple weeks but I want to use them as a check against the filter I installed today. I'm also going to leave in the DIAG_MAIN test in, but once I'm convinced the problem is definitely fixed I'd like to remove it.
Since this fix, the RZ RESIDUAL monintors have been rock solid, oscillating with low-normal-levels of noise around 0.0 nrad. This indicates that the tables are no longer slowly drifting away in yaw, and this has fixed the problem. The attached plot shows the long term trend of the RESIDUALMON channels from all HAM platforms before the fix and the few weeks since. The crosshair pin-points the data/time of this installed fixed. (The nastiness seen in HAM7, shown in brown, a few days later was the unrelated overnight issues with that chamber's ISI interface chassis; see LHO:70117.) Nice work Jim!
