RyanS reported getting HAM3 CPS noise notifications on DIAG_MAIN. Looking at the log, there have been several periods where the blrms noise monitors have been going over threshold. Digging in to HAM3's performance this morning, CPS noise seems to be affection mostly RZ, but the other horizontal dofs are affected. as well. First attached image, is comparision with HAM2. Top asds are the rotational dofs, bottom are the X/Y/Z dofs. HAM3 RX/RY/Z vertical dofs are all mostly similar to HAM2, but the cyan trace vs the pink trace on the top plot shows HAM3 RZ is moving much more that HAM2 RZ.
Second image shows asds of the individual sensors, the blue trace on the top plot is H2 CPS, which has a higher noise floor than the other sensors, so I suspect this sensor is the root cause of the HAM3s problems. All of the other senors seem more or less healthy. If I get a chance or the ISI becomes unstable, I would like to go to the floor or CER to power cycle some stuff. Otherwise, I can wait till tomorrow.
This trend shows when the H2 CPS started misbehaving. Top two plots are the 1-3hz log blrms for the RZ GS13s HAM2 is pretty steady, but something happens at ~10:30am pst on the 18th. Bottom plot shows the horizontal CPS during this time, H1 and H3 stay more or less the same, but H2 shows an increase in noise that has persisted until now.
Got permission from Sheila to try addressing this because locking is not going well. Powered off the CPS racks for ~10 secs and noise was gone when the signals came back up. Will keep monitoring.