HAM4 actuators are tripping pretty about every 10 minutes for the last hour or so. While there are people in the LVEA it seems unlikely to be related. Attached is a plot of one of the trips.
In the second attachment you can see that something seems to have broken around 6 am this morning (around 13 UTC July 27th). There is a narrow line at 54.0 Hz (3rd attachment).
The last attachment shows that the 54Hz line doesn't show up now that the platform is damping.
The fact that this line is so narrow and present in both the gs13s (on St1) and l4cs (on St0) makes me think this is likely electronic. The same line is not present in the hepi l4cs (kind of hard to see on my attached plot, but it's not visible below the legend, and dtt is too unstable and annoying to try again over ssh). I connected with my ubikey and was able to get the isi on and running stably (for 1.5 hours so far) just by turning off the St0 l4c ff path. We should try power cycling some chassises and/or restarting the whole computer tomorrow.
Thanks Jim (and Betsy) .
HAM4 is still isolated, so it seems like this is a good solution for now. I opened an FRS ticket 11177