Upon relocking this morning, after a long night of super-winds, Keita and Patrick immediately found that the ETMs were way off in alignment. We sleuthed around in the trends and can re-tell some of the confusion from yesterday's lunch time:
After Keita/Kiwamu took a break from phase measurments using a Michaelson lock, I used the lunch hour to take charge measurements. Although set to ALIGN at the start of the measurements, during them, I noticed that something was telling the ETMs to go to the MISALIGN state, even though all top guardians were set to DOWN. We ended up PAUSING some of them and then did not witness any problems afterwards. From trends we can now see that the during the full set of measurements (automated 5 sets during ~1hour) the charge script used the OPTICALIGN sliders to return the optic pointing to zero on the oplev while the optic was MISALIGNed via the normal TEST OFFSETs. A measurement later, we had reset the guardian to ALIGN and so the script drove the OPTICALIGN sliders back to nominal (again returning to zero on the oplev). However, because the optic had so far to move, it bounced for a bit and the script "timed out", resetting the OPTICALIGN to it's script-restored bad pointing values. These values were written into subsequent restore files by the script and therefore the optic never returned to the nominal pointing once the script concluded like it should.
Long story short - while there are a few places in the charge script which look for the guardian state to be correct, it may not always get returned, so watch it. In the above case, we could increase the amount of "tries" it uses for the optic to quit swinging while zeroing on the oplev (which is short when in-between measurements).