J. Kissel

After restarting the front wall Main DELTAL EXTERNAL ASD figure of merit, I saw that PCALX did not show the expected high-frequency roaming line. A quick look at the PCALX overview, and it was easy to see that the optical follower servo (OFS, the intensity stabilization and linearization servo for PCAL) was railed. The main time-series trend on the overview, of the OFS PD -- H1:CAL-PCALX_OFS_PD_OUTMON -- was steady at -7.7 V, with no sign of the expected ~kHz oscillation. 

I trended back to find out when the OFS had railed, and it appears coincident with the Beckhoff restarts this morning at 15:06:46 UTC (08:06:46 PDT). The OFS PD went to zero then, and then a few minutes later (at 15:13:44 UTC, 08:13:44 PDT), after bouncing around between OFF and RAILED, it settled on RAILED.

I fixed this by turning OFF, then turning ON the OFS loop, i.e. toggling the orange H1:CAL-PCALX_OPTICALFOLLOWERSERVOENABLE "loop enable" switch next to the diagram of the loop -- also on the PCALX overview. I did so at 23:11:55 UTC (16:11:55 UTC), before we resumed any observation ready segments, so this has no permanent impact on that quality of data.

We should consider adding "check on the PCALs!!" to the recovery checklist after any maintenance day in which a Beckhoff restart has taken place.
We should also add "Restart the wall FOMs!" to the recovery checklist after any maintenance day in which a DAQ restart has taken place.
Such activities should be considered akin to a physical VEA sweep done at the close maintenance.