Short version: ESDX HV power supply is now off and wont come back on.
DIAG_MAIN reported that the ESD X high voltage was off after the 11:02 lockloss. I toggled the HV ON/OFF button and the DC_OUTMON came back to life at ~31000. I went back to locking and then about 15min later realized that the same notification was there. I tried the same toggling and it seemed to work again, but this time only for about 30sec. Every toggle would bring up the DC_OUTMON and other quadrant chans to their normal values for anywhere between 1-30sec before dropping back to single digits. I had to look through way too many alogs to find out how to restart the HV (alog19480 or alog28304 I'll add them to an ops wiki). Drove to the EX and immediately heard a constant constant beep from the left power supply but no error messages on the display. The right one seemed OK. All of the buttons seemed to be unresponsive, so I flipped the rocker switch down but it got caught on a broken piece of plastic, see attached pic. Cleared the debris, got it off, and then it would turn itself back off when trying to flip it on. I'm out of ideas. I sent out some emails, and I'll call when it gets a bit later.
Richard came on site and drove to EX to with a spare power supply, but he never needed it. He was able to just flip the switch and turn it on. But in my defense, he has seen this before. Seems to be working now and I'm resuming locking.
Note: Don't leave one supply off while the other is on, turn both off. This could be damaging to the supplies.