I've noticed that this was not alog-ed yet, so here it is.

1: Oscillation

Right after WFSs were powered up originally on Jan/16, WFSA segment 1 and 4 showed some crazy thing going on, which continued for 9 hours or so (first plot, the vertical line in CH2 and CH3 indicates the time of installation). This is the head the installation crews reported to be hot, so the oscillation thing was there from the start, but this may be a different problem from the death of WFSs.

2: Death of WFSB at 13:17:47 local time propagated to WFSA, then to MC REFL almost instantly

On Jan/22 at 13:17:43 local time, when WFSB was malfunctioning and outputing a bogus offset, something happened to WFSB and the output went zero-ish. (Second plot).

Four seconds later, something drastic happened that caused huge spikes to everything.

Though you might not be able to see it, first WFSA WFSB saw a spike, then 4/16 seconds later WFSB WFSA, then 2/16 seconds later MC REFL PD, in this order. Though the timing of EPICS is sort of dubious, we're not talking about 1 sample time difference here and I'm sure this order is correct.

None of WFSA segments responded to light since then, and REFL DC developed a large offset since then.

When everything was dead, CM board power was tripped also. WFSA and WFSB are connected to the same chassis, but MC REFL is connected to a different chassis, and CM board is in yet another chassis.

Though it's not on the plot, we didn't find any suspicious thing in the PSL except obvious things (e.g. FSS disturbed by MC servo which was kicked by the incident).

3: Did WFSs die when the breaker of the WFS DC interface tripped?

As I reported earlier, at some time on Jan/22 the DC WFS interface went bad (it was the power protection board) when I was measuring WFSB supply voltage, and the breaker switch was tripped. I was the only person in front of the field rack at that time.

But it's not clear if I was in the LVEA at 13:17. Combining blurry memory of me, Filiberto and Kiwamu, plus an incomplete record of the card access system (it didn't register my swiping out when I went out of LVEA when somebody else opened the door to go out or come in, or I didn't bother to swipe out  and just went after somebody else), it's not impossible that the breaker tripped when WFSs died.