Displaying report 1-1 of 1.
Reports until 16:16, Thursday 06 July 2023
H1 CDS
david.barker@LIGO.ORG - posted 16:16, Thursday 06 July 2023 (71120)
dts environment IOC stopped, delayed reaction to network outage

At 15:13 the IOC which serves the DTS environment EPICS channels stopped running, causing the EDC to go red with a disconnect count of 10. This was a delayed reaction to the loss of network connection between CDS and the GC network in the DTS room of the H2 building.

The timeline is:

13:13 PDT network connection went down, all DTS EPICS channels froze at their last values

14:00 PDT Network was restored, but IOC still had frozen channels

15:13 PDT The cdsioc0 systemd process which maintains the SSH-Tunnel restarted, establishing a good tunnel but causing the IOC to crash in the process

15:58 PDT I restarted the IOC, all the channels became available and the EDC went GREEN

Displaying report 1-1 of 1.