david.barker@LIGO.ORG - posted 10:27, Saturday 03 February 2024 - last comment - 14:29, Monday 05 February 2024(75722)
CDS Status: HWSEY and PicketFence down
Two systems stopped running overnight:
HWS-EY: computer crashed at 03:07 Sat 03feb2024 PST. This is the cause of the EDC's 86 disconnected channels. h1hwsey is not pingable, but its IPMI port is working.
Picket Fence: the server on nuc5 stopped updating at 21:53 Fri 02Feb2024 PST, the IOC continues to run with flatlined channels.
Both can wait till Monday for restarts and investigations.
h1hwsey suffered a kernel crash. Photo of the console including stack trace is below. I rebooted the server and it came back up ok.