Displaying report 1-1 of 1.
Reports until 09:49, Tuesday 29 July 2025
H1 CDS
erik.vonreis@LIGO.ORG - posted 09:49, Tuesday 29 July 2025 - last comment - 16:44, Tuesday 29 July 2025(86059)
Dolphin IX switch replaced at EX

The dolphin IX switch at EX had broken management interface which prevented fencing of front end from the dolphin network for safe reboots.

I installed a new switch.  As part of the process, h1cdsrfm was turned off to avoid crashing it and in turn crashing CS and EY dolphin networks. 

When I turned h1cdsrfm back on, the cdsrfm port on the switch was already enabled, which led to timing glitches on h1susex and h1iscex.  This could have been prevented by re-fencing h1cdsrfm after the new switch was turned on.

I restarted the models on h1susex and h1iscex.  A few minutes later, h1susex crashed in the dolphin driver and had to be rebooted.  The crash was reminiscent of several susex (plus all of ey) crashes related to restarts of dis_networkmgr on the bootserver.  See 67389 for one example.

Comments related to this report
erik.vonreis@LIGO.ORG - 16:44, Tuesday 29 July 2025 (86089)

Switch sn ISX600-HN-000224 was replaces with ISX600-HN-000258.

Displaying report 1-1 of 1.