david.barker@LIGO.ORG - posted 08:46, Thursday 31 October 2024 - last comment - 09:52, Thursday 31 October 2024(80962)
h1seih45 glitch 07:09:46 Thu 31oct2024 PDT
All models on h1seih45 glitched at 07:09:46 PDT (14:09:46 UTC). This appears to have broken the lock and caused guardian to start an initial alignment.
Symptoms are:
- All IPC receivers, Dolphin and SHMEM, on all h1seih45 models reported a single receive error
- DAQ (both DC0 and DC1) reported a single CRC error for all h1seih45 models
- dmesg, DAQ and central loggers show no logs at this time
- No evidence of a timing glitch at this time
The models appeared to have resumed normal operation following the glitch, no restarts were needed. At the time of writing H1 is locked with a range of 160MPc.
At the time of the glitch we had been relocking and at TRANSITION_FROM_ETMX. There seem to have been no issues at all from this - the initial alignment was automatically started after trying to relock took us into CHECK_MICH_FRINGES for the third time during the relock period, and went smoothly and without issue.
HAM4 and HAM5 GS13 signals following the glitch:
FRS32532
There are 8 Dolphin IPC channels which are sent by h1seih45 models (h1isiham4 and h1isiham5). They are all received by h1seiproc on h1oaf0. h1seiproc runs at 4kHz.
All 8 receive channels saw 4096 errors at the time of the glitch.
rcv model type snd host snd model ipc channel
h1seiproc PCIE h1seih45 h1isiham4 H1:ISI-HAM4_CPS_X_IPC_OUT
h1seiproc PCIE h1seih45 h1isiham4 H1:ISI-HAM4_CPS_Y_IPC_OUT
h1seiproc PCIE h1seih45 h1isiham4 H1:ISI-HAM4_SUSPOINT_SR2_L_OUT_IPC
h1seiproc PCIE h1seih45 h1isiham5 H1:ISI-HAM5_CPS_X_IPC_OUT
h1seiproc PCIE h1seih45 h1isiham5 H1:ISI-HAM5_CPS_Y_IPC_OUT
h1seiproc PCIE h1seih45 h1isiham5 H1:ISI-HAM5_SUSPOINT_OFI_L_OUT_IPC
h1seiproc PCIE h1seih45 h1isiham5 H1:ISI-HAM5_SUSPOINT_SR3_L_OUT_IPC
h1seiproc PCIE h1seih45 h1isiham5 H1:ISI-HAM5_SUSPOINT_SRM_L_OUT_IPC