This afternoon I upgraded the guardian core to r1020, and cdsutils to r250. There are a couple important new guardian features that I wanted to pull in:
I then restarted all nodes with "guardctrl restart --all". All SUS nodes had been shut down for the moment, and all SEI nodes are in watchdog trip states, so this didn't do too much. Some nodes that don't specify initial request states came up with the new NONE request. I set manually reset the requests for those nodes back to what they were before the restart. No major issues:
node s m n vers request state message
---- - - - ---- ------- ----- -------
ALS_COMM * E ! 1020 DOWN DOWN not ready for handoff
ALS_DIFF * E - 1020 DOWN DOWN -
ALS_XARM * E - 1020 UNLOCKED UNLOCKED -
ALS_YARM * E - 1020 UNLOCKED UNLOCKED -
H1ECATC1PLC2 * E - 1020 OP OP -
H1ECATX1PLC2 * E - 1020 OP OP -
H1ECATY1PLC2 * E - 1020 OP OP -
H1ISCEX * E - 1020 OP OP -
H1ISCEY * E - 1020 OP OP -
H1LSC * E - 1020 OP OP -
H1LSCAUX * E - 1020 OP OP -
HPI_BS * E ! 1020 ROBUST_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
HPI_ETMX * M - 1020 ROBUST_ISOLATED ROBUST_ISOLATED -
HPI_ETMY * M - 1020 ROBUST_ISOLATED ROBUST_ISOLATED -
HPI_HAM2 * E ! 1020 ROBUST_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
HPI_HAM3 * E ! 1020 ROBUST_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
HPI_ITMX * E ! 1020 ROBUST_ISOLATED WATCHDOG_TRIPPED_DAMPING WATCHDOG TRIP: DAMPING_ONLY (3)
HPI_ITMY * M ! 1020 ROBUST_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
IMC * E ! 1020 DOWN FAULT PSL periscope PD error (low light?)
ISI_BS_ST1 * E ! 1020 HIGH_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
ISI_BS_ST2 * E ! 1020 HIGH_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
ISI_ETMX_ST1 * M - 1020 HIGH_ISOLATED HIGH_ISOLATED -
ISI_ETMX_ST2 * M - 1020 HIGH_ISOLATED HIGH_ISOLATED -
ISI_ETMY_ST1 * M - 1020 HIGH_ISOLATED HIGH_ISOLATED -
ISI_ETMY_ST2 * M - 1020 HIGH_ISOLATED HIGH_ISOLATED -
ISI_HAM2 * E ! 1020 HIGH_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
ISI_HAM3 * E ! 1020 HIGH_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
ISI_HAM4 o - - - - - -
ISI_HAM5 o - - - - - -
ISI_HAM6 o - - - - - -
ISI_ITMX_ST1 * E ! 1020 HIGH_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
ISI_ITMX_ST2 * E ! 1020 HIGH_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
ISI_ITMY_ST1 * M - 1020 HIGH_ISOLATED DAMPED -
ISI_ITMY_ST2 * M ! 1020 HIGH_ISOLATED WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
LSC * E - 1020 NONE INIT -
SEI_BS * E ! 1020 ISOLATED_DAMPED WATCHDOG_TRIPPED_DACKILL WATCHDOG TRIP: DACKILL
SEI_ETMX * E - 1020 FULLY_ISOLATED FULLY_ISOLATED -
SEI_ETMY * E - 1020 FULLY_ISOLATED FULLY_ISOLATED -
SEI_HAM2 * E ! 1020 ISOLATED WATCHDOG_TRIPPED_DACKILL WATCHDOG TRIP: DACKILL
SEI_HAM3 * E ! 1020 ISOLATED WATCHDOG_TRIPPED_DACKILL WATCHDOG TRIP: DACKILL
SEI_ITMX * E ! 1020 FULLY_ISOLATED WATCHDOG_TRIPPED_DACKILL WATCHDOG TRIP: DACKILL
SEI_ITMY * E ! 1020 FULLY_ISOLATED WATCHDOG_TRIPPED_SUBORDINATE WATCHDOG TRIP: SUBORDINATE
SUS_BS o - - - - - -
SUS_ETMX o - - - - - -
SUS_ETMY o - - - - - -
SUS_ITMX o - - - - - -
SUS_ITMY o - - - - - -
SUS_MC1 o - - - - - -
SUS_MC2 o - - - - - -
SUS_MC3 o - - - - - -
SUS_OMC o - - - - - -
SUS_PR2 o - - - - - -
SUS_PR3 o - - - - - -
SUS_PRM o - - - - - -
SUS_SR2 o - - - - - -
SUS_SR3 o - - - - - -
SUS_SRM * E - 1020 DAMPED DAMPED -
SUS_TMSX o - - - - - -
SUS_TMSY o - - - - - -