I upgraded the guardian core and cdsutils (python ezca) installs:
- guardian r1095: better error handling, minor bug fixes
- cdsutils r359: NDS interface improvements (handle channels without IFO prefix), fix avg function, add filter module decoding utility (cdsutils sfm)
I restarted most of the nodes, except for the following:
jameson.rollins@opsws2:~/src/cdsutils/trunk 0$ guardctrl list | grep -v 1095
node s k m vers state message
---- - - - ---- ----- -------
ALS_COMM o - - - - -
H1ECATC1PLC2 * E 1076 INIT
H1ECATX1PLC2 * E 1076 INIT
H1ECATY1PLC2 * E 1076 INIT
H1ISCEX * E 1076 INIT
H1ISCEY * E 1076 INIT
H1LSC * E 1076 INIT
H1LSCAUX * E 1076 INIT
HPI_HAM2 * * M 1083 ROBUST_ISOLATED
IAS_INPUT o - - - - -
IAS_MICH o - - - - -
IAS_PRC o - - - - -
IAS_SRC o - - - - -
IAS_TEST o - - - - -
IAS_XARM o - - - - -
IAS_YARM o - - - - -
IFO_ALIGN o - - - - -
ISC_LOCK o - - - - -
ISI_HAM2 * P 1083 WATCHDOG_TRIPPED_FULL_SHUTDOWN WATCHDOG TRIP: FULL_SHUTDOWN (4)
LSC * E 1076 INIT
LSC_PRMI_VAR_FINESSE * E 1083 DOWN MISALIGN PRM
LSC_PRX * P 1083 FAULT check LSC input matrix, ITMY not MISALIGNED
LSC_PRY * P 1080 FAULT PRM not ALIGNED, ITMX not MISALIGNED
SEI_HAM2 * P 1083 ISOLATED
SUS_MC1 * * E 1080 ALIGNED
SUS_MC3 * * E 1080 ALIGNED
SUS_PR3 * * E 1080 ALIGNED
SUS_PRM * * M 1083 ALIGNED
The non-running nodes (s='o') had various errors on restart, so I shut them down until they can be fixed. I shut down all the IAS nodes since I was told they're not in use. The HAM2 nodes were not restarted because Hugh was in the middle of doing some SEI work on HAM2. The nodes starting with "H1" are the "CSDEF" nodes (not sure what the status of those is). I'll restart these nodes when I'm sure it's ok.
I noticed that there are a couple of "hidden" nodes, that are not listed in the Guardian overview screen:
H1ECATC1PLC2 * E 1076 INIT
H1ECATX1PLC2 * E 1076 INIT
H1ECATY1PLC2 * E 1076 INIT
H1ISCEX * E 1076 INIT
H1ISCEY * E 1076 INIT
H1LSC * E 1076 INIT
H1LSCAUX * E 1076 INIT LSC * E 1076 INIT
LSC_PRMI_VAR_FINESSE * E 1083 DOWN MISALIGN PRM
LSC_PRX * P 1083 FAULT check LSC input matrix, ITMY not MISALIGNED
LSC_PRY * P 1080 FAULT PRM not ALIGNED, ITMX not MISALIGNED
We should be very careful about these. If they're being used, they should be on the overview. If they're not being used, they should be shut down. Do not leave running nodes off of the overview screen
I should note that a bug was fixed in the cdsutils avg function. The standard deviation calculations were bogus, off by some unknown but large factor. This new version fixes that issue so that the standard deviation calculations returned with the "stdev" option are now correct.
https://redoubt.ligo-wa.caltech.edu/svn/cdsutils/trunk/lib/cdsutils/avg.py
We have reverted the cdsutils to cdsutils-329 (the version installed here since september), we did this by changing the link in /ligo/apps/linux-x86_64
The guardians were failing when they called the new avg function, we tried using the new cds utils function from the command line but it doesn't seem to be working.