Reports until 10:57, Wednesday 08 August 2018
H1 GRD
jenne.driggers@LIGO.ORG - posted 10:57, Wednesday 08 August 2018 - last comment - 11:18, Wednesday 08 August 2018(43322)
Guardian ISC_DRMI dead

The ISC_DRMI guardian died, and it's not really clear to me why.  I append to this alog the logfile.

I did a "guardctrl restart ISC_DRMI" in a terminal window, and we seem to be back and running.  Dave is looking more into why this might have happened.

 

2018-08-08_17:34:06.034404Z ISC_DRMI [PREP_DRMI_ASC.main] ezca: H1:SUS-IM4_M1_LOCK_Y => ONLY ON: INPUT, FM1, FM2, OUTPUT, DECIMATION
2018-08-08_17:34:06.353868Z ISC_DRMI [PREP_DRMI_ASC.main] ezca: H1:SUS-BS_M1_LOCK_P => ONLY ON: FM1, FM2, FM9, LIMIT, OUTPUT, DECIMATION
2018-08-08_17:34:06.651603Z ISC_DRMI [PREP_DRMI_ASC.main] ezca: H1:SUS-BS_M1_LOCK_Y => ONLY ON: FM1, FM9, LIMIT, OUTPUT, DECIMATION
2018-08-08_17:34:06.785028Z ISC_DRMI [PREP_DRMI_ASC.main] ezca: H1:SUS-PRM_M1_LOCK_P_SW1S => 20
2018-08-08_17:34:07.079318Z ISC_DRMI [PREP_DRMI_ASC.main] ezca: H1:SUS-PRM_M1_LOCK_P => ONLY ON: INPUT, FM1, FM9, LIMIT, OUTPUT, DECIMATION
2018-08-08_17:34:07.821424Z ISC_DRMI stopping daemon...
2018-08-08_17:34:07.821424Z ISC_DRMI daemon stopped.
2018-08-08_17:34:07.821424Z Traceback (most recent call last):
2018-08-08_17:34:07.821424Z   File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
2018-08-08_17:34:07.839717Z     "__main__", fname, loader, pkg_name)
2018-08-08_17:34:07.839717Z   File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
2018-08-08_17:34:07.840626Z     exec code in run_globals
2018-08-08_17:34:07.840626Z   File "/usr/lib/python2.7/dist-packages/guardian/__main__.py", line 287, in
2018-08-08_17:34:07.851505Z     main()
2018-08-08_17:34:07.851505Z   File "/usr/lib/python2.7/dist-packages/guardian/__main__.py", line 278, in main
2018-08-08_17:34:07.852405Z     guard.run()
2018-08-08_17:34:07.852405Z   File "/usr/lib/python2.7/dist-packages/guardian/daemon.py", line 450, in run
2018-08-08_17:34:07.852405Z     raise GuardDaemonError("worker exited unexpectedly, exit code: %d" % self.worker.exitcode)
2018-08-08_17:34:07.852405Z guardian.daemon.GuardDaemonError: worker exited unexpectedly, exit code: -11
2018-08-08_17:34:08.497836Z guardian@ISC_DRMI.service: Main process exited, code=exited, status=1/FAILURE
2018-08-08_17:34:08.498048Z guardian@ISC_DRMI.service: Unit entered failed state.
2018-08-08_17:34:08.498054Z guardian@ISC_DRMI.service: Failed with result 'exit-code'.

 

Comments related to this report
david.barker@LIGO.ORG - 11:18, Wednesday 08 August 2018 (43324)
Link

nothing obvious is showing up on h1guardian1 for this node. It appears to have failed after writing the FM settings for PRM_M1_LOCK_P but before/during changing the settings on the corresponding YAW channel (line 878 ISC_DRMI.py).