Pursuant to LIGO-T1900555 [1] and discussions on the DetChar and HW injections mailing list [2,3], I've scheduled DetChar safety injections to begin at 21:00:00 UTC (16:00:00 CDT, 14:00:00 PDT) 3 Sep 2019. These should last for 390 seconds and contain 75 injections spaced 5 sec apart. The scheduled injections have been added to GraceDb [4]. We will update the events in GraceDb to record whether these injections were successful post-hoc. **Please note**, we have schedule coincident (but not necessarily coherent) injections at LLO for the same time. We include the LLO aLOG describing those injections in a comment below. More details on how these injections were generated and added to the HW injection SVN repo are available via a README [5]. [1] https://dcc.ligo.org/LIGO-T1900555 [2] detchar@ligo.org [3] hw-injections@ligo.org [4] https://gracedb.ligo.org/search/?query=Test+INJ+gpstime%3A+1251579618+..+1251580100+instruments%3A+%22H1%22&query_type=E&results_format=S [5] https://ldas-jobs.ligo.caltech.edu/~detchar/hwinj/LIGO-T1900555/README
Neither IFO was up by this time, so I removed this injection from the schedule file.
Injection failed. Guardian log below:
2019-09-03_22:37:31.316944Z INJ_TRANS RELOAD requested. reloading system data...
2019-09-03_22:37:31.360492Z INJ_TRANS module path: /opt/rtcds/userapps/release/cal/common/guardian/INJ_TRANS.py
2019-09-03_22:37:31.361022Z INJ_TRANS user code: /opt/rtcds/userapps/release/cal/common/guardian/injtools/__init__.py
2019-09-03_22:37:31.361022Z INJ_TRANS user code: /opt/rtcds/userapps/release/cal/common/guardian/injtools/inj_det.py
2019-09-03_22:37:31.361022Z INJ_TRANS user code: /opt/rtcds/userapps/release/cal/common/guardian/injtools/inj_io.py
2019-09-03_22:37:31.361022Z INJ_TRANS user code: /opt/rtcds/userapps/release/cal/common/guardian/injtools/inj_types.py
2019-09-03_22:37:35.800454Z INJ_TRANS RELOAD complete
2019-09-03_22:37:35.801803Z INJ_TRANS W: RELOADING @ WAIT_FOR_NEXT_INJECT.run
2019-09-03_22:55:00.027528Z INJ_TRANS [WAIT_FOR_NEXT_INJECT.run] ezca: H1:CAL-INJ_TINJ_START => 1251586518.03
2019-09-03_22:55:00.028168Z INJ_TRANS [WAIT_FOR_NEXT_INJECT.run] ezca: H1:CAL-INJ_TINJ_OUTCOME => 0
2019-09-03_22:55:00.148272Z INJ_TRANS EDGE: WAIT_FOR_NEXT_INJECT->CHECK_SCHEDULE_TIMES
2019-09-03_22:55:00.148896Z INJ_TRANS calculating path: CHECK_SCHEDULE_TIMES->INJECT_SUCCESS
2019-09-03_22:55:00.149298Z INJ_TRANS new target: CREATE_AWG_STREAM
2019-09-03_22:55:00.150149Z INJ_TRANS executing state: CHECK_SCHEDULE_TIMES (35)
2019-09-03_22:55:00.151587Z INJ_TRANS [CHECK_SCHEDULE_TIMES.main] USERMSG 0: INJECTION IMMINENT: 1251586818.000000
2019-09-03_22:55:00.152104Z INJ_TRANS [CHECK_SCHEDULE_TIMES.main] Skipping checking schedule times since its already been done.
2019-09-03_22:55:00.265266Z INJ_TRANS EDGE: CHECK_SCHEDULE_TIMES->CREATE_AWG_STREAM
2019-09-03_22:55:00.271061Z INJ_TRANS calculating path: CREATE_AWG_STREAM->INJECT_SUCCESS
2019-09-03_22:55:00.273132Z INJ_TRANS new target: READ_WAVEFORM
2019-09-03_22:55:00.274979Z INJ_TRANS executing state: CREATE_AWG_STREAM (50)
2019-09-03_22:55:00.275920Z INJ_TRANS [CREATE_AWG_STREAM.enter]
2019-09-03_22:55:00.277244Z INJ_TRANS [CREATE_AWG_STREAM.main] ezca: H1:CAL-INJ_TINJ_TYPE => 3
2019-09-03_22:55:00.391173Z INJ_TRANS EDGE: CREATE_AWG_STREAM->READ_WAVEFORM
2019-09-03_22:55:00.391774Z INJ_TRANS calculating path: READ_WAVEFORM->INJECT_SUCCESS
2019-09-03_22:55:00.391774Z INJ_TRANS new target: RAMP_GAIN_TO_1
2019-09-03_22:55:00.392868Z INJ_TRANS executing state: READ_WAVEFORM (60)
2019-09-03_22:55:00.395535Z INJ_TRANS [READ_WAVEFORM.main] Reading waveform data from /hwinj/Details/detchar/detchar-hwinj-schedule_LIGO-T1900555-PROPOSED_H1_INJECTIONS-1251586818-390.txt
2019-09-03_22:55:27.569259Z INJ_TRANS EDGE: READ_WAVEFORM->RAMP_GAIN_TO_1
2019-09-03_22:55:27.570014Z INJ_TRANS calculating path: RAMP_GAIN_TO_1->INJECT_SUCCESS
2019-09-03_22:55:27.570014Z INJ_TRANS new target: AWG_STREAM_OPEN_PREINJECT
2019-09-03_22:55:27.570974Z INJ_TRANS executing state: RAMP_GAIN_TO_1 (65)
2019-09-03_22:55:27.581834Z INJ_TRANS [RAMP_GAIN_TO_1.enter]
2019-09-03_22:55:27.583248Z INJ_TRANS [RAMP_GAIN_TO_1.main] ezca: H1:CAL-INJ_TRANSIENT_GAIN => 1.0
2019-09-03_22:55:29.829405Z INJ_TRANS EDGE: RAMP_GAIN_TO_1->AWG_STREAM_OPEN_PREINJECT
2019-09-03_22:55:29.830281Z INJ_TRANS calculating path: AWG_STREAM_OPEN_PREINJECT->INJECT_SUCCESS
2019-09-03_22:55:29.831068Z INJ_TRANS new target: INJECT_CBC_ACTIVE
2019-09-03_22:55:29.832679Z INJ_TRANS executing state: AWG_STREAM_OPEN_PREINJECT (70)
2019-09-03_22:55:29.838744Z INJ_TRANS [AWG_STREAM_OPEN_PREINJECT.main] USERMSG 0: INJECTION IMMINENT: 1251586818.000000
2019-09-03_22:59:40.085443Z INJ_TRANS JUMP target: INJECT_DETCHAR_ACTIVE
2019-09-03_22:59:40.086053Z INJ_TRANS [AWG_STREAM_OPEN_PREINJECT.exit]
2019-09-03_22:59:40.150495Z INJ_TRANS JUMP: AWG_STREAM_OPEN_PREINJECT->INJECT_DETCHAR_ACTIVE
2019-09-03_22:59:40.151051Z INJ_TRANS calculating path: INJECT_DETCHAR_ACTIVE->INJECT_SUCCESS
2019-09-03_22:59:40.151051Z INJ_TRANS new target: RAMP_GAIN_TO_0
2019-09-03_22:59:40.152161Z INJ_TRANS executing state: INJECT_DETCHAR_ACTIVE (104)
2019-09-03_22:59:40.153871Z INJ_TRANS [INJECT_DETCHAR_ACTIVE.main] USERMSG 0: INJECTION ACTIVE: 1251586818.000000
2019-09-03_22:59:40.154308Z awgSetChannel: awg_clnt[42][0] = NULL
2019-09-03_22:59:40.154308Z Error code from awgSetChannel: -5
2019-09-03_22:59:40.170270Z INJ_TRANS [INJECT_DETCHAR_ACTIVE.main] File "/opt/rtcds/userapps/release/cal/common/guardian/INJ_TRANS.py", line 572, in main
2019-09-03_22:59:40.170270Z self.hwinj.stream.send(self.hwinj.data)
2019-09-03_22:59:40.170928Z INJ_TRANS [INJECT_DETCHAR_ACTIVE.main] File "/usr/lib/python2.7/dist-packages/awg.py", line 621, in send
2019-09-03_22:59:40.170928Z self.append(data, scale=scale)
2019-09-03_22:59:40.170928Z INJ_TRANS [INJECT_DETCHAR_ACTIVE.main] File "/usr/lib/python2.7/dist-packages/awg.py", line 599, in append
2019-09-03_22:59:40.170928Z self.open()
2019-09-03_22:59:40.170928Z INJ_TRANS [INJECT_DETCHAR_ACTIVE.main] File "/usr/lib/python2.7/dist-packages/awg.py", line 584, in open
2019-09-03_22:59:40.170928Z + ": " + awgbase.SIStrErrorMsg(ret))
2019-09-03_22:59:40.170928Z INJ_TRANS [INJECT_DETCHAR_ACTIVE.main] <class 'awg.AWGStreamError'> can't open stream to H1:CAL-INJ_TRANSIENT_EXC: Error setting up an awg slot for the channel
2019-09-03_22:59:40.197166Z INJ_TRANS JUMP target: FAILURE_DURING_ACTIVE_INJECT
2019-09-03_22:59:40.197823Z INJ_TRANS [INJECT_DETCHAR_ACTIVE.exit]
2019-09-03_22:59:40.261389Z INJ_TRANS JUMP: INJECT_DETCHAR_ACTIVE->FAILURE_DURING_ACTIVE_INJECT
2019-09-03_22:59:40.262094Z INJ_TRANS calculating path: FAILURE_DURING_ACTIVE_INJECT->INJECT_SUCCESS
2019-09-03_22:59:40.262094Z INJ_TRANS new target: WAIT_FOR_NEXT_INJECT
2019-09-03_22:59:40.270095Z INJ_TRANS executing state: FAILURE_DURING_ACTIVE_INJECT (300)
2019-09-03_22:59:40.272312Z INJ_TRANS [FAILURE_DURING_ACTIVE_INJECT.enter]
2019-09-03_22:59:40.273597Z INJ_TRANS [FAILURE_DURING_ACTIVE_INJECT.main] ezca: H1:CAL-INJ_TRANSIENT_GAIN => 0.0
2019-09-03_22:59:42.405392Z INJ_TRANS [FAILURE_DURING_ACTIVE_INJECT.main] ezca: H1:CAL-INJ_TINJ_OUTCOME => -4
2019-09-03_22:59:42.405950Z INJ_TRANS [FAILURE_DURING_ACTIVE_INJECT.main] ezca: H1:CAL-INJ_TINJ_ENDED => 1251586800.41
2019-09-03_22:59:42.451041Z INJ_TRANS [FAILURE_DURING_ACTIVE_INJECT.run] USERMSG 0: ERROR
Was actually the 2300UTC injection from this alog (alog 51684) that failed. The 2100 UTC injection was removed from the schedule.