H1 GRD

corey.gray@LIGO.ORG - posted 17:38, Friday 02 December 2016 - last comment - 19:25, Friday 02 December 2016(32131)

GRB Verbal Alarm 23:13utc

At 23:13 we received a "Gamma Ray Burst" Alarm.

Following L1500117:

Step 1: Checked to see if this is a TEST
- What does this mean? Do Tests have the "INJ" flags? Are there any other types of flags for Tests?
- On GraceDB, there was a little confusion. We could see a (1) GRB Report (EM_COINC) from Swift, but (2) around the same time, there were also some INJ EM_COINC's from CWB (!). The GRB's event time was over 2min after the EM_COINC's.
- So was NOT a TEST
Step 2: Stand Down Time from 23:13 - 0:13utc
Step 3: Finally talked with LLO. They did not know about the GRB, so they went to go check. Heard them say they DID NOT receive audible alarm, but had "text notification".

THEN:

AT 23:21 We were bumped out of OBSERVING!! (Sheila noticed it, we didn't hear it for some reason during all the hub bub of figuring out the deluge of GraceDB reports.) It wasn't obvious why we were bumped out of OBSERVING. But since we tend to get bumped out due to SDF Diffs/changes, JimW looked at the DIAG_SDF node's log. On here he noticed we were getting DIFFS for the SDF NODE: sysecatx1plc2

Unfortunately, when the diffs appeared they only happened for a few seconds (knocked us out of OBSERVING), and then go away. So it was hard to see what channels were in question. It happened several times, and managed to catch a glance at the channels in question. Words we saw in the channel were: ALS, Fiber, polarization. Since they deal with the ALS, we can probably safely say we can NOT_MONITOR these channels (but we should get a blessing from Keita). Or if we are in a fix in the middle of the night, and you are able to figure out who these elusive channels are, be sure to note the channels, NOT_MONITOR them for the night and then alog what you did.

Epilogue:

We ended up losing lock at 23:56--it's really noisy seismically with useism & wind.

Comments related to this report

keita.kawabe@LIGO.ORG - 19:25, Friday 02 December 2016 (32134)

Link

Regarding strange sdf error of h1sysecatx1plc2:

I used a brute force method of copying OBSERVE.snap file, stripped unnecessary information including but not limited to non-ALS channels, divide into 20 line chunks, and fed them to lockloss tool for [-10,+10] seconds window centered at the first time when IFO was kicked out (GPS 1164756098).

There are three channels that changed 2 seconds prior to the event (first attachment):

H1:ALS-X_FIBR_SERVO_IN1GAIN

H1:ALS-X_FIBR_LOCK_TEMPERATURECONTROLS_ERRORSIGNAL

H1:ALS-X_FIBR_LOCK_TEMPERATURECONTROLS_POLARITY

Based on this, I looked at the fiber PLL lock status and sure enough, there was a large glitch (smaller peak on ch3 of the second attachment) in PLL. After 3 seconds or so the autolocker started to "relock" by lowering the gain and such, there's a huge swing in the beat note, and it relocked. No suspicious thing in polarization, RF and DC level during this.

It's not clear why this happened, but this is just the end station PLL that is not used for anything during OBSERVE, and there's no sign that there's an RF disaster going on at the end station.

I went ahead and unmonitored these three channels in OBSERVE in sdf.

I didn't unmonitor ALL end station ALS channels.

If this happens next time for other X end ALS channels:

Go to /ligo/home/keita.kawabe/Lockloss
for ii in test*.txt; do lockloss -c ${ii} plot -w '[-10,10]' gpstime; done
(gpstime is the time the IFO was kicked out of observation.)
Plot window opens, you note which channel changed, close the plot, and the next window opens. You'll go through 14 plot windows.
Go to sdf screen of H1SYSECATX1PLC2SDF, and look at the unmonitored channel list.
The channel that changed that is not in the unmonitored list is the new offender. Change the sdf view to all, find the offending one, click 'MON', and accept.
Back to OBSERVE.
Make alog.

Images attached to this comment