J. Kissel, S. Karki Sudarshan / Dripta have analyzed the data from the last test of PCALX vs. PCALY from back in early February (LHO aLOG 54873), and found that -- even with the factor of ~8 increase in SNR they gained from moving the usual 1153 Hz pair of lines down to 530 Hz -- the variation in the data over the 4 hour test was too large to determine the calibration discrepancy between the ends to a satisfactory precision. Thus, today, we try again, with the PCAL lines back down at 530 Hz, but with an increase in amplitude by a factor of 3 (and a flip-flip of frequency assignment). These lines will be in the data for ~4 hours, and have been installed in the observation ready segment starting at Mar 02 2020 22:13:35 UTC (1267222433). PCALX has been placed at 530.20 Hz, with amplitude 3.0*5007.0 = 15021.0 ct_pk. PCALY has been placed at 530.10 Hz, with amplitude 3.0*3619.0 = 10857.0 ct_pk. With these excitation amplitudes -- and the other "normal" PCALY calibration lines running at 17.1, 410.3, and 1083.7 as well as pulsar injections -- the RMS excitation value is 16021.6 ct_RMS for PCALX and 9189.13 ct_RMS for PCALY. The RMS for PCALX when high frequency roaming line is at 3001.3 Hz, is 21507 ct_RMS, so theoretically there is more head room to increase the amplitude more, if need be, but at the time we don't find it necessary, and we do quite enjoy this kind of head room we have before saturation. Attached are screenshots of the SDF differences that were accepted to make this temporary change, and an ASD and RMS of the total requested excitation signal coming out of each PCAL. Note -- in order to obtain this increase, we had to completely turn OFF the high frequency roaming PCALX line (which is currently at 3001.3 Hz -- but given that we've had a beautiful 70+ hour lock stretch, we won't miss these 4 hours).
Attached is a screenshot of the PCAL oscillator MEDM screens during this test.
These lines have been switched back to X=1153.1 and Y=1153.2 Hz at 2020-03-03 02:14:25 UTC, and we're back in observing by 2020-03-03 02:17:26 UTC. We have data at 530.1 and 530.2 for approx for 4 hours.
J. Kissel Full suite of standard calibration measurements on the ETMX actuator and the sensing function, again hoping to reduce the uncertainty on the unknown systematic error, like last week (see LHO aLOG 55262 and latest estimate of uncertainty [which does not yet include the last two weeks of measurement,] LHO aLOG 55055). Detector in its nominal O3B configuration with 38 W input to the IMC (33.6 W in to PRM), a 50ct SRCL offset in order to "retune" the sensing function. (SQZ engaged, medium-to-low wind and medium mircoseism.) Raw DTT templates with the data has been committed and exported here: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Measurements/FullIFOSensingTFs 2020-03-02_H1_DARM_OLGTF_LF_SS_5to1100Hz_15min.xml 2020-03-02_H1_PCALY2DARMTF_LF_SS_5t1100Hz_10min.xml 2020-03-02_H1_PCALY2DARMTF_BB_3min.xml << starts at 2020-03-02 19:00:32 UTC (for offline GDS processing.) /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Measurements/FullIFOActuationTFs 2020-03-02_H1SUSETMX_L1_iEXC2DARM_8min.xml 2020-03-02_H1SUSETMX_L1_PCAL2DARM_5min.xml 2020-03-02_H1SUSETMX_L2_iEXC2DARM_12min.xml 2020-03-02_H1SUSETMX_L2_PCAL2DARM_6min.xml 2020-03-02_H1SUSETMX_L3_iEXC2DARM_12min.xml 2020-03-02_H1SUSETMX_L3_PCAL2DARM_6min.xml Will get to including these and other data sets in the uncertainty budget when I can.
I've added some lights to TJ's REMOTE_OPERATOR screen to show who has notifications turned on. At this point it just shows which operators have global alerts enabled, not if the other settings are set properly (like start/stop time, which contact, all hours) so the light being on shouldn't be interpreted showing the full alert status. Attached screenshot shows the yellow lights that come on when the CONTACT ENABLE bit is not 0 as it is now, Jeff B, Camilla and Corey have alerts enabled.
Also, Travis and Niko don't have slots in the alert system, so they have little black bars to note that. This screen will have to be updated when they get added. There's a bunch of stuff could be added to this screen to make it more useful.
We have channels! Everything looks as well as could be expected. There's a channel in the chiller trends that seems to be on an upward slope. I'm not sure what this is about.
<b>TITLE:</b> 03/02 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
<b>STATE of H1:</b> Observing at 113Mpc
<b>OUTGOING OPERATOR:</b> Patrick
<b>CURRENT ENVIRONMENT:</b>
SEI_CONF state: WINDY
Wind: 15mph Gusts, 13mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.31 μm/s
<b>QUICK SUMMARY:</b>
TITLE: 03/02 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 119Mpc
INCOMING OPERATOR: Patrick
SHIFT SUMMARY:We've had a BBH candidate tonight! https://gracedb.ligo.org/superevents/public/O3/
LOG: Locked 57hours
Locked 53h15. All quiet.
It seems like the node that casued us to drop out of observing was due to an automatic SEI_CONF change EQ>WINDY>EQ, maybe due to EQ being requested before we were fully transitioned to WINDY. Details follow:
Looking at the logs, it looks like the HPI_BS_SC node may have stalled, which could be due to a known bug in the sensor correction parts of the simulink models. The ISIs have been restarted to address the bug, but the HEPIs may not have been. I need to get some help from Dave to determine if the HEPIs are running the most recent code or not, but it's possible we have a way to prevent this happening down the road.
This was actually a case where the worker for the Guardian node itself was terminated and then restarted. I don't why the worker timed out, but recovery was quick.
TITLE: 03/02 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 119Mpc
OUTGOING OPERATOR: Jim
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 17mph Gusts, 14mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.25 μm/s
QUICK SUMMARY: Some 20mph wind but otherwise everything looks good. Locked 49h20!
TITLE: 03/02 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: Camilla
SHIFT SUMMARY: Quiet Shift
LOG:
Nothing to report.
TITLE: 03/01 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: Patrick
SHIFT SUMMARY: Quiet evening, received one GRB Short notification. Locked 33h15
Observing at 119Mpc with some 20mph winds. Locked 29h20.
TITLE: 03/01 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 118Mpc
OUTGOING OPERATOR: Jim
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 15mph Gusts, 12mph 5min avg
Primary useism: 0.05 μm/s
Secondary useism: 0.31 μm/s
QUICK SUMMARY: Locked 25h30
TITLE: 02/29 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: Camilla
SHIFT SUMMARY: Quiet, but windy
LOG:
Nothing happened
TITLE: 02/29 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY: Quiet evening with some wind-caused range drops
LOG:
The earthquake we had last night was a very successful test of the SEI_ENV automated earthquake response. Attached trend gives the sequence of events. SEI_ENV state is the top left, SEI_CONF is top right, eq band ground is bottom left, ISC_LOCK state is bottom right.
Look like seismon/SEI_ENV got the notification just before the P/S wave arrival (where top left goes from 10 to 20), but that wasn't big enough to be a problem.
SEI_ENV switched SEI_CONF to the earthquake mode as the R waves started rolling in and the IFO actually stayed locked through the peak (at -33000 seconds), we didn't lose lock until ~400 seconds after, maybe from an aftershock? At least it wasn't from the peak or the SEI_CONF transition.
The time to transition back to nominal also seems pretty reasonable. This was a pretty good size earthquake, we probably haven't ridden out anything bigger to this point.
There are still edge cases to be thought about: the test for the SEI_CONF transitions are kind of tricked by very high winds and microseism. Down the road we may have to consider pausing SEI_ENV during big windstorms, to avoid spurious transitions. Still, great success.
SEI_ENV also worked great this evening! Switching us to EQ mode and back, keeping lock.
The 2020-02-28 08:34:15 UTC lockloss does seem strange though that we survived the massive 6000 peakmon peak and lost lock 8 minutes after plot here. Have you seen that before? There is also one of those quick (100-500Hz) DARM spikes 1s before the lockloss as you can see in the bottom of this zoomed plot.