chandra.romel@LIGO.ORG - posted 18:03, Sunday 22 March 2020 (55729)
Drove down x-arm 5:45p to 6p PST
TITLE: 03/22 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 116Mpc
INCOMING OPERATOR: Patrick
SHIFT SUMMARY: Locked 4h50. I'm leaving Patrick with still elevated mircosiesm and an incoming EQ.
LOG:
TITLE: 03/22 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
OUTGOING OPERATOR: Camilla
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 13mph Gusts, 11mph 5min avg
Primary useism: 0.05 μm/s
Secondary useism: 0.47 μm/s
QUICK SUMMARY: Incoming 6.0 magnitude earthquake from El Salvador.
Got back to observing at 18:07 UTC. No issues.
We were in EQ mode for a 4.8M EQ from Canda. The ground had mostly calmed down when we lost lock, but the high microseism may have contributed.
There is a microseism SEI_CONF state but I'm not sure how that works with the new automatic SEI_ENV. Tagging SEI to ask what is currently the best state to be in when microseism is high? (Secondary useism: 0.38 μm/s)
TITLE: 03/22 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
OUTGOING OPERATOR: Jim
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 4mph Gusts, 2mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.38 μm/s
QUICK SUMMARY: Locked 35h20. Microseism has risen over the last 12 hours, currently just below our 90th percentile.
TITLE: 03/21 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC STATE of H1: Observing at 117Mpc INCOMING OPERATOR: Jim SHIFT SUMMARY: Remained locked and in observing the entire shift. No issues.
Have remained locked and in observing. No issues.
TITLE: 03/21 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: Patrick
SHIFT SUMMARY: Locked 19h20. Dave Barker and Carlos Perez were on site this morning.
LOG:
TITLE: 03/21 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 116Mpc
OUTGOING OPERATOR: Camilla
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 7mph Gusts, 5mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.19 μm/s
QUICK SUMMARY: No issues.
Locked 15h30. Microseism has risen to our 50th percentile.
Yesterday 3/19/2020 at 21:35 PST the switch in the CER (sw-lvea-aux) rebooted by itself affecting some of the digital cameras. Today 3/20/2020 at 8:40 PST I rebooted all 3 digital video servers and restored all cameras to a working state.
TITLE: 03/21 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
OUTGOING OPERATOR: Jim
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 6mph Gusts, 4mph 5min avg
Primary useism: 0.01 μm/s
Secondary useism: 0.14 μm/s
QUICK SUMMARY: Locked 11h30. Environment is good. The PSL cameras are blank but it seems like Dave knows about this from alog 55704.
I rebooted nuc21, PSL camera FOM images are back online.
Carlos is rebooting the digital camera servers to restore the gigabit-ethernet cameras which are down following the switch reboot yesterday evening.
Okay, that took me out of observing from 15:42 to 15:45 UTC.
All GIG-E cameras are operational again.
WP8572 h1tw0 raw minute data offload
TJ, Dave:
With H1 out-of-lock, I have started the h1tw0 offloading process. I have switched the active /trend/minute_raw to an empty directory set at 12:35 PDT, its data epoch is 12:39PDT. The data directory with the last 8 months of data was named /trend/minute_raw_1268768058.
h1tw0's minute trend data is served by h1nds0. The only NDS client of h1nds0 is the guardian system. Looking at the guardian user code and speaking with TJ, we think the only guardian NDS requests are for full data (taking averages over time periods from 1 to 10 seconds). Therefore I am not going to restart h1nds0 to serve recent minute-trend data from the temporary location /trend/minute_raw_1268768058 since restarting h1nds0 will have a greater impact on guardian than the loss of the last 8 months of raw minute trends.
I'm preparing h1fw0 to start the copy of the minute-raw files from h1tw0 to their final archival location on h1ldasgw0 compressed ZFS raid. The copy is expected to take the whole weekend to complete.
I started the file copy at 13:07. Current estimate is for completion in 26 hours (15:00 Saturday).
copy completed at 15:14 PDT.
The past few nights Corey has been notified that ALS_DIFF was unable to find IR. Looking back at the logs this node would only pass through the state "NO_IR_FOUND", and not stop. This is because it would return true regardless of the values that got it to this point and move onto IR_FOUND. In ISC_LOCK, the CHECK_IR state has a different test than what was done in ALS_DIFF. So there is an inconsistency of what is acceptable for the amount of IR light in DIFF.
By this time the IFO was on its way up after maintenance, so I just made two quick changes:
The long term solution will be to make the tests the same, and hopefully find a way to have it be more reliable. I'll to talk to others and see what we come up with.
Had to move both ETMY and TMSY a decent way to get light onto the camera (20 counts on the slider for ETMX Pit and 15 for TMSY Pit).
Had dragged TMSY too far away from it's normal position so that we were not getting enough light on the QPD's and had ALS_Y in fault with message "PZT trig servo not on". I think this is a different problem to the one disrupted by TJ in alog alog 54526 but it may be related.
Keita helped me to get light back on the QPDs:
From sitemap>ALS> End Y overview > QPD_A (FE) and QPD_B (FE), the NSUM should be ~1 for green arms locking, was closer to 0.05. after deciding that the suspensions had bee moved too much, we the QPD NSUMs and the alignment sliders and reverted them to their values at the time of last green arms locking, then did the same for the BIAS on the PZT1 and PZT2 PIT and YAW filters. NSUMS then came up to 1. Rs-adjusted the PZT filter bias numbers to be closer to their outputs and then continued locking.
We have just lost PRMI lock (not total lock) 3 times after I noticed that the FIND_IR TR_Y trace was way higher than it should be attached photo (20 rather than 1). Something strange must have happened so I will try and initial alignment.
DRMI locked straight away after a smooth initial alignment. Unsure what went on before!
At the start of this locking acquisition, there was no light on ALS-Y_QPD_B (NSUM ~ 0.05). ALSY PZT1 and PZT2 were then not adjusted. It can be seen from the top 2 plots of the attached image that they were being significantly adjusted at the start of each green arms locking to maximize QPD light. Keita helped me move the PZT1 and PZT2 values closer to the values which maximized NSUM in the past so in future they should not be so far away.
It's still not clear why there was no light on QPD_B to start with. The values of PZT1 and PZT2 do drift over time to give the maximum light on the QPDs so it's possible that to was just out of range. Maybe these biases should be checked and adjusted more regularly. Tagging ISC
This level microseism should not be a problem for the nominal configuration. It needs to be up around 1 micron/s to be an issue. If the microseism gets that high, we will have to do some work on the seismic automation, but we're unlikely to see that kind of ground motion this time of year.
Great, thank you Jim.