TITLE: 12/22 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 163Mpc
INCOMING OPERATOR: Tony
SHIFT SUMMARY: In Observing and have been Locked for 4.5 hours. One commissioning-caused lockloss during my shift(74976), but relocking went normally.
LOG:
00:00 Detector Commissioning and Locked for 4hours 45mins
00:16:32 Started broadband calibration measurement
00:22:46 Finished broadband calibration measurement
01:24 Lockloss caused by commissioning activities
-- Start Relocking --
02:10 Took us to DOWN to start an initial alignment due to us cycling through CHECK_MICH_FRINGES
03:23 Reached NOMINAL_LOW_NOISE
-- End Relocking --
03:27 Into Observing
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 00:42 | VAC | Gerardo | EX | n | Working on fan | 01:26 |
Currently Observing at 156Mpc and have been Locked for 1.5 hours. Nothing to note.
Jenne, Ryan, Oli, Keita, Louis We moved into theNEW_DARMguardian state once again with the following goals: 1.) update CAL-CS to match the new actuation path configuration to getDELTAL_EXTERNALto be roughly calibrated in the control room. 2.) restart the GDS pipeline with a pyDARM parameter file and GDS FIR filter set that matches the new DARM configuration 3.) sit in the now calibrated state for 20minutes to compare DARM noise in the new state vs the nominal and hopefully show that the noise better. Previously, we'd only been able to show that the non-stationarity was better (LHO:74939). 3.b) start a calibration sweep using Simulines using the retuned injections that I prepared by scaling the nominal injection amplitudes by the changes we've made to the DRIVEALIGN filters. 4.) Test theRETURN_TO_NLN_ETMYstate that Sheila and I prepared to move fromNEW_DARMback toNLN_ETMY. 5.) Test theDARM_RECOVERstate that I prepared to move us back to ETMX in the nominal configuration from ETMY. In short, we were able to move toNEW_DARMbut we could not get DELTAL_EXTERNAL calibrated by CAL-CS to make sense. We were able to push a new configuration to the GDS pipeline but similarly could not get that look right either. We ran out of time to debug further (we were already over time before needing to go back to Observing) so we started back tracking to the nominal DARM state. Instead of trusting all of our moves back to nominal to the guardian state, we ran commands one at a time to try and track down why we've not been having success recently. Jenne caught a few steps missing fromRETURN_TO_NLN_ETMY, still be implemented: 1.) we need to clear the ETMY L2 DRIVEALIGN L2L histories 2.) we need to make sure to adjust the LSC Output Matrix accordingly when moving back to ETMY Even after fixing these two issues we promptly lost lock when swapping the L3 LOCK L gains between ETMX and ETMY. So there's something else we haven't yet caught holding us up there. -- additional information regarding what was done to temporarily update the GDS pipeline, the re-retuned Simulines injections, and general front-end/CAL-CS weirdness that we encountered will come. == I restarted the GDS pipeline with the nominal calibration state and returned the CAL-CS settings to their nominal configuration while we relock back into the nominal configuration. We will leave the IFO in the nominal configuration over the holidays.
Lockloss @ 12/22 01:24UTC from commissioning activities. ETMX HEPI, stage 1, and stage 2 all tripped.
Back to Observing as of 03:27UTC
** This measurement was run while in NEW_DARM (74790) (with calibration lines turned off)
Broadband Measurements (20231222T001713Z)
2023/12/22 00:16:32 (1387239435) Broadband measurement started
2023/12/22 00:22:46 (1387239784) Broadband measurement finished
Output file: /ligo/groups/cal/H1/measurements/PCALY2DARM_BB/PCALY2DARM_BB_20231222T001713Z.xml
TITLE: 12/21 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning (locked at NLN)
INCOMING OPERATOR: Oli
SHIFT SUMMARY: Took some time to wrangle alignment changes before locking this morning, then moved into commissioning time, which is ongoing. H1 has been locked at low noise for 5 hours.
LOG:
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 16:06 | FAC | Karen | Optics Lab | - | Technical cleaning | 16:24 |
| 16:15 | FAC | Kim | H2 | - | Technical cleaning | 16:26 |
| 17:30 | FAC | Tyler, contractor | MY | - | 18:44 | |
| 21:20 | TCS | Tony | MER | - | Check TCS chillers | 21:36 |
TITLE: 12/22 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Commissioning
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 5mph Gusts, 3mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.22 μm/s
QUICK SUMMARY:
Detector is in Commissioning and has been Locked for 4hours 45 mins.
Naoki, Camilla
We scanned ADF demod phase with cold OM2 as we did with hot OM2 in 74691. The nominal ADF demod phase was 80 with hot OM2 and we tried to scan from 70, but the ADF servo was diverging with 70 demod phase, which means the ADF error signal is out of linear range. We found that the ADF demod phase around 100 would be good with cold OM2 so we scanned ADF demod phase from 90 to 110 using the same command in 74691 as shown in the attached figure. It seems that the ADF demod phase around 90 is optimal and we set the ADF demod phase to 91.
Laser Status:
NPRO output power is 1.827W (nominal ~2W)
AMP1 output power is 67.79W (nominal ~70W)
AMP2 output power is 137.2W (nominal 135-140W)
NPRO watchdog is GREEN
AMP1 watchdog is GREEN
AMP2 watchdog is GREEN
PMC:
It has been locked 15 days, 1 hr 53 minutes
Reflected power = 16.6W
Transmitted power = 109.8W
PowerSum = 126.4W
FSS:
It has been locked for 0 days 7 hr and 23 min
TPD[V] = 0.938V
ISS:
The diffracted power is around 1.9%
Last saturation event was 0 days 7 hours and 23 minutes
Possible Issues:
ISS diffracted power is low
Gabriele's brucos with OM2 cold showed that we seem to have a slight increse in coherence with DHARD Yaw. At Sheila's suggestion, I did a quick tune-up of the ITMY A2L coefficients during commissioning this afternoon (we've been locked for a little over 2.5 hours). The new (only very slightly different) values are in lscparams. Yaw moved away from center (was -1.5, now -1.65, where center is 0 for yaw), while I think pitch moved closer to center (was -0.15, now 0, where center is 0.85 for pitch).
H1 has dropped observing for commissioning time in coordination with L1. Expected to last ~3 hours.
"BSC high freq noise is elevated for these sensor(s)!!!
ITMX_ST2_CPSINF_H1
ETMX_ST1_CPSINF_V2 "
RyanS noted this morning that the temperature change in the LVEA seems to have caused some alignment shifts to optics in the corner station (first screenshot, from RyanS), and so he had to run an initial alignment this morning. I suspect that, since the temperature change primarily happened while we were locked, the ASC was able to follow along over the last day or so, however when we lost lock (due to the earthquake) and the ASC histories got cleared (as they do at every lockloss), the residual alignments were too poor for lock acquisition. After initial alignment, things seem to be going smoothly.
Out of an abundance of caution, I ran the rubbing checking script, and all the optics in the corner station seem to be fine, and clear of egregious rubbing (I didn't check the ETMs, TMSs, or FC2), and all consistent with the last time the script was run by RyanC in alog 74522. I think this means that there is no need for Facilities to change the setpoint of the LVEA. If the system can keep us where we are right now, then this seems to be a fine place to be.
I'll note that it does look like both RM1 and RM2 have a little bit of fuzz to their spectra between 7-9 Hz. This fuzz was there in RyanC's checks on the 1st of Dec, but is not there in the reference time from the 20th of Nov. We don't have vertical control over the RMs, so we can't try moving them to see if that alleviates the fuzz. Overall we seem to be doing fine, but that fuzz zone may be something to keep an eye on. While the rubbing script looks at the L, P, Y dofs, I had a look at the individual OSEMs. The fuzz seems to only show up in Length in the dof view, and that's consistent with my finding that it looks about the same in each individual OSEM (because it's cancelled out in the P and Y dofs). I attach in the second screenshot one representative individual OSEM to show the fuzz that I'm looking at (green is a reference from 20th Nov 2023, and blue is earlier today when the IFO was down, but the earthquake was mostly done).
Erik, Jonathan, Dave:
We upgraded cdslogin to deb12 at 09:45 PST this morning. This caused the EDC on h1susauxb123 go constantly restart. For a short term solution we removed all the channels being served by cdslogin from the H1EDC.ini channel list (lock-loss-alert and remote-access channels) and restarted the DAQ. The EDC is running stably now.
The loss of EDC channels caused a secondary Guardian issue, which is now resolved.
The h1cdssdf system would not run, again because it is connecting to cdslogin lock-loss-alert channels. We have temporarily removed these channels from the monitor.req and safe.snap files. This also caused a secondary Guardian problem which has now been resolved.
Current situation:
Alarms system is running
Lock loss alert system of offline.
EDC is missing all of its cdslogin channels
No slow channels from EDC in the DAQ between the times of 09:50:00 and 10:21:00
Erik is working on a deb11 container as a temporary solution to get the LLA code running again on cdslogin. Another possible solution is to move the code to another machine. We hope to get text and phone call alerting back online before tonight's operator owl shift.
Lock loss alerts are online again. I tested twilio texts and phone calls were working. I reset the settings to 09:15 by converting the safe.snap into a set of caput operations. At next TOO we will add the LLA channels back into h1cdssdf to put them back under SDF control.
The text/phone alert system is now running in a Debian 11 container. The remote access IOC is also running in a Debian 11 container.
A crash of the alert system, similar to the crash of the EDC, seen in Debian 12 hasn't recurred in Debian 11, so we should try re-attaching the EDC to these systems.
TITLE: 12/21 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
SEI_ENV state: EARTHQUAKE
Wind: 3mph Gusts, 2mph 5min avg
Primary useism: 0.39 μm/s
Secondary useism: 0.27 μm/s
QUICK SUMMARY:
A M6.3 earthquake from near the Aleutian Islands knocked H1 out of lock around an hour ago, and H1 has been properly waiting to try relocking until the seismic configuration leaves EQ mode. The alignment spot on the AS AIR camera looks quite bad, so I imagine I'll start with an initial alignment once the ground motion calms enough.
All other systems look good.
H1 back to observing at 19:20 UTC
Also, in case this helps with troubleshooting, note that when we lost lock between the ETMX -> ETMY gain swap, the ETMX HEPI, Stage 1, and Stage 2 watchdogs all tripped.