TITLE: 01/16 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 156Mpc
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 21mph Gusts, 13mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.27 μm/s
QUICK SUMMARY:
IFO Locked for 24 hours!
Planned Comissioning time today from 16:30- 19:30 UTC where we will drop from observing for comissioning and calibration activities.


Alarm handler:
PSL Dust 101 & 102

Red but not actively alarming:
Vacuum alert: H0:VAC-LX_Y0_PT110_MOD1_PRESS_TORR
Trending this channel back it looks like its been red for days, and this Alog mentions that its not currently running

At 17:48:23 Wed 15jan2025 PST all end station receivers of long-range-dolphin IPC channels originating from h1lsc saw a single IPC receive error.

The models h1susetm[x,y], h1sustms[x,y] and h1susetm[x,y]pi all receive a single channel from h1lsc and recorded a single receive error at the same time. No other end station models receive from h1lsc.

On first investigation there doesn't appear to be anything going on with h1lsc at this time to explain this.

TITLE: 01/16 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 148Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:

IFO is in NLN and OBSERVING as of 14:57 UTC (15 hr lock!)

Ultra-smooth shift - nothing of note.

LOG:

None

TITLE: 01/16 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 158Mpc
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY:
Great Day of Observing!
Dropped down to Commissioning twice:
Once due to an earthquake causing the SQZr to  lock loss- but we stayed in NLN.
And another time due to an Xtreme PI Damping event. Once again we stayed in NLN.

LOG:

TITLE: 01/16 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 159Mpc
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 5mph Gusts, 3mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.34 μm/s
QUICK SUMMARY:

IFO is in NLN and OBSERVING as of 14:57 UTC (9 hr lock!)

 23:39:06 UTC Xtreme PI damping took us out of Observing.
We got back into Observing at  23:40:06 UTC

TITLE: 01/15 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 159Mpc
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 5mph Gusts, 3mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.35 μm/s
QUICK SUMMARY:

Dropped from Observation at 15:22:22:40 UTC due to the SQZ system got unlocked. We did not drop out of Nominal_Low_Noise.
Looking slightly more into this, It looks like a 5.7Mag Eathquake may have unlocked the SQZ system.

As an update to 80919, I have finished editing all*  plotallsus_tfs.m scripts to now include the plotting of cross-coupling. There is an on/off boolean along with a matrix near the top of each script that allows you to choose whether or not to plot cross coupling, and between which DOF.

This can help when we want to check for cross-coupling before vs after a period of time/vent/etc.

* When I say all I mean every sus matlab script whose name is a variation of plotallsus_tfs.m or plotallsus_tfs_M1.m. I did not update the plotallsus_spectra.m or plotallsus_tfs_{some other stage}.m since those are not used as often. All changes verified to work and committed to svn.

This comparison was suggested to help evaluate whether the narrow line contamination seen around violin mode regions during ring-ups is related to aliasing. The idea (as I follow it) is that because the 64 kHz channel has different aliasing, the noise ought to look different in the two channels if it is aliasing-related. In fact the two channels look similar, albeit not identical, which looks like evidence against the aliasing hypothesis. However, this test may not catch all the places where aliasing could occur, so it may not be conclusive. See the associated detchar-request issue for ongoing discussion.

The first two plots are from 2023, during time periods which were previously identified as having line contamination around the violin modes (71800, 79825). The last plot is from just a few days ago, on Jan 12.

Wed Jan 15 10:06:29 2025 ALERT: Fill done (errors) in 6min 25sec

TCs started very high, around +40C, and therefore TCmins [-28C, -27C] did not exceed the trip temp (-60C). After the TCs had flatlined I manually cancelled the fill.

OAT (-1C, 31F) and foggy.

Diag Main has now got a PSL_Chiller message telling me to Check the PSL Chiller.
I ran the PSL Status script to hopefully get some insight.

Laser Status:
    NPRO output power is 1.842W
    AMP1 output power is 70.15W
    AMP2 output power is 137.0W
    NPRO watchdog is GREEN
    AMP1 watchdog is GREEN
    AMP2 watchdog is GREEN
    PDWD watchdog is GREEN

PMC:
    It has been locked 28 days, 23 hr 15 minutes
    Reflected power = 25.55W
    Transmitted power = 102.4W
    PowerSum = 127.9W

FSS:
    It has been locked for 0 days 3 hr and 32 min
    TPD[V] = 0.781V

ISS:
    The diffracted power is around 4.2%
    Last saturation event was 0 days 3 hours and 32 minutes ago


Possible Issues:
	PMC reflected power is high
	Check chiller (probably low water)



PSL probably wants some water and someone to chill with.
The plots below seem like it might be a little lonely. 

 

TITLE: 01/15 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 155Mpc
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 3mph Gusts, 1mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.35 μm/s
QUICK SUMMARY:

When I walked in H1 had been locked and Observing for 33 minutes!

Unknown lockloss this morning at 13:17 UTC
H1 Manny relocked without assistance this morning and no one was even woken up.

 

PS: It's a little icey out in the parking lot so be prepared stepping out of your car.
 

Camilla, TJ, Marc, Fil. WP#12281

We attached the AOM drive cable from the back of the D1300649 chassis to the lowest TEST point in the PEM feed through photo on the CO2X table. We used two barrel connectors (photos attached) to do this as it looks like there used to be an AOM driver on the table that the signal went into before going to the AOM.  

We thought that we could use the digital filters in the h1tcscs model to create a loop with this output and feed to the Synrad UC-2000 PWM controller (needs 0-10VDC). The max of CTRL2 was capped at +/-2 (unsure why) and this was actually +/- 0.6V on the BNC via an oscilloscope. We'll need to increase this by ~x10 to get PWM to work. Reverted changed sdfs. 

There was an unknown cable also labeled AOM drive coming into the table, not connected to anything photo. 

TITLE: 01/15 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:

IFO is in NLN and OBSERVING as of 05:40 UTC (20 min lock!)

Overall, bad recovery from maintenance. Here’s what happened in 7 short stories:

1. Late LN2 Fill: NORCO arrived to fill CP8 at 00:34 UTC (16:34 PT) over 3 hours late while we were locking. Jim made EX as robust as possible such that we would survive their truck moving through. This worked though the BRS was all over the place as expected. Due to them driving <5mph as requested, they took 20 minutes to arrive at EX, then over 1 hr to do the fill and then another 20 minutes to get back. They left site at 02:34 UTC (18:34 PT). After NORCO left, I undid Jim’s seismic configuration changes to go back to nominal.
2. Changes made prevented locking: Earlier in the day, (before NORCO arrived), we lost lock at state 557/8, TRANSITION_FROM_ETMX. I was told by the DAY operator that if this happened, I was to revert Camilla’s changes in alog 82277 with regard to the DARM Boost. Since Camilla and Sheila were in the room when this happened, they decided not to do it since it could have just been bad luck. Erik had also made changes regarding ramp times that affect this state. Camilla and Sheila left for the day and then NORCO arrived. While NORCO was filling, we lost lock again at state 557, TRANSITION_FROM_ETMX. Camilla told me to revert her changes and Jenne told Erik to revert his. This was done by 1:31 UTC (17:31 PT).
3. Changes reverted prevented locking: We could not lock ALS until Norco left the EX area, which happened ALS locked finally at 02:10 UTC (while Norco was en route from EX). Locking then happened automatically until state 500, PREP_DC_READOUT_TRANSITION, where for 10 minutes, I was getting the error message “OMC not ready”. I then called Camilla, who didn’t answer. I then called Erik, who was still on site and came to help troubleshoot the issue (since his changes involved resetting OMs - the idea was that maybe this is related, which it is likely not). After some fiddling to no avail, I called Sheila (the first person on the call list), who walked me through the OMC troubleshooting process.
4. Troubleshooting with Sheila and Erik: I informed Sheila that the error was that the OMC TF was not letting guardian continue. We did a manual OMC scan while waiting at ASC_QPD_ON to see that there indeed were triplets, with 2 sidebands and one carrier with the right magnitude. We let guardian lock it and it did, yielding the expected signal magnitude for the DCPD_SUM and PZT2 Monitors. Looking closely at the error, Erik realized that it was the phase that was below the threshold (150) and was measuring at 144. Sheila, Erik and I then decided to lower the phase threshold to 140. This worked immediately until we also lost lock just as immediately when we transitioned to DARM_TO_DC_READOUT, which saturated DCPD, EX, EY, IX and HAM6. This happened at 04:13 UTC (20:42 PT). At 4:49 UTC, I reverted the phase changes to OMC_LOCK guardian, since clearly something else was awry.
5. “Did you try turning it off and back on?”: After the lockloss and the reversion, we went up to lock again, preparing to face the same issue, DTT template and TF in hand and then… it worked. Perhaps the lockloss from earlier was from transitioning the OMC with whitening to the OMC without whitening (which I did after achieving the full OMC lock during troubleshooting). Now our next hurdle is state 557/558 (TRANSITION_FROM_ETMX).
6. The DARM Boost is actually still on: It turns out that the DARM boost had to be turned off in two places, rather than just line 3058 (alog 82277), there was another later boost turn-on at line 3774. Sheila discovered this during troubleshooting at 5:15 UTC. Since we didn’t want to tempt H1 by turning off abruptly, we chose to leave it on and see what would happen at 557/558. After some anticipatory sighs, it worked! We automatically went all the way to NLN.
7. 05:40 UTC - reached NLN! We had one node that wasn’t right, which was the FSS being in INIT and not knowing how to go to IDLE. It wasn’t on the SDF page (no other SDF diffs). Sheila manualled into IDLE, which worked. Strangely enough, we also got a “check PSL chiller” alert but since I know that this alert has some weird timing feature with respect to its thresholds, I will just tag PSL.

Big thanks to Erik and Sheila who helped a lot with troubleshooting.

LOG:

None

The changes here 82263 were reverted.

H1OMC models were reverted to version 5.3.0 of the RCG that uses the linear ramp.  The IFO was consistently losing lock after a filter ramp down.

WP12272 h1omc0 new RCG, quadratic filter ramping

Erik, Dave:

h1omc0 models were built against RCG 5.31 which introduces quadratic smoothing to ramped filter switching. All the models running on this frontend have the new rcg (h1iopomc0, h1omc, h1omcpi).

The overview was modified to show that h1omc0 has a different rgc than h1susex by colour coding the RCG: dark_blue=5.31(quadratic filter ramp and variable duotone frequence), light-blue-5.30 (LIGO DAC) and green = 5.14 (standard)

WP12274 h1guardian1 reboot

TJ, Erik:

TJ rebooted h1guardian1 to reload all the nodes. The hope is that this will eliminate the leap-second warnings we have been seeing on certain nodes.

In 81638 we removed a DARM1 FM1 boost because of a glitch when ramping it off causing locklosses during ESD transitions in preparation to switch back to ETMX. Today the CDS team updated H1OMC0 models with an improved filter ramping: 82263. We hope this will allow us to keep the boost us which gives us more range against high microseism while relocking (it's always off by NLN).

Uncommented line 3058 from ISC_LOCK.py and reloaded: ezca.get_LIGOFilter('LSC-DARM1').turn_on('FM1'). If we have locklossses at ISC_LOCK state 557 or 558, at the operator can re-comment this line out. Tagging OpsInfo.