Sheila found this morning that the ETMY ISI seemed to have a large peak at 1.5hz, which has in the past been indicative of a failing L4C. Most of the BSCs are already running blends that bypass the horizontal L4Cs because, except for ITMX, all BSCs have a flaky H1 L4C. That seems to be the case here with ETMY, when I switched the St1 RX/RY blends to CPS/T240 blends the the 1.5hz motion immediately settled down.

The best way I've found to see which sensor is failing when this happens is to take the L4Cs out of loop and look at the local to local L4C to T240 transfer functions (first attached plot). This gives a measurement of the L4Cs sensor response, and in this case it looks like the V3 L4C (light blue traces) has now become an almost 2hz instrument. The V1 and V2 sensors are normal, what we would expect for an L4C. If you look at these tfs with the L4Cs in loop, you can't measure this  change in response. I haven't had a chance to try because of commissioning, but I suspect you also can't see this with the isolation loops off, or during a driven transfer function. The sensor has to be in a low acceleration state.

Just looking at the blend complementarity with 1 good L4C and 1 L4C with it's pendulum frequency moved to 1.9hz explains what is happening. The sum of all 3 parts of the blend multiplied by their respective sensor responses should be 1, but having mismatched sensor responses in the L4Cs causes a deep notch in the supersensor between 1 and 2hz, which I can easily imagine does bad things to the quality of the feedback controls, second image.

It's possible we might "fix" this sensor by giving the ISI a kick, but the L4C would probably eventually return to this bad state. Or we could try adding a filter to compensate for the sensor changed response, but it seem like this sensors mode has probably been drifting around for a few weeks at least. Third attached image show trends of log blrms motion for the St1 RX/RY over the last 30 days, typically these seem to sit at around -1, when the ISI is healthy, but it goes up to 1-2 when the L4C is misbehaving.

We've had a large crew in the control room today working on locking:

• Oli ran initial alignment, manual BS adjustment was needed for ALS.
• I added back some of our DRMI ASC in the guardian now that we've recovered IFO alignment, I added back everything except PRC1, we'll see if this works and check the set point for the POP QPD.
• OMC whitening gain was 45dB, it should be 0dB, Stefan reset the dark offsets and accepted in SDF.
• I set the weights in the IDLE_ALS and SHUTTER_ALS edges so that we will again shutter ALS.
• There was a large 1.5Hz (ish) motion due to ETMY ISI, Jim says this was a bad L4C which he has now taken out of the loop, separate alog.  summary page
• While Jim was working on the ISI, Jennie and I checked the OMC ASC step responses (76208)
• Elenna checked that the POP offset she set yesterday (76190) in full lock look good for DRMI, so she added it PRC1 back to the DRMI ASC and now our DRMI ASC should be back to normal.
• We lost lock at RF DARM transition a couple of times, we changed the RF DARM normalization back to the X arm (It was changed to Y arm earlier this week) and increased the wait time between transitioning the input matrix and changing the filters to 2 seconds (we previously had a 1 second wait for a 2 second ramp). 
• We had two locklosses trying to transition to DC readout, this was because I reverted the change to the RF DARM normalization, but Jenne reminded us that we also had to make a change in DC_READOUT_TRANSITION for this change.
• We've put SDF revert back in the normal path of ISC_LOCK guardian.
• We had to rerun initial alignment, then checked the DARM UGF while powering up.  We have done most of the steps to nominal low noise except for injecting squeezing and OMC whitening. 
• Jennie Wright is moving the OMC ASC offsets to increase kappa c.

This is a summary of information that is spread across different alogs in the last few days.  When recovering from a vent, in which the green camera reference may have been lost when the gate valves are actuated, these are some useful steps and guardian changes.

• Avoid having ISC_LOCK send the ALS ARM guardians to increase flashes.  This can be done by adding a weight of 5 to the edge between READY and LOCKING_ARMS_GREEN (line 6442 right now).  Ryan Short took care of this this week, he may have done this in a different way. 
• Avoid shuttering ALS in the CARM offset reduction sequence, so that we can check and reset green references when we do lock.  This can be done by changing the weights of     ('IDLE_ALS', 'CARM_OFFSET_REDUCTION', 5),   ('PARK_ALS_VCO',  'SHUTTER_ALS', 1). 
• If the alignment of the green is so far off it doesn't stay locked we need to run the green QPD offset scripts (see 50968), we can start this as soon as the ALS_XARM and ALS_YARM guardians are in RED_LOCKED, and before DHARD is engaged.
• Remove DRMI ASC by setting the flags in the ISC_DRMI guardian ENGAGE_DRMI_ASC (usePRC1, ect) to false, except for MICH.
• Depending on what has been happening and how SDF has been kept up to date, one could make CHECK_SDF the default to skip SDF_REVERT.

We have made and reverted these changes this week, I'm not sure if we've reverted the change to the INCREASE_FLASHES timer.

Louis, Vicki, Craig

Today we revived the correlated noise budget code located at https://git.ligo.org/aligo_commissioning/correlated_noise, first made in June (alog 71333).
This code is a standalone singular script which grabs the raw DCPD data, grabs the DARM calibration OLG and sensing function info, does some math from Kiwamu's DCC T1700131, and produces a correlated noisebudget.

Formerly, I had used my own noise_recorder white noise injections to get the DARM calibration information.  But since I am gone, it would be better if we used a maintained piece of software to get the calibration.
Today, Louis helped me to use a pydarm .npz file to get the DARM OLG and sensing functions.  
The code we developed today is in /ligo/gitcommon/correlated_noise/code/plot_cross_correlation_pydarm_npz_calibration.py

Attached are the correlated noisebudget results for GPS start 1387130434, over 600 seconds of data.  
I posit that we don't really need a ton of time to integrate down, both because the shot noise floor is not that far away from the correlated noise floor, and we can use logbinning averaging to achieve a higher effect number of averages in a shorter amount of time.

Results
I've used the Dec 18 noisebudget traces made by Camilla in /ligo/gitcommon/NoiseBudget/aligoNB/out/H1/lho_darm_noisebudget/lho_darm_noisebudget.hdf5.
Overall, the correlated noise results seem decently sensible, especially at high frequency.
I note that the correlated noise limit we are hitting at 1 kHz seems to be laser frequency noise (third plot attached).

At 500 Hz, we are running into our "thermal noise floor".  
Whether or not this is really thermal noise is anyone's guess, since it is above our official estimate, but the story for O4a remains the same as from O3.

At 20 Hz, the correlated noise seems to overestimate what is actually in DARM, which cannot be correct.  
This is something Sheila has been concerned with in the past (alog 70978).  
This could be chalked up to a small change in the actual vs pydarm DARM OLG, unclear what else it could possibly be.

It is worth noting that the DARM OLG cannot really affect the correlated noise results massively above 500 Hz, just because the magnitude of the DARM OLG is around 0.1 there and falling quickly.  

Future Work
The next steps is for Vicki to import this code to the aligoNB, removing all calls to my nds2utils library (avoiding a dependency there).
Louis also wants to add some functionality to aligoNB so that the noisebudget can call pydarm at a GPS time and get the most recent calibration.  (Craig loves this).
Louis will clone the aligoNB env to verify everything still works before making a "pull request" to the actual aligoNB repo.

Updated the H1:ASC-OMC_[A/B]_SEG[1..4]_OFFSET (OMC QPD dark offsets) for the channels to read 0 inthe dark state.
Cleared the H1:ASC-OMC_A_[PIT/YAW]_OFFSET, since we have a new OMC, so the old offsets are no longer meaningful.
Updated the DCPD offsets, even thout they were tiny.

SDF attached.

Trent Matt and Jennie

We looked at the pitch and yaw of the IFO output mirrors (OM1-3+C) this morning around 6am and the same pitch and yaw channels at 6am yesterday; the only one that we saw with significant differences were OM2 Pitch and Yaw, which both went down about 30 and 20 microradians (we think this is the unit counts represents), respectively.

Fri Mar 08 10:11:29 2024 INFO: Fill completed in 11min 26secs

Gerardo confirmed a good fill curbside. TCs started around -30C, so I've transitioned to -100C trip temps. Plot's y-cursors have been changed accordingly.

Stefan B, Jennie W, Sheila

Summary: Checked OMC step responses and they converge in 30s or less without saturating suspensions. These loops can be slowed down if need be.

Since locking yesterday there has been no light through the OMC.

Sheila decreased the OMC QPD whitening gain as the QPDs were saturating. This was turned up during the vent so she changed it back to what it was in O4a.

We still had some sort of signal into the OMC QPDs. We turned off the pitch and yaw offsets to these QPDs and then went into the whitening filters and turned off the bottom row of gain filters so only the low pass filters are on and set the offsets to be the negative of OUTMON. First image is original filters for QPD B and second is the filter once setting the offsets (QPD A was done in the same way).

We did this dark offset procedure for QPD A and B on OMC and then Stefan manually set the offsets on DCPD A and B to 0.03 on input to filters.

Sheila did the step responses for the OMC ASC loops by switching on offsets in the POS X, Y and ANG X, Y filters in turn.

POX X offset causes POS X and ANG X to move, POS X converges in 30s and ANG X converges in less than 5s. This means we could decrease the gain in ANG X loop. See image 3.

POS Y offset coupled to both POS Y and ANG Y (half as much) and they both converge in 30s.

ANG Y offset does not couple to other DOFs and converges in 20s.

ANG X offset also does not couple in to other DOFs and converges in 20s.

First image shows POS X and ANG X offsets being inplemented and second shows POS Y and ANG Y.

TITLE: 03/08 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 2mph Gusts, 1mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.32 μm/s
QUICK SUMMARY:

More locking today!

The pressures:
HAM7: ~2.7E-7 Torr
HAM8: ~5.34E-7 Torr
Corner: ~4.4E-8 Torr
EX: ~5.0E-9 Torr

Today's activities:
- At the MY and EY stations the turbo-side Hepta header filters have been added, closing WP11570
- At these stations, Turbo functionality test was also done, see here more in-detail: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=76196

- HAM8 Turbo was valved out, and HAM8 was valved together with IP24 (FCV-9 open). However, there is still a big jump in pressure, the Ion pumps turned around only as of this writing
- HAM7 Annulus Ion Pump has been taken care of. It was believed that the G3/8 plug is the reason for the railing Ion-pump, but in the end a simple controller swap solved the issue
- Relay tube - HAM7 - HAM8 further schedule: This volume will be valved in to the main volume tomorrow, as soon as the commissioning team needs it - RV1; FCV1; FCV2; FCV3; FCV4; FCV8 will be opened

While H1 was running an initial alignment this evening, I took the opportunity to do a quick tune-up of the FSS RefCav alignment from the control room using the two picomotor-controlled mirrors in the FSS path. I was only able to get a marginal improvement, but I did get the signal on the TPD to increase from 0.7V to 0.8V as a result of the alignment tweak.

[Jenne, Stefan, Georgia, Matt, Elenna, Ryan S]

We have made it almost through the "Max Power" state of the guardian! During the last power up step from 55 to 60W we had a fast lockloss. It appears to be about 16 Hz and in the DARM loop itself. Since we are down and reset the green references, we are running an initial alignment. Once that completes, we will try locking again, possibly with a slow power up by hand and measurements of the DARM loop gain. The violins are a bit high but Ryan engaged the 2W damping settings and they were coming down well.

Trent, Vicky, Nutsinee

As part of the 8dB HD squeezing recovery effort we went and tweaked the HD visibility on SQZT7. We now have 99.15% fringe visibility on PDA and 98.17% on PDB. Green power coming out of the SHG dropped so SHG ISS was railing. I've increased the amount of green availible to the pump fiber coupler. More squeezing plots to follow. 

We ran the functionality test on the main turbopumps in MY and EY. The scroll pump is started to take pressure down to low 10^-02 Torr, at which time the turbo pump is started, the system reaches low 10^-08 Torr after a few minutes, then the turbo pump system is left ON for about 1 hour, after the hour the system goes through a shut down sequence.
 

MY Turbo:

Bearing Life:100%

Turbo Hours: 211

Scroll Pump Hours: 76

EY Turbo:

Bearing Life:100%

Turbo Hours: 1279

Scroll Pump Hours: 75

No issues encountered for either turbopump

Closing WP 11755

Jonathan, Patrick, Dave:

Around 14:00 yesterday (Wed 06mar2024) we upgraded the camera network link to h1digivideo3 from 1GE TPE to 10GE fiber. After running overnight, all of the cameras on this server are still running and, with the exception of ETMY, no further VALID=0 have been seen (2 day trend attached, upgrade at the 18h mark).

ETMY continues to have an hourly VALID=0 which flashes the camera client images blue-screen for a few seconds. This happens at roughly the 20 minute mark in the hour and it slowly advances through the hour.

The ETMY periodicity is not changed by h1digivideo3 reboots, suggesting it is in the camera itself. To test this, I power cycled the ETMY camera at 08:34 this morning.

I made a LOCK_PMC state in SQZ_MANAGER guardian. The LOCK_PMC state is between LOCK_TTFSS and LOCK_SHG states. The LOCK_PMC state is copied from LOCK_SHG state, but I commented out the PZT checker because the PMC_PZT_OK function is not defined now.