After remotely rebooting h1seiey and seeing that the 4th ADC is now completely absent from the PCI bus, the next test was a power cycle of the IO Chassis.

Procedure was:

Stop h1seiey models, fence from Dolphin and power down the computer.

Dave (@EY):

power down the IO Chassis (front switch, then rear switch)

Power down the 16bit-DAC AI Chassis (prevent overvoltage when IO Chassis is power up)

Power up the IO Chassis (rear switch, then front switch).

The chassis did not power up. Tracking it back, the +24V-DC power strip was unpowered, the laser interlock chassis which is also plugged into this was powered down. This tripped all the lasers.

Fil came out to EY with a spare ADC

Dave & Fil (@EY):

We opened the IO Chassis and removed the 4th ADC. With this slot empty Fil powered up the DC power supply with the IO Chassis on, it did not trip.

We powered down the IO Chassis to install a new ADC. We are skipping the slot the old ADC was in, because it could be a bad slot.

The second DAC was moved from A2-slot4 to A3-slot1, the new ADC was installed in A2-slot4, leaving the suspect A2-slot3 empty.

We powered the IO Chassis on with no problems, then we powered up the h1seiey computer. The models started with no issues, I was able to reset the SWWD.

The chassis was buttoned up, pushed into the rack, and the AI Chassis were powered back up.

Marc is fixing a cracked cap on the old ADC so we can test it offline.

ADCs:

The PSL is now back online after the interlock trip.  During recovery I did a quick beam alignment into the PMC, as PMC Refl came back at ~26.5W, 1.2W higher than before the interlock trip.  After the alignment I was able to get PMC Refl down to ~25.4W.  I then changed the pump diode current for power supply 3 (Amp2, pump diodes 1 and 2), to 9.4A from 9.3A.  This made no change to Amp2 Out but brought PMC Trans back to ~106.2W and PMC Refl back to ~25.2W.  The FSS and ISS engaged without issue.  I had to change the ISS RefSignal to -2.02V from -1.99V to get the % diffracted power back to ~4.0; with the ISS engaged PMC Trans is now ~106.0W.  The % diffracted power will likely slowly change as things reach thermal equilibrium, as it usually does, so I'll monitor this over the weekend.

The watchdogs are back on and the PSL is good to go.

Thu Oct 30 10:07:57 2025 INFO: Fill completed in 7min 53secs

Gerardo confirmed a good fill curbside.

front end server h1seiey went down at 13:05 with an ADC timeout on the last ADC.  Restart of the server did not bring the ADC back.  Dave is headed out with a replacement.

This could be the cause of the last lockloss.

3rd lock loss of the day. This one was not like the others though, it was very fast.

A little late to observing since we thought that the estimator was injecting noise, but it's possible that it was some of Robert's equipment. Still not entirely sure but the scattering that we saw in DARM looks to be gone for now.

J. Kissel

Comparison of local performance metrics for H1 SUS PR3 after turning on the LP estimators this morning.
2025-10-30 13:00 UTC -- only Y estimator ON
2025-10-30 17:00 UTC -- all LPY estimators ON.

Both times are when IFO is in nominal low noise with no commissioning happening.

In short -- L is far more limited by residual suspension point motion that P or Y, so the reduction in off resonance drive is not *as* awesome as in P or Y. But there is reduction in off-resonance drive, and damping on resonance is about the same.
Nice!

More plots to come...

There was a small amount of ground motion at the time, but this lock loss seemed very similar to the previous one.

Some shots that were shared on MM by me and Elenna attached below.

With the great success of the ASC high gain seismic guardian state that TJ has been testing, 87788, we have decided to use the ASC high gain script for a different purpose, to damp the 1 Hz oscillation at the beginning of the lock.

From various testing, it appears that the pitch ASC is the likely culprit of this problem, but it's still not clear which loop in particular. So, now this script is designed to transition the pitch ASC only to the high bandwidth configuration. This is done in all one step, where CHARD P, DHARD P and DSOFT P are all transitioned to HBW. However, there is a 20 second time.sleep at the end of the script because that is the longest ramp of the filters that are changed. After this sleep, the script will print "Done!". The reverse happens for the "go back" script.

I would prefer if we could use this script for now to avoid the 1 Hz oscillation, as a way to test what effects the oscillation. If the script does not work, then I recommend using the seismic guardian high gain state, which will also transition the yaw ASC and turn of the LSC FF.

Jim created templates for exciting the HAM7 ISI in jim.warner/ham7/isi_injections

I copied these over to /ligo/home/sheila.dwyer/SEI/HAM7, and ran the x and y ones.  While I was getting ready to run the Z one (but not making any excitation), we lost lock for an unknown reason. 

J. Kissel, O. Patane, E. Bonilla

The week of Oct 20-24 2025 was pretty rough on the duty factor for the IFO. Among other things (see excellent summary in LHO:87755), the L and P estimators for H1SUSPR3 and SR3 were installed/updated on Tuesday Oct 21 2025 but had a bug in the filter installation script (LHO:87689) that ended up poorly damping some of those SUS' modes, so the L and P filters were turned OFF on Wednesday Oct 22 (LHO:87659).

These bugs have been fixed on Thursday Oct 23 2025 (LHO:87705), but they've remained off.

Today, by 2025-10-30 16:45:16 UTC (09:45:16 PDT), I've turned the L and P estimators again.

Since Oli had configured all of the updated estimator filters already, accepted them in both SUS' safe and OBSERVE SDF snaps, and left them "off" in the "controlled" way by setting the estimator on/off switch to OFF (H1:SUS-[P,S]R3_M1_EST_[L,P]_SWITCH_NEXT_CHAN = 2), it was trivial to turn them ON by setting H1:SUS-[P,S]R3_M1_EST_[L,P]_SWITCH_NEXT_CHAN = 3, i.e. clicking the "Use ESTIM" button on each estimator's MEDM overview screen.

I've accepted having these estimators ON in both SUS's SAFE and OBSERVE snaps.

Ran a broad band and then started simulines for 4 minutes before we had a lock loss.

Start:

PDT: 2025-10-30 08:36:06.772736 PDT
UTC: 2025-10-30 15:36:06.772736 UTC
GPS: 1445873784.772736
 

Lock loss at 154010UTC

In the middle of the simulines portion of the calibration sweep we lost lock. Robert was out on the floor at the time, but we're unsure of that was the cause at this time.

TITLE: 10/30 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 4mph Gusts, 0mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.33 μm/s 
QUICK SUMMARY: Locked for 19 hours. The useism has gone down, now a calm environment. There is an FMCS alarm about the EY VEA temp rising, and LVEA 1A.C in fault. Temps look to be correcting themselves or the FAC crew is already on it. ETMX violin mode 6 is not being damping and is moderately high, looks like they weren't able to get a good setting last night.

Calibration and commisioning coming up in 30 min.  at 1530-2000UTC.

Summary

Round 2 of testing the guardianization of turning on and off the high ASC gains (Round 1 - alog87462). SEI_ENV will now automatically move us into the high gain ASC state when a.) we are in the earthquake state b.) there is an incoming or ongoing earthquake that is at or below the dotted line on the "rasta plot". The transition takes 11 seconds to complete, and it will transition back when the ground motion is low enough to bring us out earthquake state.

Details from today

I started testing with a few 10 and 5 second waits between steps, just as is done in the script that we currently use. Once those ran successfully a few times I started to decrease the wait times between steps. Eventually, I had success transitioning all the ASC at the same time, then the FF 10 seconds after. since this was the same configuration that I had last time I tried this, tried to reporduce the lock loss by requesting the High ASC state, then immediately requesting the Low ASC state. This did, again, cause a lock loss. To avoid this I have a wait timer in the High state so it won't switch quickly from one to the other.

Transitioning back out of the high ASC state has the same thresholds as the earthquake state currently. We didn't want to transition back and then have to do it all over again, or wait in earthquake for another 10 minutes for it to calm down. We might make this a bit shorter or smarter after we've seen it work a few times.