front end server h1seiey went down at 13:05 with an ADC timeout on the last ADC. Restart of the server did not bring the ADC back. Dave is headed out with a replacement.
This could be the cause of the last lockloss.
3rd lock loss of the day. This one was not like the others though, it was very fast.
A little late to observing since we thought that the estimator was injecting noise, but it's possible that it was some of Robert's equipment. Still not entirely sure but the scattering that we saw in DARM looks to be gone for now.
J. Kissel Comparison of local performance metrics for H1 SUS PR3 after turning on the LP estimators this morning. 2025-10-30 13:00 UTC -- only Y estimator ON 2025-10-30 17:00 UTC -- all LPY estimators ON. Both times are when IFO is in nominal low noise with no commissioning happening. In short -- L is far more limited by residual suspension point motion that P or Y, so the reduction in off resonance drive is not *as* awesome as in P or Y. But there is reduction in off-resonance drive, and damping on resonance is about the same. Nice! More plots to come...
There was a small amount of ground motion at the time, but this lock loss seemed very similar to the previous one.
Some shots that were shared on MM by me and Elenna attached below.
Both of these locklosses saw a very similar 0.6 Hz wobble in PR3 and SR3. In the second one the estimator was on, in the first it was off.
We looked at the OSEMINF signals for both PR3 and SR3, and there isn't one osem that is obviously glitching.
With the great success of the ASC high gain seismic guardian state that TJ has been testing, 87788, we have decided to use the ASC high gain script for a different purpose, to damp the 1 Hz oscillation at the beginning of the lock.
From various testing, it appears that the pitch ASC is the likely culprit of this problem, but it's still not clear which loop in particular. So, now this script is designed to transition the pitch ASC only to the high bandwidth configuration. This is done in all one step, where CHARD P, DHARD P and DSOFT P are all transitioned to HBW. However, there is a 20 second time.sleep at the end of the script because that is the longest ramp of the filters that are changed. After this sleep, the script will print "Done!". The reverse happens for the "go back" script.
I would prefer if we could use this script for now to avoid the 1 Hz oscillation, as a way to test what effects the oscillation. If the script does not work, then I recommend using the seismic guardian high gain state, which will also transition the yaw ASC and turn of the LSC FF.
Jim created templates for exciting the HAM7 ISI in jim.warner/ham7/isi_injections
I copied these over to /ligo/home/sheila.dwyer/SEI/HAM7, and ran the x and y ones. While I was getting ready to run the Z one (but not making any excitation), we lost lock for an unknown reason.
J. Kissel, O. Patane, E. Bonilla The week of Oct 20-24 2025 was pretty rough on the duty factor for the IFO. Among other things (see excellent summary in LHO:87755), the L and P estimators for H1SUSPR3 and SR3 were installed/updated on Tuesday Oct 21 2025 but had a bug in the filter installation script (LHO:87689) that ended up poorly damping some of those SUS' modes, so the L and P filters were turned OFF on Wednesday Oct 22 (LHO:87659). These bugs have been fixed on Thursday Oct 23 2025 (LHO:87705), but they've remained off. Today, by 2025-10-30 16:45:16 UTC (09:45:16 PDT), I've turned the L and P estimators again. Since Oli had configured all of the updated estimator filters already, accepted them in both SUS' safe and OBSERVE SDF snaps, and left them "off" in the "controlled" way by setting the estimator on/off switch to OFF (H1:SUS-[P,S]R3_M1_EST_[L,P]_SWITCH_NEXT_CHAN = 2), it was trivial to turn them ON by setting H1:SUS-[P,S]R3_M1_EST_[L,P]_SWITCH_NEXT_CHAN = 3, i.e. clicking the "Use ESTIM" button on each estimator's MEDM overview screen. I've accepted having these estimators ON in both SUS's SAFE and OBSERVE snaps.
Ran a broad band and then started simulines for 4 minutes before we had a lock loss.
Start:
PDT: 2025-10-30 08:36:06.772736 PDT
UTC: 2025-10-30 15:36:06.772736 UTC
GPS: 1445873784.772736
Lock loss at 154010UTC
The broadband shows the calibration looks pretty good, although it is different, probably from Sheila's change of the gain scaling which has effected the measured optical gain, 87806.
It would be helpful if we got a full calibration this weekend, just to see if this is only due to the kappa C change.
In the middle of the simulines portion of the calibration sweep we lost lock. Robert was out on the floor at the time, but we're unsure of that was the cause at this time.
Back up to low noise at 1722UTC, ran initial alignment and the 1Hz ringup DID NOT show up.
TITLE: 10/30 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 4mph Gusts, 0mph 3min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.33 μm/s
QUICK SUMMARY: Locked for 19 hours. The useism has gone down, now a calm environment. There is an FMCS alarm about the EY VEA temp rising, and LVEA 1A.C in fault. Temps look to be correcting themselves or the FAC crew is already on it. ETMX violin mode 6 is not being damping and is moderately high, looks like they weren't able to get a good setting last night.
Calibration and commisioning coming up in 30 min. at 1530-2000UTC.
EY VEA temperature started deviation around 05:30 this morning.
TITLE: 10/30 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 153Mpc
INCOMING OPERATOR: Tony
SHIFT SUMMARY: Observing at 150 Mpc and have been locked for almost 9.5 hours. I ended up not turning the damping gains back on for ETMX mode 6 or ETMY mode 1/6 in case I rang them up and lost damping progress, but they seem to be making their way downtown, walking slowly. Quiet shift with nothing else to report.
LOG:
23:04-05UTC Helicopter flying overhead
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 21:29 | SPI | Corey, Ryan | Opt lab | n | First contacting optics | 23:52 |
Observing at 150 Mpc and have been Locked for 7.5 hours. Nothing to report
(C.Gray, R.Short)
Spent about 1-day of work getting set-up, organized, game-plan in motion and finally starting the first round of cleaning for the in-vac SPI optics (Suspension Point Interferometer). Ryan and I did this cleaning on a flow bench in the Optics Table, and we have decided to clean all the optics at once step by step. The first step we did was cleaning all the barrels of these 1" (mostly) & 2" optics. This step took roughly about 2-hrs.
Next steps would be drag wipe & First Contact-ing the front & back faces of the optics.
NOTES:
1) There are some 2" PLCX (plano convex) optics which are a bit odd to clean/handle. (1) The curved convex surface might be tough to First Contact. (2) These optics don't fit in our PETG "clean" optic containers (just mentioned to Betsy and she said we might install them into their optical mounts or lay them in a pan wrapped in lens tissue). (See images 9302 & 9300 below)
2) Some of the 1" optics are in PETG containers which are not great for our "thick" beamsplitters. So one needs to be careful unwrapping these beamsplitters later on---so the lid & optic don't fall out--OPEN CAREFULLY please!
(Pretend this was posted by TJ)
TITLE: 10/29 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 150Mpc
INCOMING OPERATOR: Oli
SHIFT SUMMARY: Two relocks, both lock losses caused by earthquakes. Both relocked needed an initial alignment and needed the high gain ASC to avoid the 1 Hz ringup. While locked recently, EY violin mode1/6 started to slowly ring up. I tried many settings and the best I found was Mode 1 +60 -0.2 gain, but I wouldn't say it worked, just made it not grow as fast. While we were looking at this ETMX mode 6 also started to grow. When Oli tried to damp it we noticed large changes in the DCPD window traces on the FOM. See attached. The line at 500Hz in DARM also went down by about an order of magnitude, all almost instantly. The violin mode monitors didn't show a change though. No clue what happened but the violins aren't terrible.
We've been observing for 4 hours.
LOG:
1558 Observing
1722 Lock loss
1940 Observing
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 14:44 | FAC | Kim | Opt lab | n | Tech clean | 15:36 |
| 15:20 | FAC | Randy | Xarm | n | BTE sealing between MX and EX | 22:03 |
| 17:42 | SPI | Corey, Betsy, Ryan S | Opt Lab | n | Optics cleaning (Betsy out 1905) | 19:38 |
| 19:17 | IO | Rahul | Opt Lab | LOCAL | JM optic measurements | 20:02 |
| 19:59 | IO | Keita | Opt Lab | n | Cleanup of ISS Array | 21:55 |
| 21:29 | SPI | Corey, Ryan | Opt lab | n | First contacting optics | ongoing |
| 21:47 | PCAL | Tony | PCAL Lab | LOCAL | Swapping spheres | 22:12 |
Round 2 of testing the guardianization of turning on and off the high ASC gains (Round 1 - alog87462). SEI_ENV will now automatically move us into the high gain ASC state when a.) we are in the earthquake state b.) there is an incoming or ongoing earthquake that is at or below the dotted line on the "rasta plot". The transition takes 11 seconds to complete, and it will transition back when the ground motion is low enough to bring us out earthquake state.
I started testing with a few 10 and 5 second waits between steps, just as is done in the script that we currently use. Once those ran successfully a few times I started to decrease the wait times between steps. Eventually, I had success transitioning all the ASC at the same time, then the FF 10 seconds after. since this was the same configuration that I had last time I tried this, tried to reporduce the lock loss by requesting the High ASC state, then immediately requesting the Low ASC state. This did, again, cause a lock loss. To avoid this I have a wait timer in the High state so it won't switch quickly from one to the other.
Transitioning back out of the high ASC state has the same thresholds as the earthquake state currently. We didn't want to transition back and then have to do it all over again, or wait in earthquake for another 10 minutes for it to calm down. We might make this a bit shorter or smarter after we've seen it work a few times.
| Time (hhmmss UTC) | Transition to | Notes |
| 150251 | High | 10/5s timers |
| 150457 | Low | 10/5s timers |
| 150616 | High | Repeat of above |
| 150724 | Low | Repeat of above |
| 150754 | High | 1/5s timers |
| 150930 | Low | 1/5s timers |
| 151113 | High | All ASC engaged at once |
| 151218 | Low | All ASC engaged at once |
| 151326 | High | All ASC engaged at once |
| 151340 | Low | Lock loss |
I forgot that this would eventually trigger IFO_NOTIFY if the high gain state were to keep us out of Observing for longer than 10 minutes while IFO_NOTIFY was running. I've changed IFO_NOTIFY to not notify when the SEI_ENV node is in the high asc or transition states.
Lock loss caused by h1iopseiey adc card failure - alog87862