All models on h1seih45 glitched at 07:09:46 PDT (14:09:46 UTC). This appears to have broken the lock and caused guardian to start an initial alignment.

Symptoms are:

• All IPC receivers, Dolphin and SHMEM, on all h1seih45 models reported a single receive error
• DAQ (both DC0 and DC1) reported a single CRC error for all h1seih45 models
• dmesg, DAQ and central loggers show no logs at this time
• No evidence of a timing glitch at this time

The models appeared to have resumed normal operation following the glitch, no restarts were needed. At the time of writing H1 is locked with a range of 160MPc.

TITLE: 10/31 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 5mph Gusts, 3mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.47 μm/s
QUICK SUMMARY:

Looks like we just finished an initial alignment and are starting to relock

H1 called for assistance at 13:13 UTC when SRM watchdogs tripped during the SRC alignment steps of initial alignment. After resetting the WDs, I just let Guardian try SRC align again, but SRM tripped during ACQUIRE_SRY. I reset the WDs again and took ALIGN_IFO to DOWN to manually align SRM a bit (didn't get too much improvement on the AS_A fringing, but some), and had Guardian try again. This time, there were no issues, and initial alignment finished without issue.

H1 is currently relocking and just got through DRMI.

TITLE: 10/31 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:  Currently relocking, but we've had a few lock losses after DRMI and one at engage_asc_for_full_ifo. We've just made it past that point again.

• Gave up trying to bring PR2 back to its old position, it was close but as I get closer it seemed to only make things worse. I then ran an initial alignment sans green because it I had moved the PRs a fair amount and DRMI was definitely not going to catch. During the IA, it seemed the PR2 preferred to be at it's place post earthquake.
• DRMI post 2nd initial alignment was a great sucess!
• Lost lock at LOWNOISE_ESD_ETMX - saw a pitch ringup in what looks like CHARD.
• Next time around started to see a ringup again, but it calmed itself down after a minute. Made it to low noise at 0201UTC

Lockloss 0349

• It has been able to reaquire DRMI, but there have been a many lock losses, seeming to be from the FSS glitching, or something else that I don't know of.

Averaging Mass Centering channels for 10 [sec] ...
2024-10-30 14:34:04.266740


There are 14 T240 proof masses out of range ( > 0.3 [V] )!
ETMX T240 2 DOF X/U = -0.755 [V]
ETMX T240 2 DOF Y/V = -0.707 [V]
ETMX T240 2 DOF Z/W = -0.473 [V]
ITMX T240 1 DOF X/U = -1.549 [V]
ITMX T240 1 DOF Y/V = 0.328 [V]
ITMX T240 1 DOF Z/W = 0.452 [V]
ITMX T240 3 DOF X/U = -1.611 [V]
ITMY T240 3 DOF X/U = -0.797 [V]
ITMY T240 3 DOF Z/W = -1.997 [V]
BS T240 1 DOF Y/V = -0.404 [V]
BS T240 3 DOF Y/V = -0.317 [V]
BS T240 3 DOF Z/W = -0.502 [V]
HAM8 1 DOF Y/V = -0.436 [V]
HAM8 1 DOF Z/W = -0.758 [V]


All other proof masses are within range ( < 0.3 [V] ):
ETMX T240 1 DOF X/U = -0.04 [V]
ETMX T240 1 DOF Y/V = -0.066 [V]
ETMX T240 1 DOF Z/W = -0.098 [V]
ETMX T240 3 DOF X/U = -0.055 [V]
ETMX T240 3 DOF Y/V = -0.111 [V]
ETMX T240 3 DOF Z/W = -0.058 [V]
ETMY T240 1 DOF X/U = 0.015 [V]
ETMY T240 1 DOF Y/V = 0.136 [V]
ETMY T240 1 DOF Z/W = 0.192 [V]
ETMY T240 2 DOF X/U = -0.069 [V]
ETMY T240 2 DOF Y/V = 0.187 [V]
ETMY T240 2 DOF Z/W = 0.029 [V]
ETMY T240 3 DOF X/U = 0.206 [V]
ETMY T240 3 DOF Y/V = 0.04 [V]
ETMY T240 3 DOF Z/W = 0.128 [V]
ITMX T240 2 DOF X/U = 0.129 [V]
ITMX T240 2 DOF Y/V = 0.265 [V]
ITMX T240 2 DOF Z/W = 0.229 [V]
ITMX T240 3 DOF Y/V = 0.124 [V]
ITMX T240 3 DOF Z/W = 0.102 [V]
ITMY T240 1 DOF X/U = 0.083 [V]
ITMY T240 1 DOF Y/V = 0.084 [V]
ITMY T240 1 DOF Z/W = -0.063 [V]
ITMY T240 2 DOF X/U = 0.05 [V]
ITMY T240 2 DOF Y/V = 0.182 [V]
ITMY T240 2 DOF Z/W = 0.1 [V]
ITMY T240 3 DOF Y/V = 0.072 [V]
BS T240 1 DOF X/U = -0.174 [V]
BS T240 1 DOF Z/W = 0.082 [V]
BS T240 2 DOF X/U = -0.099 [V]
BS T240 2 DOF Y/V = 0.002 [V]
BS T240 2 DOF Z/W = -0.126 [V]
BS T240 3 DOF X/U = -0.213 [V]
HAM8 1 DOF X/U = -0.286 [V]
 

1414381791

IMC tag on this one, ending a 1 hr 48min lock.

Following instructions from alog76751 and from Sheila herself, I ran the CARM olg python script located in /ligo/gitcommon/psl_measurements/. Sheila had already hooked up the SR785 at the PSL racks earlier in the day while we were relocking, so it was easy enough for me to check that we could connect to it through GPIB (checked with ipython like Craig did in alog64204), then run the commands listed in alog76751. After I ran it, I then went out into the LVEA and unplugged and turned off the SR785, as well as the LVEA lights.

Since this plot is saved as a pdf, its hard to say exactly what the UGF is, but it looks around 17kHz just as it has been, consistent with 76751 & 76448  but a little higher than 70920 and 65676 and 67584.

• Gave up trying to bring PR2 back to its old position, it was close but as I get closer it seemed to only make things worse. I then ran an initial alignment sans green because it I had moved the PRs a fair amount and DRMI was definitely not going to catch. During the IA, it seemed the PR2 preferred to be at it's place post earthquake.
• DRMI post 2nd initial alignment was a great sucess!
• Lost lock at LOWNOISE_ESD_ETMX - saw a pitch ringup in what looks like CHARD.
• Next time around started to see a ringup again, but it calmed itself down after a minute. Made it to low noise at 0201UTC

TITLE: 10/31 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 14mph Gusts, 9mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.49 μm/s
QUICK SUMMARY: After the earthquake we started an initial alignment, in which Oli only had to adjust IM4. Now while locking PRMI, the POP90 signal isn't as high has before the earthquake when locked. Trending the PRs, PR2 is a little bit off in Y from before, so I've tried to keep PRC1 on and move PR2 slowly toward its old position while PRM followed. I'm not too sure this is helping at all though. Once this is done I'll try more DRMI.

TITLE: 10/30 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Earthquake
INCOMING OPERATOR: TJ
SHIFT SUMMARY: Trying to relock after the huge earthquake off the coast of Oregon knocked us down. Currently at PRMI_ASC and TJ is trying to touch up PRMI.
LOG:

14:30 In tail end of an initial alignment
14:33 Initial alignment done, relocking
15:26 NOMINAL_LOW_NOISE
15:26 Observing

15:49 Out of Observing to adjust CARM gain
    - Trying to recreate FSS glitches
15:50 Back into Observing
16:09 Lockloss

16:10 Initial alignment
    - SRC wouldn't lock and SRM tripped twice
16:37 Took us to DOWN and just started relocking
    - Lockloss from LOCKING_ARMS_GREEN
    - Lockloss from ACQUIRE_DRMI_1F
    - IMC took 15 minutes to relock. Relocked after I took it to OFFLINE for a few minutes
17:55 NOMINAL_LOW_NOISE
17:58 Observing

18:00 Out of Observing for PSL makeup air check
18:32 Back to Observing

20:17 Lockloss due to earthquake
    Tripped:
    - ISI HAM{2,3,4,5,7,8}, BS, {E,I}TM{X,Y}
    - SUS PR2, PR3, SR2, SRM, IM4, OFI, TMSY
21:45 Starting initial alignment
    - Could not get XARM IR due to the IMs having drifted during the earthquake. I trended them back with driftmon and adjusted sliders until they were back to where they were before the earthquake.
22:36 Initial alignment finished, relocking
    - Lockloss from PRMI_ASC x2
    - Lockloss from LOCKING_ARMS_GREEN
    - Lockloss from FIND_IR

Reran all O4b locklosses on the locklost tool with the most up to date code, which includes the "IMC" tag.

I used the lockloss tool data since the emergency OFI vent (back online ~23rd August) until today and did some excel wizardry (attached) to make the two attached plots, showing the number of locklosses per day tagged with IMC and without tag IMC. I made this plots for just locklosses from Observing and all NLN locklosses (can include commissioning/maintenance times), including:

• No IMC locklosses August to September 13th (Iain checked prior to August G2401576)
• Setpeber 12th: IMC gain redistribution 79998
• October 9th: IMC gain redistribution reverted 80566
• October 15th: NPRO current increased 80687
• Ocotber 18th: NPRO current reverted 80746

J. Freed

I followed Step 1 of the double mixer test plan in T2400327 under "1. Characterize frequency references." 

PNSPIDACTestStand.pdf Shows the results in dBc/Hz. And PNSPIDACTestStandrad.pdf shows the result in rad/sqrt(Hz).

Besides the 80MHz standard reference there were 2 devices under test, the IFR 2023A and the SRS SG382. These two devices went under 3 tests involving different combinations of

1. Timing to a 10MHz frequency standard produced by the 80MHz Standard. (Time Standard)

2. PLL locking to a different 80MHz frequency ref through the tune in port. (Lock)

During this process it was assumed that the 80MHz standard and the added 80MHz ref would have a simmilar noise profile as they come from a simmialar OCXO.

A 4th test was added to the Time Standard/No Lock test which used the SR785s internal high pass filter (-3dB at 0.16Hz) to remove some of the DC components from the test. This is because the results of the initial test were inconclusive as the noise floor of the SR785 was too high. The noise floor was too high because there was a DC signal that caused the input range of the SR785 to be about 20dB. The high pass filter, removed the DC signal, lowering the noise floor at the cost of signals below about 0.2Hz not being accurate. We believe this is caused by the fact that while the PLL locking can be controled (tries to lock signals to destructivly interfere so to lower output power for Phase noise tests), there was no control by the Time Standard on phase differences between oscilators. As Time Standard/ No lock, showed the best results above 0.06Hz and is closest to the SPI set up design, this fact may be important when doing futher tests.

Between IFR and SRS, the SRS showed better phase noise performance below 7kHz. While IFR showed better performance above 7kHz. As such, SRS shows more promise in SPIs phase noise range of interest 0-4096Hz.

Lockloss @ 10/30 20:17UTC due to 4.8 earthquake in Oregon. We lost lock very soon after I noticed it coming in on Picket fence. We had been Locked for 2 hours 22 mins

Our frequency noise and contrast defect (measured at the OMC) are lower than they ever have been in O4 (contrast defect, frequency noise). As a part of trying to track down what may have caused this reduction, I tracked the coherence of DARM with the LSC REFL A RIN, using that as a witness to frequency noise, as it was used in OAF for online frequency noise cleaning in O4a (source: 72276 and poking around the OAF screens).

I tried to pick times that were a few hours into the lock to avoid thermalization confusion. I also used only observing data.

There are a couple ideas about what could have caused this improvement. A short list:

• ring heater/power changes: we have stepped up and down the ring heaters by 0.1 W to avoid PIs, and briefly stepped up the input power by 1 W for the same reason
• AS WFS yaw offset: this was turned on due to an investigation in the O4 break, and then turned off after the OFI vent recovery
• OFI change: this doesn't help match the arms, but changes the mode matching to the OMC which could effect the contrast defect

These are the times I used. All of these times were at the same 60 W input power from the PSL:

Dec 10 2023: O4a time, no WFS offset in use, EY ring heater set to 1 W (1386244818)

Apr 11 2024: O4b time, before the OFI problems, WFS offset in use, EY ring heater set to 1 W (1396950886)

Jul 10 2024: O4b time, just before OFI repair vent, WFS offset in use, EY ring heater set to 1 W (1404656647)

Sept 14 2024: O4b, after OFI repair, WFS offset in use, EY ring heater set to 1 W (1410432000)

Sept 20 2024: O4b, after OFI repair, WFS offset set OFF, EY ring heater set to 1 W (1410855190)

Oct 22 2024: O4b, after OFI repair, WFS offset set OFF, EY ring heater set to 1.1 W (1413635762)

Here are some notes for the comparison of these times:

• the frequency noise coherence dropped significantly from O4a to O4b (red trace to blue trace)
• the frequency noise coherence increased again after the problems with the OFI (blue trace to green trace)
• the frequency noise coherence improved again after the OFI vent recovery (green trace to brown trace)
• the frequency noise coherence improved further after we turned off the WFS offset (brown trace to pink trace)
• the frequency noise coherence stayed the same when we changed the EY ring heater to 1.1 W (pink trace to light blue trace)

All of these O4 times show less coherence than the O4a time. Based on this data, it seems like the WFS offset did have an impact on the frequency noise. It also seems like the various vents with output port changes could affect the frequency noise, but the overall beam alignment in the arms could have changed during the vent. For example, we did adjust the camera offsets/ADS gains during the vent commissioning times. The change in frequency noise during the OFI problems (between April and July) could have a similar source, since we had to change a lot of alignment during that time. I'm not sure if any arm alignment was significantly changed though. Finally, it seems like this small ring heater change had no effect on the frequency noise.

Continuing on from Jenne's observation that there are still glitches in the new NPRO, I've tried to make a plot we can use to compare the glitch rate in the new and old NPROs, using the NPRO_PWR channel instead of the FSS channel which isn't available for the new NPRO. 

I've used a 3 hour stretch of observing time for the old NPRO, and a three hour stretch before the time when Jenne made a plot in 80837.  The ISS is not on for the new NPRO time, which is probably why the intensity is flcutuating and making it harder to see the small steps in power that are the glitches we are looking for.  In the second panel, I've plotted the data high passed with a 0.05 Hz butterworth, this helps to show the glitches, although not perfectly (in either case).  Based on this, the glitches look to be happening at a roughly similar rate, although somewhat less with the new NPRO.

This script in in sheila.dwyer/DutyCycle)4/dutycycleplots/PSL_glitches.py

-Brice, Sheila, Camilla

We are looking to see if there are any aux channels that are affected by certain types of locklosses. Understanding if a threshold is reached in the last few seconds prior to a lockloss can help determine the type of lockloss, which channels are affected more than others, as well as

We have gathered a list of lockloss times (using https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi) with:

1. only Observe and Refined tags (plots, histogram)
2. only Observe, Refined, and Windy tags (plots, histogram)
3. only Observe, Refined, and Earthquake tags (plots, histogram)
4. Observe, Refined, and Microseism tags (note: all of these also have an EQ tag, and all but the last 2 have an anthropogenic tag) (plots, histogram)

(issue: the plots for the first 3 lockloss types wouldn't upload to this aLog. Created a dcc for them: G2401806)

We wrote a python code to pull the data of various auxilliary channels 15 seconds before a lockloss. Graphs for each channel are created, a plot for each lockloss time are stacked on each of the graphs, and the graphs are saved to a png file. All the graphs have been shifted so that the time of lockloss is at t=0.

Histograms for each channel are created that compare the maximum displacement from zero for each lockloss time. There are also a stacked histogram based on 12 quiet microseism times (all taken from between 4.12.24 0900-0930 UTC). The histrograms are created using only the last second of data before lockloss, are normalized by dividing by the numbe rof lockloss times, and saved to a seperate pnd file from the plots.

These channels are provided via a list inside the python file and can be easily adjusted to fit a user's needs. We used the following channels: