aLIGO LHO Logbook

H1 ISC - Comment to Comparison of PSL glitches before locklosses tagged: glitches dramatically changed after NPRO swap

victoriaa.xu@ligo.org — Wed, 20 Nov 2024 18:43:50 -0800

Author: victoriaa.xu@ligo.org

Report ID: 81393

Comparing spectra of the FSS_FAST_MON (calibrated Hz/rtHz), ISS_AOM_DRIVER control signal, and FSS_TPD ref cav trans.

Green = O3 laser in O4 (2024)
Red = O3 laser at the end of O3b (2020)
Blue = O4 original laser

FSS FAST MON looks higher for O3 laser vs. the original O4 laser (maybe this is related to the laser itself?). But ISS spectra of the same O3 laser was noisier in O3 than O4, and similarly the FSS_TPD for the same O3 laser was noisier in O3 than in O4.

Images attached to this report

H1 PSL - Comment to PMC Glitch Testing

victoriaa.xu@ligo.org — Wed, 20 Nov 2024 18:39:28 -0800

Author: victoriaa.xu@ligo.org

Report ID: 81391

Sheila, Daniel, Jason, Ryan S, many others

We think the problem is not the PMC, and likely the laser.

Daniel looked at PMC mixer gliches on the remote scope: 81390. If PMC mixer glitches are indicative of the problem, we can try to track down the origin of the glitches.

What can cause PMC mixer glitches?
1. NPRO Laser frequency glitches .... seems likely
2. PMC cavity length glitches (e.g. servo electronics or PZT) .... most likely no 81390.
3. Likely PDH is optically sensing real glitches, as we have already switched the 35.5 MHz LO to the Marconi 81277, that made no difference ...... rule out the 35.5 MHz LO.
Unlikely PMC, because the PMC mixer glitches on the scope seem too fast for the PMC locking electronics.
1. Daniel found that the "PMC servo drives the PZT with a 3.3K series resistor forming a ~1kHz low pass with the PZT capacitance of ~45nF. This yields a chracteristic time constant of ~150us. The glitches as seen by the PMC mixer are at least a few times faster than this." 81390
2. So, this means that glitches originating in the PMC locking servo or PZT electronics would change the PMC cavity length too slowly to create the fast PMC mixer glitches we observe. Seems unlikely that PMC servo locking electronics are the issue.
3. Unlikely mechanical / PZT issues given the very nice PMC cavity scans - optically, the cavity scans don't show glitches or issues as a function of the PZT drive (remote scope traces + sheila pmc pzt scan photo).
Biggest change to the glitches came from swapping the original O4 laser to now the O3 laser. Glitches changed when lasers changed: see Camilla lho:81386. Seems like a clue that the glitches are related to the specific lasers themselves.

Talking with Jason and RyanS, some summary of the NPROs available:

(S/N 1661) The O3 NPRO was swapped in recently. Potentially we now see the same problems with 1661 was seen in the end of O3b.
- Camilla's 81386 shows that the FSS oscillations with this laser in O4 look a lot like its issues at the end of O3b.
- In 2020, these FSS issues originally prompted Camilla to create the FSS_OSCILLATION tags. These FSS_OSCILLATION tags started to be flagged regularly again in mid-Sept 2024, but with the original O4 laser, the FSS oscillations looked totally different: see her log 81386.
(S/N 7974) The original O4 NPRO was in use Sept 2017 - June 2018, when it was uninstalled (42652) seemingly due to similar fast FSS glitches, leading to a broken noise eater. Jason looked at alogs from time of removing this NPRO in 2018, and the 2018 glitches look like the original Sept 2024 glitches. 7974 was then sent back to Coherent for repair + refurbished. This is the newest NPRO bought in 2015, and the only one that Coherent will service. Then 7974 was installed for O4.
(S/N 1639F) The "third" NPRO is from O1, and was in use for many years 2011-2017. It was wearing out its lifetime and running out of power, then removed and refurbished. Since, was only used a few times in the optics lab. Will try this 3rd NRPO.

ISC, PSL

Images attached to this report

H1 PSL - Comment to Fast time domain trace of the PMC glitches

daniel.sigg@ligo.org — Wed, 20 Nov 2024 17:15:28 -0800

Author: daniel.sigg@ligo.org

Report ID: 81392

A tend plot of a glitch train together with some PEM channels.

Images attached to this report

H1 PSL - Fast time domain trace of the PMC glitches

daniel.sigg@ligo.org — Wed, 20 Nov 2024 17:07:02 -0800

Author: daniel.sigg@ligo.org

Report ID: 81390

The attached plots show the PMC mixer error signal and the ISS PDB readout. The later implements a fair amount of whitening so tends to fluctuate at a slow rate compared to the glitches.

The PMC error signal goes as high as 1.25Vpp during a PDH sweep. The channel recorded by the DAQ has a lot of gain and clips around +/-80mV, but it is calibrated correctly. So, the calibration is 1.25V/1.19MHz = 1.05V/MHz.

First plot shows a train of glitches, going as high as ~70mVpk (or ~70kHz).

The second plot shows a zoomed in version of a glitch of about the same size. The PMC servo drives the PZT with a 3.3K series resistor forming a ~1kHz low pass with the PZT capacitance of ~45nF. This yields a chracteristic time constant of ~150us. The glitches as seen by the PMC mixer are at least a few times faster than this.

The third plot shows a PDH scan.

The forth plot shows a typical trace when nothing happens.

Images attached to this report

H1 General - OPS Wednesday DAY shift summary

ryan.crouch@ligo.org — Wed, 20 Nov 2024 16:32:13 -0800

Author: ryan.crouch@ligo.org

Report ID: 81378

TITLE: 11/20 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Microseism
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY: We spent the day in IDLE again investigating the PSL issues, we've had only the PMC locked for most of the day. At the end of the shift, the power supply for the PLS was swapped. Microseism has slowly starting to come down in the last ~5 hours. We are going to remain DOWN again tonight.
LOG:

Start Time	System	Name	Location	Lazer_Haz	Task	Time End
21:17	SAF	LVEA LASER SAFE	LVEA	N	LVEA IS LASER SAFE	01:17
17:05	PSL	Jason	CER, LVEA	N	Revive the PSL NPRO	17:17
17:26	FAC	Karen	Optics lab, vac prep	N	Tech clean	18:26
17:40	PSL	Jason, Marc	CER	N	Turn off PMC HV and unplug db37 cable	17:46
18:41	FAC	Christina	Mids	N	Check out storage	20:02
18:48	PSL	Jason	LVEA, CER	N	Take pictures	18:51
19:03	FAC	Kim	H2 enc	N	Tech clean	20:17
19:13	FAC	Karen	Woodshop, fire pump	N	Tech clean	19:26
19:25	VAC	Janos	MidY	N	Mech room, pump checks	20:25
21:37	VAC	Janos	MidX	N	Pump checks	22:00
23:10	PSL	Jason, RyanS	LVEA, Optics lab	N	Grab power supply and swap it	23:43
23:56	PSL	Fil	CER	N	Plug back in DB37 for the FSS	00:08

15:00:26 UTC a minor frequency power glitch alog81376 earlier this morning tripped off the NPRO. The power discrepancy only lasted ~50ms.
17:15 UTC Jason brought the NPRO back
2ndary microseism seems to be decreasing, satellite images show the cyclone moving NW away from the WA coast today
17:55 UTC Jason and Marc turned off the PMC high voltage and unplugged the DB376 cable again to resume the test from yesterday (No IMC, FSS, or ISS, only PMC)
I ran a spectra of the PMC_MIXER_OUT_DQ channel comparing times of similar range from the good lock we had on 11/19/24 to a lock back in 09/07/24

Images attached to this report

H1 ISC - Comment to Comparison of PSL glitches before locklosses tagged: glitches dramatically changed after NPRO swap

camilla.compton@ligo.org — Wed, 20 Nov 2024 13:49:40 -0800

Author: camilla.compton@ligo.org

Report ID: 81389

Adding in plots from the end of O3b: 30 seconds and zoomed to 1 second before locklosses tagged FSS. The FSS channel looks simular to after the NPRO swap, but were worse in O3b.

Images attached to this report

H1 TCS - TCSY CO2 air filter replaced and small coolant leak found

thomas.shaffer@ligo.org — Wed, 20 Nov 2024 13:37:27 -0800

Author: thomas.shaffer@ligo.org

Report ID: 81388

Yesterday I went to check on the chillers and found a bit of coolant on the CO2Y chiller air filter and on the bottom of the unit. The last time we saw this we had inconsistent supply temperatures, something we don't really see now but might explain our more frequent lock losses lately (alog 81362). We'll keep an eye on this and swap the chiller if necessary. It would be great if this could make it another 6 months and then we can send them in for service.

I replaced the air filter since it had a handful of dead bugs, coolant, and other junk on it. I used up our last filter so I'll have to order more.

Images attached to this report

H1 ISC - Comparison of PSL glitches before locklosses tagged: glitches dramatically changed after NPRO swap

camilla.compton@ligo.org — Wed, 20 Nov 2024 12:45:39 -0800

Author: camilla.compton@ligo.org

Report ID: 81386

Some of the "PSL story so far" is in 81193. Code saved to /camilla.compton/Documents/Locklosses/PSL_channels.ipynb

I've plotted H1:PSL-PWR_NPRO_OUT_DQ and H1:PSL-FSS_FAST_MON_OUT_DQ in the 30 seconds and 1 second before each locklosss tagged both IMC and FSS by the lockloss tool.

There is clear difference in the channels before and after the NPRO swap:

Plots before the NPRO swap here and zoomed here. There is large individual glitches, seen by both FSS and the NPRO PWR in an otherwise quiet signal before the NPRO swap.
Plotted after the NPRO swap here and zoomed here. There is none of these individual glitches after the swap but there is the slower building oscillation in the FSS channel, which is overall noisier.

The FSS oscillations we are seeing now are similar to the FSS_OSCLATION locklosses we saw at the end of O3b (G2201762: O3a_O3b_summary.pdf). Note that the current NPRO is the same NPRO as we used in O3. Is it possible that we fixed the glitches issues but reinstalled a laser with similar issues as we saw in O3b? We could revisit that time period to see if we had similar IMC locking issues then too.

Reminder that the lockloss tool tags "IMC" if the IMC looses lock within 50ms of the arms (ASC_AS_A) and tags "FSS" if H1:PSL-FSS_FAST_MON_OUT_DQ with outside of +/- 3 within 1-5 seconds before the lockloss.

Lockloss, PSL

Images attached to this report

H1 PSL - Comment to (My) First Attempt at Measuring H1 PSL PMC Cavity Sweep (Process and Raw Data)

elenna.capote@ligo.org — Wed, 20 Nov 2024 12:37:26 -0800

Author: elenna.capote@ligo.org

Report ID: 81387

Sheila and I used this data to get a rough calibration of the PZT HV channel: 0.0236 V/MHz, as in alog 81385.

H1 PSL - PMC Calibration

elenna.capote@ligo.org — Wed, 20 Nov 2024 12:36:27 -0800

Author: elenna.capote@ligo.org

Report ID: 81385

Sheila and I spent some time today trying to calibrate the PMC channels into units of frequency so we can compare the glitches seen by the PMC with other channels. Peter's alog comment last night (81375) leads us to understand the glitches are around 2 kHz in frequency, so above the PMC bandwidth. Therefore, we want to calibrate the PMC error signal.

Luckily for us, Jeff Kissel recently did some PMC scans to determine the PMC PZT Hz/V calibration for a different purpose, alog 73905 (thanks Jeff!). We used his DTTs to determine the time it took to scan one FSR, see screenshot. We determined that it took 0.52 seconds to scan one FSR, and the scan rate is approximately 6.75 V/s (we know the PZT is nonlinear, but we figure this estimate is good enough for our purposes). This gives us 3.51 V/FSR on the PZT, or 0.0236 V/MHz, using the FSR = 148.532 MHz from T0900616.

We are still thinking about how to calibrate the PMC mixer signal.

ISC

Images attached to this report

LHO VE - Wed CP1 Fill

david.barker@ligo.org — Wed, 20 Nov 2024 10:47:57 -0800

Author: david.barker@ligo.org

Report ID: 81384

Wed Nov 20 10:11:17 2024 INFO: Fill completed in 11min 13secs

Images attached to this report

H1 AOS - SQZ TTFSS error check changes, PSL FSS off

sheila.dwyer@ligo.org — Wed, 20 Nov 2024 10:17:28 -0800

Author: sheila.dwyer@ligo.org

Report ID: 81383

Vicky, Sheila

Last night the squeezer FSS was off during our test of running the PMC with the FSS unlocked and unplugged. This is because of some error checking that was written when the squeezer (and ALS) pick offs were from the reference cavity transmission, so there were too check on the reference cavity transmitted power, and an error check on the reference cavity transmission PD. Since the reference cavity was unlocked the automation had an error overnight and didn't lock the SQZ TTFSS.

This morning we changed the H1:PSL-REFCAV_TRANS_DC_LOW from 0.3 to -0.1, and changed H1:SQZ-FIBR_LOCK_REFCAV_TRANSLIM to -0.1, so that the squeezer FSS will stay locked.

The PSL FSS has been powered down with the squeezer FSS on since 17:45 UTC Nov 20th.

H1 PSL - Comment to PMC Glitch Testing

victoriaa.xu@ligo.org — Wed, 20 Nov 2024 09:47:44 -0800

Author: victoriaa.xu@ligo.org

Report ID: 81381

Sheila, Jason, Vicky - Compared the PSL + PMC mixer glitches between last night (Nov 22, 2024, no FSS no ISS) and the emergency vent (Aug 2024, PSL+PMC+FSS+ISS), as in 81354.

Previously 80999, the IMC could not stay locked for long periods of time: "IMC losses lock when it's locked by itself, with PMC, FSS, and ISS."
Fixed a bunch of things last week, but then still had many fast "IMC"-tagged locklosses over the weekend. Hard to tell the cause.
Start removing stuff again this week, to isolate problem upstream of the IMC.
Laser + PMC + FSS (no ISS) = bad still glitchy 81356. Rules out ISS and likely IMC as cause of glitches.
Laser + PMC (no FSS no ISS) = bad still glitchy 81371. FSS depowered and HV off to really turn FSS "off". Rules out FSS as cause of glitches.

As a reference, "before" during the emergency vent in August 2024, the Laser + PMC + FSS + ISS were all locked with no PMC mixer glitches for >1 month.

Updating our matrix of tests to isolate the problem, and thinking things through:

Before (vent Aug 2024)	Now (Nov 2024)
laser + PMC + FSS + ISS good = no glitches	laser + PMC + FSS bad = glitches 81356
laser + PMC ??? (presumably good)	laser + PMC bad = same PMC mixer glitches 81371

1) Are these +/-0.1 V PMC mixer glitches the problem? Yes, probably.

2) Are these big PMC mixer glitches caused or worsened by the FSS? No. PMC mixer glitches basically same with FSS on 81356 and off 81371.

3) Are the laser + PMC mixer glitches new? Yes, probably. If these PMC glitches always there, could it be that we were previously able to ride them out before, and not now? But this would imply that in addition to the new glitches, the FSS secondarily degraded. Seems very unlikely: already several problems (bad amp, new eom needed new notch, etc) have been fixed with FSS, and the FSS OLTFs and in-loop spectra look good. FSS on/off does not change the PMC mixer glitches, so the problem seems most likely laser or PMC.

Images attached to this report

H1 CDS - Comment to h1susex powered down for DAC change

david.barker@ligo.org — Wed, 20 Nov 2024 09:45:29 -0800

Author: david.barker@ligo.org

Report ID: 81382

Tue19Nov2024
LOC TIME HOSTNAME MODEL/REBOOT
09:59:38 h1susex h1iopsusex
09:59:51 h1susex h1susetmx
10:00:04 h1susex h1sustmsx
10:00:17 h1susex h1susetmxpi

H1 PSL - Comment to PMC Glitch Testing

sheila.dwyer@ligo.org — Wed, 20 Nov 2024 08:55:20 -0800

Author: sheila.dwyer@ligo.org

Report ID: 81379

After Ryan's shift ended last night, there were some larger glitches, with a similar amplitude in the PMC mixer channel to the ones that we saw unlocking the reference cavity 81356 (and IMC in 81228)

The first plot shows one of these times with larger glitches, the second one zooms in for 60ms when the glitches were frequent, this looks fairly similar to Peter's plot above.

The period of large glitches started around 2 am (7:37 UTC on Nov 20th), and ended when a power glitch turned off the laser at 7 am (15 UTC) 81376. Some of the small glitches in that time frame time seem to be at the same time that the reference cavity was resonanting (with low transmission), but many of the large glitches do not line up with times when the reference cavity was resonanting.

I've zoomed in on most of the times when the PMC mixer glitches reached 0.1, and see that there are usually small jumps in NPRO power at the time of these glitches, although the times don't always line up well and the small power glitches are happening very often so this might be a coincidence.

Images attached to this report

H1 General - Comment to OPS Wednesday DAY shift start

ryan.crouch@ligo.org — Wed, 20 Nov 2024 08:53:22 -0800

Author: ryan.crouch@ligo.org

Report ID: 81377

The minor frequency power glitch earlier this morning 07:00:26 tripped off the NPRO, 50ms after the glitch started the NPRO tripped.

PSL

Images attached to this report

H1 CDS - More EY CNS-II 1PPS jumps to -0.8uS

david.barker@ligo.org — Wed, 20 Nov 2024 08:43:21 -0800

Author: david.barker@ligo.org

Report ID: 81380

For about 50 minutes starting at 01:45 PST this morning we had another jump of the EY CNS-II reference GPS 1PPS compared with the LIGO timing. Previous jump was Mon 18nov2024 05:55. Prior to that no jumps for 3 weeks since the power supply was replaced.

Images attached to this report

H1 CDS - Power glitch 07:00:26 PST

david.barker@ligo.org — Wed, 20 Nov 2024 08:05:17 -0800

Author: david.barker@ligo.org

Report ID: 81376

We had a slight mains power glitch at 07:00:26 PST this morning, the GC UPS reported being on battery power for 4 seconds.

Attached plot shows all three phases in the corner station CER.

Images attached to this report

H1 PSL - Comment to PMC Glitch Testing

peter.fritschel@ligo.org — Wed, 20 Nov 2024 07:46:20 -0800

Author: peter.fritschel@ligo.org

Report ID: 81375

A zoom-in on some of the glitches from the third figure above.

Images attached to this report

H1 PSL - FSS high voltage restored

ryan.crouch@ligo.org — Wed, 20 Nov 2024 07:36:58 -0800

Author: ryan.crouch@ligo.org

Report ID: 81374

Richard turned back on the high voltage and plugged back in the DB37 cable that was unplugged yesterday afternoon.

H1 General - OPS Wednesday DAY shift start

ryan.crouch@ligo.org — Wed, 20 Nov 2024 07:30:57 -0800

Author: ryan.crouch@ligo.org

Report ID: 81373

TITLE: 11/20 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Microseism
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM_EARTHQUAKE
    Wind: 17mph Gusts, 11mph 3min avg
    Primary useism: 0.40 μm/s
    Secondary useism: 1.56 μm/s
QUICK SUMMARY:

Wind is moderate, 2ndary microseism is quite high. We're also being shook by a 4.9 from the Mariana islands
The IMC, FSS, PMC, and ISS are all unlocked

H1 PSL - PMC Glitch Testing

ryan.short@ligo.org — Tue, 19 Nov 2024 22:13:01 -0800

Author: ryan.short@ligo.org

Report ID: 81371

Since the secondary microseism has been very high this evening and preventing H1 from locking, we decided to leave just the PMC locked (no FSS, ISS, or IMC) for an extended time and watch for any glitches. At around 23:45 UTC, we unlocked the FSS, Richard turned off the high voltage supply for the FSS, and Jason and I unplugged the DB37 cable from the FSS fieldbox in the PSL-R1 rack in order to ensure no feedback from the FSS made it to the NPRO. Pictures of the DB37 cable's location are attached.

The first attachment shows the changes seen when the FSS was unlocked. Since then, I've seen several instances of groups of glitches come through, such as those shown in the second and third attachments. These glitches in the PMC_MIXER channel are smaller than ones seen previously that have unlocked the IMC (like in alog81228). There have also been times where the PMC_MIXER channel gets "fuzzier" for a bit and then calms down, shown in the fourth attachment; it's possible this is due to the NPRO frequency not being controlled so the PMC sees some frequency changes? Finally, I only recall one instance of the NPRO jumping in power like in the final attachment; the PMC doesn't seem to care much about this, only having one very small glitch at this time.

I'll leave the PSL in this configuration to collect more data overnight as the secondary microseism is still much too high for H1 to successfully lock.

ISC

Images attached to this report

LHO General - Ops Eve Shift Summary

ryan.short@ligo.org — Tue, 19 Nov 2024 22:10:56 -0800

Author: ryan.short@ligo.org

Report ID: 81372

TITLE: 11/20 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Microseism
INCOMING OPERATOR: None
SHIFT SUMMARY: H1 remained unlocked the whole evening due to very high secondary microseismic motion, and there is no plan to attempt locking overnight. In the meantime, we took the opportunity to leave just the PMC locked to look for glitches (more details in alog81371). Very quiet evening otherwise with nothing else to report.

Images attached to this report

H1 CDS - BSC3 vacuum gauge glitch, vacstat reset

david.barker@ligo.org — Tue, 19 Nov 2024 21:28:34 -0800

Author: david.barker@ligo.org

Report ID: 81370

At 17:52:10 PST Tue 19 Nov 2024 we had another sharp positive glitch in the BSC3 vacuum gauge signal (PT132_MOD2). VACSTAT did not alarm because no other gauge tripped around this time. The glitch was a 2 second wide square wave, and as before it was the delta-P which tripped.

vacstat was reset at 18:39 to restore it to its monitoring state.

Images attached to this report

H1 PSL - Comment to PSL PMC + FSS was continuously locked over emergency vent

victoriaa.xu@ligo.org — Tue, 19 Nov 2024 17:53:45 -0800

Author: victoriaa.xu@ligo.org

Report ID: 81369

Trying to compare PSL-related spectra between the original O4 NPRO and the current NPRO using DTT is kinda confusing.

Comparing times with before O4 NPRO from 21 Oct 2024 04:55 UTC (blue, 169 Mpc), and now with O3 NPRO at 16 Nov 2024 08:20 UTC (red, 169.9 Mpc).

For the fast mon spectra in the top left plot, FSS FAST MON OUT looks 2-5x noisier now than before. H1:PSL-FSS_FAST_MON_OUT_DQ calibrated into Hz/rtHz using zpk = ([], [10], [1.3e6]) (see 81251, this makes H1 and L1 spectra into comparable Hz/rtHz units 81210).

But for FSS-TPD (ref cav trans), it looks like there's some extra 1-10Hz noise, but otherwise the trans spectra might be quieter? Similarly confusing, the ISS AOM control signal looks quieter. Not a clear takeaway from just these spectra on how to compare O3/O4 NPROs.

Images attached to this report

H1 DAQ - PCALX_STAT Guardian Node

anthony.sanchez@ligo.org — Tue, 19 Nov 2024 17:18:58 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81368

I startedtesting the PCALX_STAT Guardian node today.

/opt/rtcds/userapps/release/cal/h1/guardian/PCALX_STAT.py

It created a new ini file, But the DAQ was not restarted after this new ini file creation.
As it currently stands this is a draft of the final product that will be tested for a week and further refined.
This Guardian node, does not make any changes to the IFO, it's only job is to determine if PCALX arm is broken or not. TJ has already added it to the Guardian Ignore list.

CAL, CDS, GRD

H1 AOS - Comment to NN Seismic Array HS-1 Install

jim.warner@ligo.org — Tue, 19 Nov 2024 16:45:02 -0800

Author: jim.warner@ligo.org

Report ID: 81367

Fil and I tried again today, we couldn't get this sensor to work. We started from the PEM rack in the CER, plugging the HS1 through the SR560 into the L4C interface chassis, confirming the HS1 would see something when we tapped it. We then moved out to the PEM bulkhead by HAM4, again confirmed the HS1/SR560 combo still showed signal when tapping the HS1. Then we moved to the biergaren and plugged in the HS1/SR560 right next to the other seismometers. While watching the readout in the DAQ of the HS1 and one of the Guralps I have connected to the PEM AA, we could see that both sensors could see when I slapped the ground near the seismometers, but the signal was barely above what looks like electronics noise on the HS1, while the Guralp showed lots of signal that looked like ground motion. We tried gains from 50-200 on the SR560, none of them really seemed to improve the snr of the HS1. The HS1 is still plugged in over night, but I don't think this particular sensor is going to measure much ground motion.

SEI

H1 General - OPS Tuesday day shift summary

ryan.crouch@ligo.org — Tue, 19 Nov 2024 16:35:16 -0800

Author: ryan.crouch@ligo.org

Report ID: 81349

TITLE: 11/19 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY: 2ndary microseism has been consistently high (above the 90th percentile) for the whole day and has risen even more in the last few hours of the shift and has reached the top of the plot. It will likely will get even worse tomorrow as the storm off the pacific coast closes in. Wind has been pretty low today. ITMY5/6 is still kind of high.

As of 23:30 UTC we're back to sitting in idle, we unlocked the FSS, ISS, and IMC just leaving the PMC locked.
LOG:

Start Time	System	Name	Location	Lazer_Haz	Task	Time End
21:41	SAF	Laser	LVEA	YES	LVEA is laser HAZARD	17:55
16:08	FAC	Eric	EndX	N	Glycol piping famis task, upright fallen portapotty by FCES	16:50
16:14	TCS	TJ, Camilla	LVEA	Y	CO2Y table cable jiggling	17:01
16:17	FAC	Kim & Karen	FCES	N	Tech clean	16:31
16:19	FAC	Tyler	LVEA	N	Verify strobes in H2/PSL area	16:28
16:20	FAC	Tyler + Cascade Fire	OSB, Ends	N	Fire alarm testing	18:54
16:27	FAC	Tyler, Tristan	EndX	N	Fire alarm testing	18:47
16:30	FAC	Kim	EndX	N	Tech clean	17:19
16:31	FAC	Karen	EndY	N	Tech clean	17:35
17:00	ISC	Sheila	LVEA	Y	Unplug ISS Feed forward	17:08
16:34	VAC	Jordan	LVEA	Y->N	Prep new scroll pumps, gauge swap	20:04
16:41	FAC	Chris + Fire	LVEA	Y	Check fire ext.	17:10
16:59	VAC	Gerardo	LVEA	Y->N	Join Jordan, Scroll pumps	20:04
17:03	OPS	Camilla	LVEA	Y -> N	LASER HAZARD transition	17:20
17:08	FAC	Chris + Cascade fire	EndY	N	Fire checks, End then mid	18:54
17:10	VAC	Janos	LVEA	Y	Join VAC team	17:53
17:20	CDS	Marc, Fil	EndX	N	DAC swap	18:12
16:45	EPO	Corey+1	EndY	N	Pictures	17:15
17:30	FAC	Kim	FCES	N	Tech clean	18:00
17:32	VAC	Janos	LVEA	N	Pump/Gauge work	18:35
17:36	FAC	Karen	FCES	N	Tech clean	18:00
17:42	OPS	Oli	LVEA	N	Grab power meter	17:47
17:47	SEI	Jim	CR	N	BS and ITM sei tests	18:22
17:00	ISC	Daniel	LVEA	Y	Investigations, scope installation	17:52
17:56	SAF	LASER	LVEA	N	LVEA is LASER SAFE	20:46
17:57	ALS	Camilla, Oli	EndY	Y	Beam profiling	19:54
18:01	FAC	Richard	LVEA	N	Walk around	18:21
18:02	ISC	Keita	LVEA	N	Cable checks	18:29
18:16	ISC	Sheila	CER	N	Plug and unplug FF cable	18:40
18:22	VAC	Janos	EndX then EndY	N	Scroll pumps	19:25
18:22	SEI	Jim	LVEA Biergarten / CR	N	working on Seismic sub systems	19:55
17:30	EPO	Corey+1	LVEA	N	B-Roll	18:25
18:37	FAC	Kim & Karen	LVEA	N	Tech clean, Kim out 19:37	19:53
18:54	FAC	Tyler+Cascade	Mids	N	Fire checks	20:20
19:25	VAC	Janos	LVEA	N	Join VAC team	20:04
19:56	SEI	Jim	LVEA	N	Take pictures	20:06
20:03	OPS	Oli	LVEA	N	Return power meter	20:11
20:11	CAL	Tony	PCAL lab	Y	Make it laser safe	21:07
20:40	CAL	Francisco	PCAL lab	Y	PCAL work	22:28
20:43	OPS/TCS	TJ	LVEA	N-> Y -> N	HAZARD TRANSITION then CO2Y table adjustment, back to safe	21:13
20:48	TCS	Camilla	LVEA	Y	Join TJ, CO2Y table	21:13
21:17	SAF	LVEA LASER SAFE	LVEA	N	LVEA IS LASER SAFE	01:17
21:31	ISC	Keita	LVEA	N	AS_B SEG3 testing	21:49
21:47	OPS	Oli	LVEA	N	Sweep	22:13
23:37	PSL	RyanS, Jason	CER, PSL racks	N	Pull cable

IFO/Locking:

16:35 UTC While we were down Sheila set the IMC to offline and turned off the ISS first loop off (PMC and FSS locked).
16:53 UTC HAM7 ISI tripped, from a CPS glitch, possibly from the fire crew checking out extinguishers. This is a sensitive area, people walking around can cause CPS glitches.
17:22 UTC LVEA back to LASER SAFE (Transition)
21:23 UTC I started relocking, ALS_X had to be recovered due to the restart. I just trended top mass OSEMs and set the sliders accordingly
21:55 - 22:02 UTC I ran a manual IA to align the SRC as DRMI didn't look good but PRMI did. DRMI looked great afterwords
22:00 UTC Oli swept the LVEA alog81363
22:15 UTC "FAST_SHUTTER test failed", Keita's AS_B FM4 gain filters were still on when they shouldn't have been
23:12 UTC Lockloss at MOVE_SPOTS, lockloss tool. ASC loops were working pretty hard
23:30 UTC ISC_LOCK to IDLE

Some of the maintenance work completed includes:

Glycol piping checks
Fire alarm testing & inspection around site (confirmed PSL encl strobes are working)
CO2Y table loose cables investigated and beam mask adjustment alog81362
New VAC scroll pumps at LVEA and Ends
Replaced bad pressure gauge at PT154 alog81361
EPO pictures and videos of site
ALS beam profiling alog81358
ETMX DAC swap, back to the old DAC alog81350
LVEA oscilloscope installed alog81359
IMC testing alog81356
- IMC offline ISS off, PMC and FSS locked
- Unplugging and plugging in ISS FF cable
- IMC offline, ISS off, FSS off, only PMC

We did not do a DAQ restart

Laser Transition

Images attached to this report

H1 ISC - Beam Profile Measurements of ALS-Y path

camilla.compton@ligo.org — Tue, 19 Nov 2024 16:04:04 -0800

Author: camilla.compton@ligo.org

Report ID: 81358

Oli, Camilla WP12203. Repeat of some of the work done in 2019: EX: 52608, EY: 52636, older: part 1, part 2, part 3.

We misaligned ITMY and turned off the ALS-Y QPD servo with H1:ALS-Y_PZT_SWITCH and placed the Ophir Si scanning slit beam profiler to measure both the 532nm ALSY outgoing beam and the ALSY return beam in the HWS path.

The outgoing beam was a little oblong in the measurements but looked pretty clean and round by eye, the return beam did not! Photos of outgoing and return beam attached. Outgoing beam was 30mW, return beam 0.75mW.

Attached is the 13.5% and D4sigma measurements, I also have photos of the 50% measurements if needed. Distances are measured from the optic where HWS and ALS beams combine, ALS-M11 in D1400241.

We had previously removed HWS-M1B and HWS-M1C and translated HWS-M1A from whats shown in D1400241-v8 to remove clipping.

TCS

Images attached to this report

Non-image files attached to this report

20241119_EY_beamscan.pdf

LHO General - Ops Eve Shift Start

ryan.short@ligo.org — Tue, 19 Nov 2024 16:00:29 -0800

Author: ryan.short@ligo.org

Report ID: 81366

TITLE: 11/19 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 10mph Gusts, 8mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 1.47 μm/s
QUICK SUMMARY: Due to very rapidly rising microseism, H1 will not be locking this evening or overnight, which allows us to run a long test for PSL glitch hunting by leaving just the PMC locked (no FSS, ISS, IMC). The DB37 cable was unlplugged from the FSS fieldbox at 23:45 UTC.

H1 PSL - Comment to Overview of PSL Story so far

camilla.compton@ligo.org — Tue, 19 Nov 2024 15:06:18 -0800

Author: camilla.compton@ligo.org

Report ID: 81365

Novmeber 14th:
- PMC OLG measured 81283
November 19th:
- Scopes setup at PSL racks 81359
- PMC+FSS only test shows glitches 81356
- Reverted EX 28bit LIGO-DAC change back to 20-bit 81350

ISC

H1 ISC - Comment to ASC-AS_B_DC interface analog gain was put back to HIGH (i.e. +20dB)

ryan.crouch@ligo.org — Tue, 19 Nov 2024 14:46:58 -0800

Author: ryan.crouch@ligo.org

Report ID: 81364

I accepted in SDF in both SAFE and OBSERVE to keep off FM4

Images attached to this report

H1 TCS - Recent TCS CO2Y lock loss and table work today

thomas.shaffer@ligo.org — Tue, 19 Nov 2024 14:29:04 -0800

Author: thomas.shaffer@ligo.org

Report ID: 81362

Camilla C., TJ

Recently, the CO2Y laser that we replaced on Oct 22 has been struggling to stay locked for long periods of time (alog81271 and trend from today). We've found some loose or bad cables in the past that have caused us issues, so we went out on table today to double check they are all ok.

The RF cables that lead into the side of the laser can slightly impact the output power when wiggled, in particular the ones with a BNC connector, but not to the point that we think it would be causing issues. The only cable that we found loose was for the PZT that goes to the head of the laser. The male portion of the SMA that comes out of the laser head was loose, and cannot be tightened from outside of the laser. We verified that the connection from this to the cable were good, but wiggling it did still introduce glitched in the PZT channel. I don't think that we've conviced ourselves that this is a problem though, because the PZT doesn't seem to glitch when the laser loses lock and instead it will run away.

An unfortunate consequence of the cable wiggling was that one of the Beckhoff plugs at the feedthrough must have been unseated slightly and caused our mask flipper read backs to read incorrectly. The screws for this plug were not working so we just pushed the plug back in to fully seat it and all seemed to work again.

We still are not sure why we've been having these lock losses lately, the 2nd and 3rd attachments show a few of them from the last day or so. They remind me of back in 2019 when we saw this - example1, example2. The fix ultimately a chiller swap (alog54980), but the flow and water temp seem more stable this time around. Not completely ruling it out yet though.

Images attached to this report

H1 General - LVEA has been Swept

oli.patane@ligo.org — Tue, 19 Nov 2024 14:17:36 -0800

Author: oli.patane@ligo.org

Report ID: 81363

LVEA has been swept. Just had to unplug a(n already off) PEM function generator and turn off the lights for the HAM5/6 giant cleanroom.

LHO VE - Replacement of Faulty Gauge (PT 154) on FC-A Cross A2

jordan.vanosky@ligo.org — Tue, 19 Nov 2024 13:51:18 -0800

Author: jordan.vanosky@ligo.org

Report ID: 81361

Today we swapped the PT154 gauge on the FC-A section that was reported faulty in alog 81078 with a brand new gauge, same make/model.

FCV1 & 2 were closed to isolate the FCA section, and the angle valve on the A2 cross closed to isolate the gauge. Volume was vented with dry nitrogen and gauges swapped. CF connection was helium leak checked, no He signal above the HLD background ~2e-10 Torr-l/s.

Closing WP 12200

Images attached to this report

H1 ISC - Comment to ASC-AS_B_DC interface analog gain was put back to HIGH (i.e. +20dB)

keita.kawabe@ligo.org — Tue, 19 Nov 2024 13:50:19 -0800

Author: keita.kawabe@ligo.org

Report ID: 81360

After the IMC was relocked I went to the floor, opened the fast shutter, switched the analog gain switch back and forth several times and confirmed that all segments responded correctly.

H1 CDS - Scope setup

daniel.sigg@ligo.org — Tue, 19 Nov 2024 13:44:16 -0800

Author: daniel.sigg@ligo.org

Report ID: 81359

Setup a scope near the PSL rack. The channels are FSS test2, PMC mixer out, ISS PDB, and IMC servo test 1. The trigger has been connected to the IMC REFL shutter. The shutter usually triggers upon a lock loss, or when more than ~4W are detected in reflection of the IMC.

H1 PSL - Comment to PSL PMC + FSS was continuously locked over emergency vent

victoriaa.xu@ligo.org — Tue, 19 Nov 2024 13:20:25 -0800

Author: victoriaa.xu@ligo.org

Report ID: 81357

Some trends showing several PSL signals over O4b, and the end of O4a.

Key points- PMC + FSS stayed locked continuously during both the O4a/b break (Feb 2024), and the emergency vent (Aug 2024), with minimal glitching in PMC TRANS and Ref Cav Trans PD (FSS-TPD).

Images attached to this report

H1 PSL - FSS + PMC test this morning

sheila.dwyer@ligo.org — Tue, 19 Nov 2024 12:46:49 -0800

Author: sheila.dwyer@ligo.org

Report ID: 81356

Sheila, writing for a large crew (Jason, Vicky, Daniel, Keita, Marc, Rick)

This morning we spent about 3 hours with the IMC offline and the ISS autolocker requested to off, (from 16:30 UTC) then unplugged the IMC feedback to the IMC VCO (from 17:08 UTC to 11:54 UTC) .

In the attached screenshot you can see that there are a few large disturbances during this time, which show up with an amplitude of 0.1 on the PMC mixer channel, and large oscillations in the FSS fast monitor (30 counts), dips in the reference cavity transmission, and glitches in the PMC high voltage. The second screenshot shows a zoomed in version of the problem times.

We also noticed that unplugging the feedback from the IMC to the VCO changed the reference cavity transmitted power, by about 1%, Daniel suggests that this might be OK because the change in laser frequency causes a small change in alignment out of the AOM.

We set the FSS autolocker to off at 19:54 UTC. This doesn't acutaly disable the FSS, but we can track times when the reference cavity would go through resonance by watching in the trans PD. At 21:19 UTC, Ryan Crouch started locking the IFO as the winds are supposed to be calm this afternoon (we did not see any glitches in this short test but need a longer one).

We will plan to continue this PMC only test overnight when the wind is supposed to come back up.

Images attached to this report

H1 ISC - ASC-AS_B_DC interface analog gain was put back to HIGH (i.e. +20dB)

keita.kawabe@ligo.org — Tue, 19 Nov 2024 11:54:50 -0800

Author: keita.kawabe@ligo.org

Report ID: 81355

Sometime ago (July/09/2024) the analog DC gain of H1:ASC-AS_B_DC was set from "High" to "Low" when I was looking to extract more information from WFSs about the power coming past the Fast Shutter when the shutter bounces down after high power lock losses. Since this is not necessary any more (see plots in alog 81130) and since keeping this means that we have to either adjust dark offset once in a while and/or making the "shutter closed" threshold for Fast Shutter test (alog 81320), I set the gain switch back to "HIGH", and disabled the +20dB digital gain in H1:ASC-AS_B_DC_SEG[1234] FM4.

Interestingly, when I flipped the gain switch, SEG3 output didn't change (1st attachment). It could be that the specific DC channel is stuck to HIGH or LOW, but it could be that the analog offset happens to be really small for that channel. I cannot test it right now as we're leaving IMC unlocked. As soon as the light comes back I will go to the floor and test.

The second attachment is the relevant MEDM screen (that shows the state it should be in now) and the third one is the picture of the interface chassis (the switch I flipped is circled in red, and it's supposed to be HIGH now).

ISC

Images attached to this report

H1 PSL - PSL PMC + FSS was continuously locked over emergency vent

victoriaa.xu@ligo.org — Tue, 19 Nov 2024 11:42:48 -0800

Author: victoriaa.xu@ligo.org

Report ID: 81354

It looks like usually the PSL PMC + FSS can stay locked for long periods of time, for example >1 month over the emergency vent in August. See the attached trends of the # of locked days and hours in the last few months (ndscope saved as /ligo/home/victoriaa.xu/ndscope/PSL/pmc_fss_can_stay_locked_over_emergency_o4b_vent.yaml).

The last time the PMC was locked for almost a month ended on Sept 23, 2024. Since then the PSL PMC has not stayed locked for over 5 days, but this is most likely due to commissioning tests and debugging which started around then.

Images attached to this report

H1 CDS - Comment to h1susex powered down for DAC change

marc.pirello@ligo.org — Tue, 19 Nov 2024 11:01:53 -0800

Author: marc.pirello@ligo.org

Report ID: 81353

WP12201

D. Barker, F. Clara, M. Pirello

Swap to original 20 bit DAC complete. Here are the steps we took to revert from LD32 to GS20

Power off IO-SUS-ETMX with Dave's help, suspension set to safe.
Move PI DB9 cable from new PI AI chassis (S1500301) to old PI AI chassis (S1500299). This is on the front of the chassis (channels 6 and 7)
Unplugged ESD-LD32 cable from the old PI AI chassis.(S1500299) on the back, DAC adapter 0.
Unplugged ESD-LD32 cable from IO-SUS-ETMX slot 8 (LD32 bank 0)
Move ETMX-86 from the new PI AI chassis (S1500301) to old PI AI chassis (S1500299) on the back DAC adapter 0
Removed WD-Bypass-Cable from slot 5 of the IO-SUS-ETMX and DAC adpater 1 of old PI AI chassis (S1500299) on the back.
Move ETMX-42 from IO-SUS-ETMX slot 10 (LD32 bank 2) to IO-SUS-ETMX slot 5 (GS 20 bit DAC).
Power on IO-SUS-ETMX and verify watchdogs reapplied.

Included images of front and rear of rack prior to changes.

Images attached to this report

H1 CDS - Temporarily bypassing fire-pump alarms while they are being tested

david.barker@ligo.org — Tue, 19 Nov 2024 10:46:41 -0800

Author: david.barker@ligo.org

Report ID: 81352

Bypass will expire:
Tue Nov 19 10:45:37 PM PST 2024
For channel(s):
H0:FMC-CS_FIRE_PUMP_1
H0:FMC-CS_FIRE_PUMP_2

LHO VE - Tue CP1 Fill

david.barker@ligo.org — Tue, 19 Nov 2024 10:20:46 -0800

Author: david.barker@ligo.org

Report ID: 81351

Tue Nov 19 10:12:46 2024 INFO: Fill completed in 12min 42secs

Images attached to this report

H1 CDS - h1susex powered down for DAC change

david.barker@ligo.org — Tue, 19 Nov 2024 09:33:51 -0800

Author: david.barker@ligo.org

Report ID: 81350

WP12201

Marc, Fil, Dave, Ryan

h1susex is powered down to restore cabling from the new 28bit LIGO-DAC to the original 20bit General Standards DACs.

Procedure:

Put h1iopseiex SWWD into long bypass

Safe the h1susetmx, h1sustmsx and h1susetmxpi models

Stop all models on h1susex

Fence h1susex from the Dolphin fabric

Power down h1susex.

H1 General - Lockloss at 15:22 UTC (9:56 lock)

ryan.crouch@ligo.org — Tue, 19 Nov 2024 07:40:40 -0800

Author: ryan.crouch@ligo.org

Report ID: 81348

15:22 UTC lockloss - 1416065051

FSS_FAST_MON grew just below the LL, IMC lost it at the same time as ASC. FSS, and ISS lost it at the same time.

Lockloss

Images attached to this report

H1 General - OPS Tuesday day shift start

ryan.crouch@ligo.org — Tue, 19 Nov 2024 07:31:30 -0800

Author: ryan.crouch@ligo.org

Report ID: 81347

TITLE: 11/19 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Preventative Maintenance
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 7mph Gusts, 5mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.54 μm/s
QUICK SUMMARY:

We lost lock at 15:22, 2 minutes into the magnetic injections (we had been locked for 9:56!)
- We're not going to get the inlock charge measurements this morning
15:30 UTC brought ISC-_LOCK to IDLE

H1 General - Monday Ops Day Shift End

anthony.sanchez@ligo.org — Mon, 18 Nov 2024 22:08:08 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81346

TITLE: 11/19 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 161Mpc
INCOMING OPERATOR: Oli
SHIFT SUMMARY:
Shift consisted of 3 locks and 2 IMC locklosses
Locking has been fairly straight forward once the IMC decides to get and stay locked.
Survived a 5.6M Tonga Quake, and a few PI ring ups.

Tagging SUS because ITMY Mode 5 is ringing up. I have turned OFF the Gain to it as the current Nominal state is making IYM5 worse and has been for the last few locks.

LOG:

Start Time	System	Name	Location	Lazer_Haz	Task	Time End
00:24	EPO	Corey +1	Overpass, Roof	N	Tour	00:24
01:06	PCAL	Rick S	PCAL Lab	Y	Looking for parts	02:16

SUS

Images attached to this report

H1 General - Monday Eve Lockloss #2

anthony.sanchez@ligo.org — Mon, 18 Nov 2024 20:50:37 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81345

4:20 UTC Lockloss https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi?event=1416025235

This Alog was brought to you by your favorite IMC Lockloss tag.

Lockloss

Images attached to this report

H1 General - Another IMC Lockloss from Nominal Low noise.

anthony.sanchez@ligo.org — Mon, 18 Nov 2024 19:24:50 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81344

TITLE: 11/19 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 16mph Gusts, 9mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.67 μm/s
QUICK SUMMARY:
1:57:53 UTC IMC Lockloss
https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/?event=1416016690

Fought with IMC a little to get the IMC to lock, by requesting Down, and Init, and offline a few times. But once it finally locked, relocking went fast.
2:55:01 UTC Nominal LowNoise

Incoming 5.6M EQ.

3:04 UTC Observing Reached

Lockloss

Images attached to this report

H1 General - Monday Ops Eve Shift Start

anthony.sanchez@ligo.org — Mon, 18 Nov 2024 16:39:20 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81342

TITLE: 11/19 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 9mph Gusts, 6mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.70 μm/s
QUICK SUMMARY:
Despite the Usiesm being elevated, H1 has been Locked for 20 minutes.
The Plan is to continue Observing all night.

LHO General - OPS Day Shift Summary

ibrahim.abouelfettouh@ligo.org — Mon, 18 Nov 2024 16:26:54 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81341

TITLE: 11/19 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing
INCOMING OPERATOR: Tony
SHIFT SUMMARY:

IFO is in NLN and OBSERVING as 00:17 UTC

A rough day for locking.

First, we had planned comissioning from 8:30AM to 11:30AM, during which comissioners continued investigating the IMC.

Sheila went into the LVEA to realign onto the POP and ALS beatnotes following a PR3 move that got higher buildups, which was successful. For this, we stayed at CHECK_VIOLINS_BEFROE_POWERUP (losing lock twice from there due to IMC losing lock).

We then got to locking and were able to acquire NLN and go into OBSERVING for 30 minutes (starting 21:44 UTC). The IMC glitch then caused a Lockloss (alog 81338).

After this LL, we were unable to lock DRMI despite just having done an initial alignment. Tried touching up PRM and SRM during PRMI and DRMI respectively, succeeding in the former and failing in the latter.

I decided to do an initial alignment, which happened fully automatically. We were able to lock DRMI but a lockloss at CARM_5_PICOMETERS brought us down due to the IMC glitch. We were then able to come back up to NLN fully automatically, finally getting to OBSERVING.

Overall, high microseism and IMC glitches made locking difficult. We still got there in the end though.
LOG:

Start Time	System	Name	Location	Lazer_Haz	Task	Time End
21:41	SAF	Laser	LVEA	YES	LVEA is laser HAZARD	08:21
16:36	FAC	Karen	Optics Lab	N	Technical Cleaning	16:36
16:53	FAC	Kim	MX, EX	N	Technical Cleaning	17:57
16:58	FAC	Karen	MY	N	Technical cleaning	18:57
17:38	VAC	Janos	EY	N	Compressor measurement	17:59
18:28	ISC	Sheila, Oli	LVEA	Y	Realigning ALS Beatnote	19:28
22:33	CDS	Patrick	MSR	N	Timing removed from the Beckhoff system	22:41
00:24	EPO	Corey +1	Overpass, Roof	N	Tour	00:24

H1 PSL - Comment to Overview of PSL Story so far

camilla.compton@ligo.org — Mon, 18 Nov 2024 15:45:40 -0800

Author: camilla.compton@ligo.org

Report ID: 81340

November 12th:
- Took TFs 81247
- Tightened loose ISS AOM RF cable 81247
- Tested and characterized in-service TTFSS, SN LHO01: tuned notches and replaced a bad PA85 op amp 81247
November 13th:
- IMC stayed locked without IFO 81262
- FSS OLG and gain change 81254
- IMC OLG and gain change 81259.
Novmeber 14th:
- PSL Beckhoff power cycle 81281
- Replace 35MHz with Marconi: 81277
- Tightened loose ISS AOM cable: 81280
Novmeber 18th:
- LSC POP un-clipped 81329
- Issues found with IMC locked checker in ISC_LOCK's READY 81336 (maybe keeping us in DOWN longer than needed)

Updated list of things that have been checked above and attache a plot where I've split the IMC only tagged locklosses (orange) from those tagged IMC and FSS_OSCIALTION (yellow). The non IMC ones (blue) are the normal lock losses and (mostly) only once we saw before September.

ISC

Images attached to this report

H1 CDS - Removed timing fiber from Beckhoff spare computer in MSR

patrick.thomas@ligo.org — Mon, 18 Nov 2024 15:06:46 -0800

Author: patrick.thomas@ligo.org

Report ID: 81339

Dave, Fernando, Patrick Fernando and I powered off the computer, unplugged the power and other cables from the back of it to slide it forward in the rack, took the top off the chassis, disconnected the blue timing fiber from the PCI board, capped the fiber ends, pulled the fiber out of the chassis, slid the computer back into the rack, and plugged the cables back in, except for the kvm switch cable that is not used anyway. We left the computer on. This was to stop the timing overview from turning red when the computer is powered on. We did not put in a work permit, but cleared it with Daniel and informed the operator.

H1 ISC - Lockloss 22:15 UTC

ibrahim.abouelfettouh@ligo.org — Mon, 18 Nov 2024 14:18:11 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81338

IMC caused Lockloss 30 mins into observing.

Lockloss

H1 ISC - Comment to IMC TRANS power changes up to 1% once we get to NLN and are thermalizing

camilla.compton@ligo.org — Mon, 18 Nov 2024 13:35:40 -0800

Author: camilla.compton@ligo.org

Report ID: 81337

Elenna and Craig noted this in 68370 too.

H1 ISC - Comment to Edit to ISC_LOCK to make sure the IMC is locked before leaving READY

thomas.shaffer@ligo.org — Mon, 18 Nov 2024 11:20:18 -0800

Author: thomas.shaffer@ligo.org

Report ID: 81336

The reason it has been doing this is because there is a return True in the in main method of ISC_LOCK's READY state. Then a state returns True in its main method, it will skip the run method.

I've loaded the removal of this in ISC_LOCK.

OpsInfo

H1 CDS - EY CNS-II GPS receiver glitched again this morning 05:47 PDT for 12 mins

david.barker@ligo.org — Mon, 18 Nov 2024 11:13:59 -0800

Author: david.barker@ligo.org

Report ID: 81335

We had another instance of the -800nS timing glitch of the EY CNS-II GPS receiver this morning from 05:47:50 to 06:00:04 PDT (duration 12min 11 sec).

A reminder of this receiver's history:

At EX and EY we had no CNS-II issues for the whole of O4 until the past few months.

The EY 1PPS, nominally with a -250 +/- 50 nS difference to the timing system, went into a -800nS diff state on 30th Sept, then again 1st October and again 18th October.

We replaced its power supply Tue 22nd October 2024, after which there have been no glitches for 27 days until this morning.

Images attached to this report

H1 TCS - Comment to TCSy Laser Unlocks & Relocks, Dropping H1 From Observing Briefly

camilla.compton@ligo.org — Mon, 18 Nov 2024 10:52:06 -0800

Author: camilla.compton@ligo.org

Report ID: 81334

CO2Y has only unlocked/relocked once since we power cycled the chassis on Thursday 14th (t-cursor in attached plot).

Images attached to this report

H1 SEI - H1 ISI CPS Noise Spectra Check - Weekly FAMIS

ryan.crouch@ligo.org — Mon, 18 Nov 2024 10:44:23 -0800

Author: ryan.crouch@ligo.org

Report ID: 81331

Closes FAMIS26017, last checked in alog81183.

HAMs:

HAM sensor noise at 7-9Hz seems to be reduced for most chambers (HAMs and BSCs both stages of BSCs).

BSCs:

ITMY_ST1_CPSINF_V2 looks reduced above 30 Hz.

All the BSC ST2 specifically, sensor noise at 6-9Hz is reduced.

SEI

Non-image files attached to this report

CPS_spectra_10:29-11_18_2024.pdf

LHO VE - Mon CP1 Fill

david.barker@ligo.org — Mon, 18 Nov 2024 10:30:56 -0800

Author: david.barker@ligo.org

Report ID: 81332

Mon Nov 18 10:13:07 2024 INFO: Fill completed in 13min 4secs

Images attached to this report

H1 ISC - Comment to LSC POP found to be clipping, PR3 being moved to improve this.

sheila.dwyer@ligo.org — Mon, 18 Nov 2024 10:20:08 -0800

Author: sheila.dwyer@ligo.org

Report ID: 81330

We have been holding 2W with ASC on before powering up, and using the guardian state PR2_SPOT move, which lets us move the sliders on PR3 and moves PR2, IM4 and PRM sliders to follow PR3.

Moving PR3 by -1.7 urad (slider counts) increased the power on LSC POP, and POPAIR 18, but slightly misaligned the ALS comm beatnote. We continued moving PR3 to see how wide the plateau is on LSC POP, we moved it another -3urad without seeing power drop on LSC POP, but the ALS paths started to have trouble staying locked so we stopped there. (POP18 was still improving, but I went to ISCT1 after the OFI vent and adjusted alignment onto that diode 79883 so it isn't a nice reference). I moved PR3 yaw back to 96 urad on the yaw slider (we started at 100urad), a location where we were near the top for POP and POPAIR 18, so in total we started with PR3 yaw at 100 on the slider and ended with 96 on the slider.

Images attached to this report

H1 ISC - LSC POP found to be clipping, PR3 being moved to improve this.

camilla.compton@ligo.org — Mon, 18 Nov 2024 10:03:59 -0800

Author: camilla.compton@ligo.org

Report ID: 81329

Sheila, Ibrahim, Jenne, Vicky, Camilla

After Ibrahim told Sheila that DRMI has been struggling to lock, she checked the POP signals and found that somethings been drifting and POP now appears to be clipping, see POP_A_LF trending down. We use this PD in full lock so don''t want any clipping.

We has similar issues last year that gave us stability issues. These issues last year didn't have "IMC" lock-losses so we think this isn't the main issue we've having now but maybe effecting stability.

Trends showing the POP clipping last year, and now. Last year we moved PR3 to removing this slipping while looking at the coherence between ASC-POP_A_NSUM (one of the QPD on the sled in HAM3) and LSC-POP_A (LSC sensor in HAM1): 74578, 74580, 74581.

Similar coherence plots to 74580, for now are show the coherence is bad:

It's got worse since July, ca see as step down once we moved PR2 in 78878. Compare Green (July 1st) then Red (July 10th), then Blue (now), plot here.
It's been bad since the emergency vent recovery in August plot here.
The increased coherence seen now <1Hz is the micoseism being elevated.

Sheila is moving the PRC cavity now which is improving POPAIR signals, plot attached is the ASC-POP_A_NSUM to LSC-POP_A coherence with DRMI only locked before and during the move. See improvements. Sheila is checking she's in the best position now.

Images attached to this report

H1 ISC - Microsiem level seemed correlated with IMC isuues this weekend but not further back.

camilla.compton@ligo.org — Mon, 18 Nov 2024 09:15:17 -0800

Author: camilla.compton@ligo.org

Report ID: 81328

Ibrahim, Camilla

Over the weekend we had a long ~13 hour lock and 2 days with minimal IMC issues, this was during a time when the microseism was low. Plot attached. We see similar long locks around Novmeber 6th/7th when the microseism was low, but we always have less issues locking when the micorsiem is low, plot attached.

We checked back to the quiet IMC time shown in 80951, between Oct 11th to 16th, but the microseism was actually relatively elevated during this time. Plot attached.

In both plots, between the t-cursors show quieter IMC glitch times.

Images attached to this report

H1 PSL - PSL 10-Day Trends

ryan.short@ligo.org — Mon, 18 Nov 2024 09:09:04 -0800

Author: ryan.short@ligo.org

Report ID: 81327

FAMIS 31060

Enclosure incursion last Tuesday (alog81247); enclosure temps came back as expected, but power out of AMP2 has been lower by about 1W since then.

PSL Beckhoff was restarted last Thursday (alog81281), which apparently brought the chiller back with a slightly lower flow rate. At this time, there was also a slight jump up in temperature for AMP2, DB2, CB2, and the diode room (?).

PMC reflected power has also been on the rise again, but generally steady for the past ~4 days.

Images attached to this report

LHO General - OPS Day Shift Start

ibrahim.abouelfettouh@ligo.org — Mon, 18 Nov 2024 07:37:02 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81326

TITLE: 11/18 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 15mph Gusts, 11mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.58 μm/s
QUICK SUMMARY:

IFO is LOCKING at ACQUIRE DRMI

When I arrived, IFO had just found PRMI. It seems this is the first lock acquisition since an auto-initial alignment following the last lockloss at 14:23 UTC. Microseism is high and increasing.

H1 General - Ops Owl Update

thomas.shaffer@ligo.org — Mon, 18 Nov 2024 04:07:11 -0800

Author: thomas.shaffer@ligo.org

Report ID: 81325

The IFO timed out after a long period of IMC locking struggles, then it got stuck trying to lock ALSY during initial alignment. I minor tweak to ETMY fixed the latter issue, but then SRM tripped during the SRC alignment step. I adjusted SRM to fix this. We then made it up to just after the power up before losing lock again (141596598, IMC tag).

I'll let it try a few more times by itself before stepping in again.

H1 General - Sunday Ops Day Shift End

anthony.sanchez@ligo.org — Sun, 17 Nov 2024 22:08:34 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81324

TITLE: 11/18 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 20mph Gusts, 13mph 3min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.42 μm/s
QUICK SUMMARY:

I ran an Initial_Alignment early on and had the IMC unlock during the SRC_ ALIGN Section. The asc signals looked real strange, so I turned off the ISS and did a Manual_Initial_Alignment.

but....

Between the increasing Sencondary microSeism , the 30-40+ mph wind gusts, and the IMC issues have made locking for the past 2 shifts very diffucult.
I tried using the SEI_CONF MORE_WINDY early on to see if I could get locked. But as the rising micro seism made this increasingly less like to be successful.
I held in IDLE for an hour waiting for the wind to get below the 30 mph mark and tried to lock a number of times. Getting no Futher than LOWNOISE_ESD_ETMX

I took some time to do another Initial Alignment, and didn't get H1all the way locked.

So , Good luck to TJ and H1 Manni.

H1 General - Sunday Ops Eve Shift Start

anthony.sanchez@ligo.org — Sun, 17 Nov 2024 16:55:29 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81323

TITLE: 11/18 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Wind
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 29mph Gusts, 18mph 3min avg
    Primary useism: 0.06 μm/s
    Secondary useism: 0.44 μm/s
QUICK SUMMARY:
45 MPH wind gusts are present.
I'll be trying to lock it a few times just incase I get lucky with a break in the wind.
IMC Has been unlocking as well. I turned off the ISS again to get some more stability from the IMC. I'm not sure if this actually works or not.
I'm also taking the SEI_CONF to MORE_WINDY, to see if that also helps.

Wish me luck!

LHO General - OPS Day Shift Summary

ibrahim.abouelfettouh@ligo.org — Sun, 17 Nov 2024 16:29:15 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81322

TITLE: 11/18 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Wind
INCOMING OPERATOR: Tony
SHIFT SUMMARY:

IFO is DOWN due to ENVIRONMENT

Very rough shift with 0 time spent locked. Today, this was mostly due to 3 reasons.

1. Environment:

Wind gusts are currently 40 mph, mostly above 30 mph for the whole day. There were a few gusts above 50 mph with a couple above 60mph (with one at 62 mph, the highest I've seen).
Secondary microseism is back over the 90% threshold for microseism. This is the main reason we were unable to lock during the shift.

2. IMC: The IMC glitch has dominated our locking attempts during lulls in the wind speed. It occured during initial alignment as well as during normal locking multiple times. This occured a few times after we had already locked DRMI under such windy conditions and so getting back to DRMI was either glitchy or windy and only happened twice today. The glitch has happened during all non-windy lock acquisitions.

3. Fast Shutter Error: At 20:01, we were at CHECK_AS_SHUTTERS during a lull in wind speed when we lost lock. After this locklossm the HAM6 SHUTTER status alert went red and guardian warned me not to power up. After calling a couple of people, Keita was able to diagnose the issue as a code threshold error. I went into the fast shutter guardian and changed the threshold of PdBDarkLo from -15 to -40 and PdBDarkHi from 15 to 40. More background and info is available in Keita's comment to my midshift alog 81320. This took 1hr 43 mins to diagnose and solve.

A calibration sweep has not been done yet (since Saturday). If at all we are able to get locked let alone thermalized that, Tony will do it.

LOG:

None

H1 PSL - PSL Weekly Report - Weekly FAMIS 26318

ibrahim.abouelfettouh@ligo.org — Sun, 17 Nov 2024 16:08:58 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81321

Closes FAMIS 26318, Last checked in alog 81105

Laser Status:
    NPRO output power is 1.909W (nominal ~2W)
    AMP1 output power is 68.55W (nominal ~70W)
    AMP2 output power is 137.8W (nominal 135-140W)
    NPRO watchdog is GREEN
    AMP1 watchdog is GREEN
    AMP2 watchdog is GREEN
    PDWD watchdog is GREEN

PMC:
    It has been locked 0 days, 21 hr 38 minutes
    Reflected power = 18.21W
    Transmitted power = 107.2W
    PowerSum = 125.4W

FSS:
It has been locked for 0 days 0 hr and 18 min
TPD[V] = 0.8546V

ISS:
The diffracted power is around 4.0%
Last saturation event was 0 days 0 hours and 0 minutes ago

Possible Issues:
PMC reflected power is high - this is new from last week's report.
ISS diffracted power is high

PSL

H1 ISC - Comment to OPS Day Midshift Update

keita.kawabe@ligo.org — Sun, 17 Nov 2024 13:59:20 -0800

Author: keita.kawabe@ligo.org

Report ID: 81320

Fast shutter worked and is working. (I was worried that the mirror was broken off but that's not the case, the shutter still opens and closes without an issue.)

From the guardian log (attached), it failed because AS_B_DC_NSUM was 20.3 when the shutter was closed and the threshold is 15. AS_B was 0.28. Fundamentally, this is because I reduced the gain of this DC path by 20dB and digitally compensated in search for an existing sensor that won't rail when lock is lost, and the same fast shutter test failure was observed (see alog 79104). Back then, I wrote "I adjusted the dark offset while IMC was unlocked but we can probably increase the threshold to 30 or so if it continues to bother us."

The dark offset has drifted enough to bother us at times (i.e. it is about 12 counts or so on average now), so I told Ibrahim to increase the thresholds only for AS_B, i.e.

PdBDarkLo=-40 # dark level low
PdBDarkHi= 40 # dark level high

in FAST_SHUTTER.py.

Since we now have a dedicated diode to look at lock loss power spikes, we can turn down the analog gain of AS_B_DC by 20dB and turn off +20dB filter in FM4 of H1:ASC-AS_B_DC_SEG[1234] filter, and adjust the dark offset again on Tuesday.

The mirror is still attached to the fast shutter and is moving. It actually closed using HV (see GS13 signal) when the test failed (2nd attachment), and later when I opened/closed it with low voltage drive it responded correctly (3rd attachment).

Images attached to this report

H1 ISC - OPS Day Midshift Update

ibrahim.abouelfettouh@ligo.org — Sun, 17 Nov 2024 12:18:54 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81319

We had a lull in the heavy winds during which I was able to get an initial alignment in and then up to DRMI locked.

The wind is now back up to the 30s mph and the IMC glitch is happening quite frequently (both on the rocky start-up to DRMI and thereafter).

Thus, I will be staying in DOWN until conditions improve.

Meanwhile, the HAM6 SHUTTER red alert came on with guardian telling me not to power up due to SHUTTER_FAILURE. I am investigating this.

LHO VE - Sun CP1 Fill

david.barker@ligo.org — Sun, 17 Nov 2024 10:12:24 -0800

Author: david.barker@ligo.org

Report ID: 81318

Sun Nov 17 10:10:23 2024 INFO: Fill completed in 10min 20secs

Images attached to this report

LHO General - OPS Day Shift Start

ibrahim.abouelfettouh@ligo.org — Sun, 17 Nov 2024 07:53:17 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81317

TITLE: 11/17 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Wind
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 44mph Gusts, 33mph 3min avg
    Primary useism: 0.12 μm/s
    Secondary useism: 0.32 μm/s
QUICK SUMMARY:

IFO is IDLE due to ENVIRONMENT.

In the last hour since TJ's OWL alog, the wind has not died down, and the microseism has only gone up.

With 50mph gusts, we are unable to lock. Will stay in idle until we can re-attempt lock.

On the bright side, the IMC is locked.

H1 General - Ops Owl Update

thomas.shaffer@ligo.org — Sun, 17 Nov 2024 06:59:00 -0800

Author: thomas.shaffer@ligo.org

Report ID: 81316

The IMC has been struggling to stay locked for the last few hours, interrupting locking and restarting the process. Ground motion has also not been kind with a 6.1M earthquake from Japan a few hours ago, and useism and wind on the rise. Since the wind is getting over 40mph gusts, I'm switching the Observatory Mode to Environment - Windy.

H1 General - Saturday Ops Eve Shift End

anthony.sanchez@ligo.org — Sat, 16 Nov 2024 22:15:30 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81315

TITLE: 11/17 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: TJ
SHIFT SUMMARY:

3:01 UTC GRB-Short E 525935 We were not locked for the begining of this stand down event.

2024-11-17 3:27:53 NOMINAL_LOW_NOISE reached
3:30:27 OBSERVING reached

Violin ITMY Mode 5 is still rising on this lock.
I changed the Gain to 0 and waited 15 minutes to see if it will stop going up or even turn around.
I then tried about 3 minutes with the gain at -0.02 ( Nominal gain just negative) but it was obvious that the no gain is better than opposite gain.
I ended up just turnin goff the gain.

05:11 UTC Lockloss https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi?event=1415855535
This was a very fast lockloss , it doesn't look like the same IMC /FSS locklosses we have been seening. But the PSL-FSS_FAST_MON_OUT_DQ was still the first signal to change behavior.

Relocking notes:

Relocking has been made very difficult with the IMC constantly unlocking. Most of the Locklosses are in locking Green arms, but everyfew attempts it gets a little further like DRMI, then the IMC stops working again.
I tried turning the ISS off again, but it hasn't stopped it from unlocking the IMC.
I turned back on the ISS so just in case H1 manager gets to about ISC_LOCK state 500 it will auto lock for him.

Lockloss, SUS

Images attached to this report

H1 General - Relocking notes & Mid shift .

anthony.sanchez@ligo.org — Sat, 16 Nov 2024 18:46:48 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81314

I couldn't get Past DRMI 1f, and ended up in a PRMI & Check Mich loop.

I tried to do an Intial_Alignment, But IMC would not stay locked for long enough.
I eventually waited for the IMC to lock then turned off the ISS Auto Locker for the duration of the Initial alignment.

Initial alignment went well with the ISS turnt off.
Trying to relock again.

Images attached to this report

H1 General - Comment to Saturday Ops Eve Shift Start

anthony.sanchez@ligo.org — Sat, 16 Nov 2024 17:39:17 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81313

Lockloss https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi?event=1415840300

This looks like there is some IMC issue that cause this Lockloss.
The Lockloss plots show that at -906ms there was blip on the PSL-FSS_FAST_MON _OUT_DQ channel.

Relocking the IMC was difficult as the IMC kept going into fault. I took ISC_LOCK to idel and took the IMC to idle for 10 minutes.
Relocking started fine, except I had to manual green arms.

Lockloss, PSL

Images attached to this report

H1 General - Saturday Ops Eve Shift Start

anthony.sanchez@ligo.org — Sat, 16 Nov 2024 16:37:01 -0800

Author: anthony.sanchez@ligo.org

Report ID: 81312

TITLE: 11/17 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 159Mpc
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 15mph Gusts, 11mph 3min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.24 μm/s
QUICK SUMMARY:

H1 Has been locked for 1 Hour and 45 minutes.
DCPD's are diverging, Which I usually attribute to the Violins growing.
Turns out it's one of the usual suspects..... ITMY Mode 5 increasing.

I have turned the Gain off of ITMY m5 to see if it stays at that level or decreases.

The IFO stays locked until 1:25 UTC then I can do a Calibration sweep are the calibration time early today was missed by a lockloss.

SUS

Images attached to this report

LHO General - OPS Day Shift Summary

ibrahim.abouelfettouh@ligo.org — Sat, 16 Nov 2024 16:30:16 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81311

TITLE: 11/16 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 157Mpc
INCOMING OPERATOR: Tony
SHIFT SUMMARY:

IFO is in NLN and OBSERVING as of 22:35 UTC

There was one IMC-caused Lockloss which ended our 13hr and 36 min lock acquisition. (alog 81310)

Stayed locked for the majority of this shift with one somewhat troublesome lock acquisiton for a number of reasons:

ALSX was stuck at 0.15 cts, with a max of 0.3 cts after intervension. I went into manual initial alignment to continue troubleshooting. Jenne advised to touch ITMX, which worked. I was able to get ALSX to the normal ~0.8cts and let guardian successfully take it from there. Initial alignment happened fully auto.
We were able to lock up until OFFLOAD_DRMI_ASC until a lockloss occured due to the IMC glitch. The IMC was still coming in and out, locking and unlocking for 4 more minutes.
We were able to get all the way up to LOWNOISE_ESD_ETMX, before losing lock again.
IFO was able to lock to NLN on the third and final try.

I was not able to take a calibration measurement due to the Lockloss, and we're not thermalized yet. I've informed Tony, the EVE operator, and he will take a calibration sweep once thermalized (01:24 UTC).

LOG:

None

H1 ISC - Lockloss 19:07 UTC

ibrahim.abouelfettouh@ligo.org — Sat, 16 Nov 2024 11:17:22 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81310

Lockloss seemingly caused due to the IMC with MC2 losing lock within 1ms of ASC (attached).

Lockloss

Images attached to this report

LHO VE - Sat CP1 Fill

david.barker@ligo.org — Sat, 16 Nov 2024 10:18:03 -0800

Author: david.barker@ligo.org

Report ID: 81309

Sat Nov 16 10:10:51 2024 INFO: Fill completed in 10min 47secs

Images attached to this report

LHO General - OPS Day Shift Start

ibrahim.abouelfettouh@ligo.org — Sat, 16 Nov 2024 07:33:41 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81308

TITLE: 11/16 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 164Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 12mph Gusts, 9mph 3min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.20 μm/s
QUICK SUMMARY:

IFO is in NLN and OBSERVING as of 05:34 UTC (10 hr lock!)

LHO General - Fri Ops Eve Summary

corey.gray@ligo.org — Fri, 15 Nov 2024 22:00:42 -0800

Author: corey.gray@ligo.org

Report ID: 81305

TITLE: 11/16 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Aligning
INCOMING OPERATOR: TJ
SHIFT SUMMARY:

Most of the meat of the shift has already been posted earlier.

Had a return of the FSS/IMC glitches this afternoon/evening. Have made it back to Observing at the end of the shift though.

For the night we are set for locking for the OWL. But if locking becomes an issue that lasts over 2hrs, plan is to take ISC LOCK to IDLE for the night.
LOG:

0007-0028 initial alignment
0111 OBSERVING
- 0122-0131 PI 31 begins increasing pretty steadily over atleast 5min and started causing DCPD glitches, but then guardian finally was able to bring it down
- 0208 LOCKLOSS (looks like an FSS/IMC glitch, @0211 IMC also caused on the order of min of downtime before relocking)
- Several FSS and/or IMC glitches for a few locking attempts

0534 OBSERVING

H1 PSL - 02:08utc NLN Lockloss due to FSS/IMC Glitch After 1hr Lock

corey.gray@ligo.org — Fri, 15 Nov 2024 20:47:15 -0800

Author: corey.gray@ligo.org

Report ID: 81306

Had about 20hrs with no IMC/FSS glitches from 9pm Thurs night until about 340pmPT Fri afternoon. But have now had several for the last 5hrs.

Also for one ~60min stretch, H1 almost made it to NLN, but had rung up violins which kept us at OMC Whitening for 20min until the next IMC glitch.

Here are some running notes for the rough 5hrs filled with glitches (currently have H1 in IDLE).

Notes During Return of IMC/FSS Glitch Locklosses

2340utc (340pmPT) LOCKLOSS with IMC tag was the beginning of IMC/FSS glitch locklosses (where we went almost 20hrs since the last glitch around 5utc (9pmPT).

0111-0208: OBSERVING

0208utc (608pmPT): Attached screenshot shows a coincident glitch lockloss seen on FSS & IMC. Here is the LOCKLOSS page also tagged with IMC glitch.

Starting an FRS Ticket to mark the DOWNTIME (FRS-32644) for 3.5hrs of DOWNTIME so far. Where there was notable part of downtime due to IMC not locking easily.

0241 Lockloss at DARM to DC ReadOut looked like a FSS/IMC glitch lockloss---> IMC + FSS lockloss tags.
0255 Lockloss at RESONANCE also looked like a FSS/IMC glitch lockloss---> IMC lockloss tag
0256 Lockloss at READY ---> FSS lockloss tag
0334 lock made it almost to NLN, Yay Yay Yay!!!! BUT it was a little delayed due to....

VIOLINS Rung Up A Bit: (haven't had this since my shift on Tues night!) So were stuck at OMC WHITENING for over 20min before H1 had....

0357 Lockloss at OMC WHITENING (with an IMC LOCKLOSS tag)

So for the last 3hrs:

0208-0256: 50min with a flurry of FSS/IMC glitches causing locklosses at various states also causing IMC locking grief.
0256-0357: Hour of NO glitch locklosses, and almost made it to NLN (but rung up violins prevented that). Then another IMC glitch.
0420 Went to IDLE due to more locklosses and for a pause

Not sure what is more useful. Keep having ISC LOCK continue to locking, or to leave it in IDLE for IMC/PMC/FSS monitoring....

IOO, PSL

Images attached to this report

H1 PEM - HVAC Fan Vibrometers FAMIS Check (FAMIS 26340)

corey.gray@ligo.org — Fri, 15 Nov 2024 19:12:24 -0800

Author: corey.gray@ligo.org

Report ID: 81307

For FAMIS #26340: All looks well for the last week for all site HVAC fans (see attached trends).

PEM

Images attached to this report

H1 AOS - OPS Day Shift Summary

ibrahim.abouelfettouh@ligo.org — Fri, 15 Nov 2024 16:43:43 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81304

TITLE: 11/16 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Aligning
INCOMING OPERATOR: Corey
SHIFT SUMMARY:

IFO is LOCKING in ENGAGE_ASC_FOR_FULL_IFO

Just had a lockloss and trying to get back to observing. IFO has been quite well behaved today. Both lock acquisitions were fully automatic with H1 needing no intervention.

That being said, we did have two locklosses.

Lockloss 1: ETM glitch caused.

Lockloss 2: IMC caused.

LOG:

None

LHO General - Fri Ops Eve Transition

corey.gray@ligo.org — Fri, 15 Nov 2024 16:24:12 -0800

Author: corey.gray@ligo.org

Report ID: 81303

TITLE: 11/16 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Aligning
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 14mph Gusts, 11mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.28 μm/s
QUICK SUMMARY:

Walked in to a locking h1, which then has moved on to INITIAL ALIGNMENT. Ibrahim shared how H1 has had a better time--so am cautiously optimistic for the night---with the goal of OBSERVING!

Environmentally we are becoming quieter with µseism dropping below the 95th percentile and winds not too breezy.

H1 ISC - Comment to Lockloss 23:40 UTC

ryan.short@ligo.org — Fri, 15 Nov 2024 16:23:45 -0800

Author: ryan.short@ligo.org

Report ID: 81302

Adding more channels for the sake of investigation. As Ibrahim mentions, no glitchy behavior was seen leading up to this IMC lockloss, so this was either a large fast glitch out of nowhere or something else giving out. Seemingly all at once (or at least very close to each other), the FSS_FASTMON, RefCav trans, NPRO power, PMC high voltage, PMC mixer, and all IMC signals quickly change, so it's hard to tell which is truly happening first as there is lots of feedback between these signals.

Images attached to this report

H1 ISC - Lockloss 23:40 UTC

ibrahim.abouelfettouh@ligo.org — Fri, 15 Nov 2024 16:04:35 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81301

Ibrahim, Ryan S

Seems like an IMC lockloss. Interestingly, there was no obviously glitchy behavior (as previously observed) before or after the lockloss. Attached screenshot of ASC and IMC losing lock.

Lockloss

Images attached to this report

H1 SQZ - More low pass filtering for ADF SQZ readbacks

victoriaa.xu@ligo.org — Fri, 15 Nov 2024 15:53:25 -0800

Author: victoriaa.xu@ligo.org

Report ID: 81300

Sheila, Vicky.

We changed the low pass filtering on the ADF SQZ angle servo to be lower, going from 1 Hz low pass to 0.1 Hz low pass on the ADF I/Q demod filter banks. I think this reduced the sqz angle readback noise a bit (trends before/after). It didn't make a huge impact on the control signal that goes to the CLF_RF6 demod phase (bottom purple trace), and it might be worth considering low pass filtering that control signal too. I accepted the SDFs for this filter bank LPF change.

SQZ

Images attached to this report

H1 GRD - automation got stuck in a PRMI loop overnight

sheila.dwyer@ligo.org — Fri, 15 Nov 2024 12:10:28 -0800

Author: sheila.dwyer@ligo.org

Report ID: 81298

Ibrahim, Sheila, Ryan S

Ibrahim noticed that the guardian was in a loop of locking PRMI something like 20 times overnight.

What happened was that PRMI was poorly aligned (POP18 and 90 I ERR both below 20 counts), but PRMI was able to grab lock for a couple of seconds, but would not hold lock as the guardian engaged the top mass offloading, adjsuted gains and filters. Eventually this hit the timer in H1 manager to do an initial alignmemt, after just over an hour.

There is also a timer in ISC that checks if PRMI hasn't locked in 10 minutes it should move onto MICH fringes. This didn't happen in this case because of the very short locks. Ryan and I read through the ISC_LOCK AQUIRE_PRMI state, and realized that it had a check (line 1371) for DRMI arrived (in the state PRMI locked), which didn't include a check for the state being done, which would return true in ISC LOCK before ISC_DRMI finished the final steps in PRMI locked . In the second, zoomed screenshot, you can see that as soon as the DRMI guardian enters state 35 (PRMI locked), ISC lock moves on immediately from state 50 (ACQUIRE PRMI) to 51 and 52 (PRMI ASC), which resets the timer so that we never went to MICH fringes.

We added a check for the ISC_DRMI state to be both arrived and done in PRMI locked, so that PRMI will have to survive the offloading and boost engagement for the timer to be reset. Ibrahim has now reloaded this. We think that if this situation came up again, now we would only spend 10 minutes relocking PRMI, before going to MICH fringes. If H1 is under the manager control, it would run initial alignment if running MICH fringes didn't help.

Images attached to this report

H1 General - Comment to Range_compare for a range drop midlock

ryan.crouch@ligo.org — Fri, 15 Nov 2024 12:08:04 -0800

Author: ryan.crouch@ligo.org

Report ID: 81299

The SQZ blrms didn't really change during this time either.

Images attached to this report

H1 General - Range_compare for a range drop midlock

ryan.crouch@ligo.org — Fri, 15 Nov 2024 11:27:04 -0800

Author: ryan.crouch@ligo.org

Report ID: 81296

I ran the range_compare script using the 10-12 minute low range stretch from the most recent lock and some time before it of the same lock where the range was better, the construction crew by staging was also seen moving large equipment around this time on the cameras.

To produce this figure I ran "python3 /ligo/gitcommon/ops_tools/rangeComparison/range_compare.py --span 600 1415725641 1415728983", I only did 600 seconds of data. The range drop looks to be from low frequency noise, mostly below ~300Hz, the most obvious peak difference looks to be at 46/47 (mostly on sensmon), 36 (mostly on sensmon), and 60Hz (both sensmon and DARM) for the low range stretch. The lines are hard to see, but if you open the pdf in LibreOffice (linux default pdf viewer) and right-click -> Arrange -> Behind Object, then you can hover your cursor over the curves and it will be much easier to see the differences.

Images attached to this report

Non-image files attached to this report

compare_darm_1415725641_1415728983.pdf

H1 ISC - Comment to Lockloss 18:36 UTC

ryan.short@ligo.org — Fri, 15 Nov 2024 11:21:52 -0800

Author: ryan.short@ligo.org

Report ID: 81297

Does not look like an IMC/PSL lockloss; trends attached. The NPRO temp has been glitching this morning, not drastically, which has been causing the EOM drive to be higher in general, but these glitches were not happening at the time of this lockloss. The shape of of the signal on AS_A is indicative of an arms lockloss and happens before the IMC loses lock according to MC2_TRANS. Additionally, the lockloss tool shows glitches on ETMX starting up to second before the lockloss, although the ETM_GLITCH tag was not applied in this case as the glitches did not meet the required threshold.

(The ndscope template I'm using lives in my home directory ~/templates/psl_glitch_hunting.yaml)

Images attached to this report

H1 ISC - Lockloss 18:36 UTC

ibrahim.abouelfettouh@ligo.org — Fri, 15 Nov 2024 10:57:23 -0800

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 81295

Ryan S, Ibrahim

Unexpected lockloss being investigated but confirmed not the IMC nor the environment. Ryan S is seeing that it is looking like the ETM Glitch.

Lockloss

H1 PSL - PSL 10-Day Trends

ryan.short@ligo.org — Fri, 15 Nov 2024 10:37:01 -0800

Author: ryan.short@ligo.org

Report ID: 81293

FAMIS 31059

Late entry; these are trends taken on Monday as usual but I neglected to post them until now.

Since troubleshooting of laser glitches is still ongoing, several things in the PSL have been changing more than usual, including enclosure incursions, temperature changes, and ISS diffracted power increases. The only unexpected thing of note is that PMC reflected power seems to have been rising slowly over the past few days, but I think it's too soon to tell if this is a similar increase to what we were seeing here pre-NPRO swap or if it's related to laser troubleshooting. Will certainly be keeping an eye on this.

Images attached to this report

H1 CDS - Comment to Restarted Beckhoff PSL computer in diode room

david.barker@ligo.org — Fri, 15 Nov 2024 10:36:54 -0800

Author: david.barker@ligo.org

Report ID: 81294

As Patrick mentioned, channel access clients which had been connected to the IOC on h1pslctrl0 would not reconnect after its restart.

The EDC stayed in its disconnect state for almost an hour, even though cagets on h1susauxb123 itself were connecting, albeit with "duplicate list entry" warnings:

(diskless)controls@h1susauxb123:~$ caget H1:SYS-ETHERCAT_PSL_INFO_TPY_TIME_HOUR
Warning: Duplicate EPICS CA Address list entry "10.101.0.255:5064" discarded
H1:SYS-ETHERCAT_PSL_INFO_TPY_TIME_HOUR 18

The restart of the DAQ EDC did not go smoothly, I had added a missing channel to H1EPICS_CDSRFM.ini (WP12195) in preparation for next Tuesday maintenance and so the EDC came back with a different channel list to that of the rest of the DAQ. I reverted this file change and a second EDC restart was successful.

11:38:35 h1susauxb123 h1edc[DAQ]
11:46:17 h1susauxb123 h1edc[DAQ]

The slow controls h1pslopcsdf system was also unable to reconnect to the 4 PSL WD channels it monitors. This was restarted at 12:08 14nov2024 PST.

Erik found that MEDM on some workstations would continue to show white-screen for h1pslctrl0 channels and a full restart of MEDM was needed to resolve this.

LHO VE - Fri CP1 Fill

david.barker@ligo.org — Fri, 15 Nov 2024 10:22:25 -0800

Author: david.barker@ligo.org

Report ID: 81292

Fri Nov 15 10:12:59 2024 INFO: Fill completed in 12min 55secs

Images attached to this report