aLIGO LHO Logbook

H1 ISC - Briefly tested LD32 DAC SUS-EX

marc.pirello@ligo.org — Mon, 15 Jul 2024 17:03:32 -0700

Author: marc.pirello@ligo.org

Report ID: 79151

Today we conducted a brief test on the SUS-EX ESD DAC vs the SUS-EX LD32 DAC, both driving the PEM AA chassis at end X per WP11976. This testing was cut short due to relocking attempts. We changed the cable to the PEM output ~1:10pm PST, and replaced the cable back to its original position at ~2:15pm PST. We are still looking at the results of this testing.

F. Clara, R. McCarthy, M. Pirello, D. Sigg

ISC, SUS

H1 SYS - Comment to Fast shutter is OK

keita.kawabe@ligo.org — Mon, 15 Jul 2024 17:02:41 -0700

Author: keita.kawabe@ligo.org

Report ID: 79148

FYI, fast shutter has never failed to fire up when the trigger voltage crossed the threshold of 2V since I connected the ASC-AS_C analog sum to the PEM ADC to monitor the trigger voltage at 2kHz. Firing of the FS has been without any problem.

In the attached, top is the trigger voltage monitored by the ADC. Red circles indicate the time when the trigger crossed the threshold. Bottom plot shows when the fast shutter was driven with high voltage (sensed by GS13).

The reason why it looks as if the trigger level changed before and after t=0 on the plot is because a 1:11 resistive divider with a total resistance of 10kOhm was installed at t=0. Before that, the 2V threshold was 32k counts, 2980 counts after.

The connection to the ADC was first made at 18:30 UTC on Jul/11, which is at t~-1D in the plot.

Images attached to this report

H1 SEI - Comment to HAM2 moving a lot more than HAM3 around 40hz, probably time to do some re-tuning

jim.warner@ligo.org — Mon, 15 Jul 2024 17:00:25 -0700

Author: jim.warner@ligo.org

Report ID: 79150

I looked at some of my design scripts and realized that the HAM3 FF X filter is probably a bit better fit, so I copied that into the HAM2 foton file, loaded and engaged it on HAM2. It improved the 50ish hz motion quite a bit, but HAM2 is moving more than HAM3 still. Probably some tuning that could still be done here.

Images attached to this report

H1 ISC - Comment to Corrected DARM spectra for DCPD whitening on/off and DARM offset on/off on 3rd May

jennifer.wright@ligo.org — Mon, 15 Jul 2024 17:00:07 -0700

Author: jennifer.wright@ligo.org

Report ID: 79149

Added in a third plot of the power scaling in the anti-symmetric port with DARM offset changing the power at the DCPDs.

The inverse of the slope of this plot gives the loss term as in this entry.

loss term = 1/1.219 = 0.820,

the amount of light at the anti-symmetric port insensitive to DARM is 837.5 mW.

Non-image files attached to this report

P_as_vs_P_DCPD_gps_start_1367176162_430_kW__18_hr_lock__ETMX_RH_1_3__ETMY_RH_1_2.pdf

H1 ISC - Comment to DARM Offset Test

jennifer.wright@ligo.org — Mon, 15 Jul 2024 16:34:01 -0700

Author: jennifer.wright@ligo.org

Report ID: 79146

I added a plot showing the loss ( inverse of the slope of attached graph) between the input of HAM 6 (AS port) and the DCPDs as in this entry.

This loss term is 1/1.247 = 0.802 with 653.7 mW of light insensitive to DARM at the AS port.

Non-image files attached to this report

P_as_vs_P_DCPD_gps_start_1394637965_370_kW__25_hour_lock.pdf

H1 ISC - Comment to moved SR3, relocking after fast shutter test

keita.kawabe@ligo.org — Mon, 15 Jul 2024 16:33:38 -0700

Author: keita.kawabe@ligo.org

Report ID: 79147

IFO lost lock even before going DC and we had another FS test forced by the guardian, and it didn't fail, but the throughput is even worse.

A_DC_SUM=3.4k when the shutter was open, Closed/Open ratio is about 1000ppm, and a tiny part of the beam is being missed by the shutter (attached). Note that I'm NOT eyeballing the "open" Y cursor in log scale, I set it while in linear Y scale but changed it to log to show that the power after the shutter was closed seems to be measurably larger than it should be.

Maybe this is going on because of tiny alignment differences from lock to lock, but anyway this doesn't look to be the place we want.

Images attached to this report

H1 General - OPS Monday day shift summary

ryan.crouch@ligo.org — Mon, 15 Jul 2024 16:30:45 -0700

Author: ryan.crouch@ligo.org

Report ID: 79123

TITLE: 07/15 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Oli
SHIFT SUMMARY: We're testing/investigating if we can still lock, DRMI locks pretty easily so this alignment might be ok.

Rahul confirmed HAM5 SUSes looks healthy
Recreating the trend from alog77392 back in May, HAM6s power looks a little lower but not as large as previously, less than 2% vs 5% loss
17:22 UTC Camilla, Naoki, and Sheila ran an single bounce SQZ test and tried to do some centering
18:17 UTC A decent sized earthquake passed through
21:10 UTC We started an IA to move SR3 before trying to lock DRMI to check the fast shutter again
22:20 UTC Successful fast shutter test
22:30 UTC Daniel reset the atomic clock alog79119

LOG:

Start Time	System	Name	Location	Lazer_Haz	Task	Time End
15:08	FAC	Karen	Optics lab,. vpw	N	Tech clean	15:35
15:36	pem	Kim	MidX	N	Tech clean	16:17
16:14	CAL	Francisco	PCAL lab	LOCAL	PCAL work	17:09
16:14	FAC	Karen	PCAL lab	LOCAL	Tech clean	16:29
16:30	FAC	Karen	MidY	N	Tech clean	17:44
18:59	SUS	Jeff	TCSX rack	N	Take pictures, RMs	19:11
19:08	SUS	Jeff	Mech room racks	N	Take pictures	19:18
20:07	EE/PEM	Fil, Marc	EndX	N	Testing AA chassis for new DAQ	20:38
21:13	EE	Fil	EndX	N	Reset test setup	21:40
21:48	SUS	Jeff	TSCX rack	N	Pictures	22:11

Images attached to this report

H1 SEI - HAM2 moving a lot more than HAM3 around 40hz, probably time to do some re-tuning

jim.warner@ligo.org — Mon, 15 Jul 2024 16:29:55 -0700

Author: jim.warner@ligo.org

Report ID: 79145

Genevieve and Sam asked about HAM1 and HAM2 motion around 40 hz. I hadn't look in detail in this frequency band for a while, this is typically higher frequency than we are worried about with ISI motion. But it turns out HAM2 is moving a lot more than HAM3 generally over 25-55hz, and particularly around 39hz. It looks like it might be due to gain peaking from the isolation loops, but could also be from something bad in the HEPI to ISI feedforward. The extra motion is so broad I don't think it's just one loop has a little too much gain, so I'm not sure what is going on here.

First image are spectra comparing the motion of the HEPIs for those chambers (HAM2 HEPI is red and HAM3 is blue) and the ISIs (HAM2 ISI is green, HAM3 is brown). The HEPI motion is pretty similar, so I don't think it's a difference in input motion. HAM2 is moving something like 10x as much as HAM3 over 25-55hz. The sharp peak at 39 hz looks like gain peaking, but I'm not sure that explains all the difference.

Second plot shows the transfer functions from HEPI to ISI for each chamber. Red is HAM2, blue is HAM3. The 25-55hz tf for HAM3 is not very clean probably because HAM3 is well isolated. HAM2 tf is pretty clean, it makes me wonder if maybe something is messed up with feedforward on that chamber. Maybe that is something I could (carefully) fix while other troubleshooting for the detector is going on.

Images attached to this report

H1 ISC - Comment to DARM offset step

jennifer.wright@ligo.org — Mon, 15 Jul 2024 16:11:20 -0700

Author: jennifer.wright@ligo.org

Report ID: 79144

I plotted the power at the antisymmetric port as in this entry to find out the loss term between the input to HAM6 and the DCPDs, which in this case is (1/1.652) = 0.605 with 580.3 mW of light at the AS port insensitive to DARM length changes.

Non-image files attached to this report

P_as_vs_P_DCPD_gps_start_1402330422_380kW__Post_OFI_burn_and_pressure_spikes.pdf

H1 General - Ops Eve Shift Start

oli.patane@ligo.org — Mon, 15 Jul 2024 16:07:52 -0700

Author: oli.patane@ligo.org

Report ID: 79143

TITLE: 07/15 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 21mph Gusts, 18mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.05 μm/s
QUICK SUMMARY:

Currently in ENGAGE_ASC_FOR_FULL_IFO, which is currently doing okay. Winds have jumped up but look like they're going back down.

H1 ISC - Comment to DARM Offset step with hot OM2

jennifer.wright@ligo.org — Mon, 15 Jul 2024 15:58:54 -0700

Author: jennifer.wright@ligo.org

Report ID: 79130

I added an additional plotting step to the code and it now makes this plot which shows us how the power at AS_C changes with the DARM offset power at the DCPDs. The slope of this graph tells us what fraction of the power is lost between the input to HAM6 (AS_C) and the DCPDs.

P_AS = 1.770*P_DCPD + 606.5mW

Where the second term is light that will be rejected by the OMC and that which gets through the OMC but is insensitive to DARM length changes.

The loss term between the anti-symmetric port and the DCPDs is 1/1.77 = 0.565

Non-image files attached to this report

P_as_vs_P_DCPD_gps_start_1404768832_2_5_hours_into_lock__hot_OM2.pdf

H1 ISC - Comment to moved SR3, relocking after fast shutter test

keita.kawabe@ligo.org — Mon, 15 Jul 2024 15:55:33 -0700

Author: keita.kawabe@ligo.org

Report ID: 79141

Fast shutter behavior is attached. It's working fine, but the throughput to HAM6 is ~14% down compared with before.

Before the shutter was closed, ASC-AS_A_DC_NSUM was ~3.7k counts, and ~0.75 counts after (fractional number because of decimation to 2k). That's about 200ppm.

However, it used to be ~4.3k and 1cts on one of the "happy" plots alog 79131, 3.7k/4.3k~0.86, so the throughput to HAM6 seems to be ~14% lower than before.

Images attached to this report

H1 DAQ - Comment to Previous/spare timing master

daniel.sigg@ligo.org — Mon, 15 Jul 2024 15:54:22 -0700

Author: daniel.sigg@ligo.org

Report ID: 79142

Today's readback voltage is at +0.951V.

H1 ISC - moved SR3, relocking after fast shutter test

sheila.dwyer@ligo.org — Mon, 15 Jul 2024 15:43:11 -0700

Author: sheila.dwyer@ligo.org

Report ID: 79140

Sheila, Keita, Ryan Short, Camilla and TJ

Part way through initial alignment, we stopped and moved SR3 towards the positive yaw spot from 79103, and used the SR2 osem values from that time. A small manul adjustment of AS_C was needed, otherwise initial alingment was uneventful.

With DRMI locked, we ran the fast shutter test that Keita and Ryan S have both looked at.

We also looked at the ratios of AS PDs to compare to 78667, they are most similar to the good times in that alog. After these checks we have decided to try locking.

as_c/as_a is now 1.22e-5
as_c/as_b is now 1.17e-5

H1 CDS - Comment to Atomic Clock lost sync with timing-system

david.barker@ligo.org — Mon, 15 Jul 2024 15:40:41 -0700

Author: david.barker@ligo.org

Report ID: 79139

Daniel resynced the Atomic Clock to the timing 1PPS at 15:30 PDT this afternoon. The corner station timing error has gone away.

Images attached to this report

H1 AOS - Comment to April 18 Seismic Timeline, Lock, Lockloss Summary

neil.doerksen@ligo.org — Mon, 15 Jul 2024 15:34:41 -0700

Author: neil.doerksen@ligo.org

Report ID: 79138

Using KAPPA_C channel.

Images attached to this report

H1 ISC - Comment to Moved SR3 with SR2 following to find spots with good AS port power

camilla.compton@ligo.org — Mon, 15 Jul 2024 15:15:42 -0700

Author: camilla.compton@ligo.org

Report ID: 79135

	Before	+P move	-P move	+Y move	-Y move
Time	1404952764 2024/07/14 00:39:06 UTC	1404955465 2024/07/14 01:24:07 UTC	1404958133 2024/07/14 02:08:35 UTC	1404959716 2024/07/14 02:34:58 UTC	1404963518 2024/07/14 03:38:20 UTC
H1:SUS-SR3_M1_OPTICALIGN_P_OFFSET	438.7	651.1	138.7	438.7	438.7
H1:SUS-SR3_M1_OPTICALIGN_Y_OFFSET	438.7	122.2	122.2	322.2	-167.8
H1:SUS-SRM_M1_DAMP_P_INMON	-1033.7	-1035.2	-1036.3	-1036.1	-1037.1
H1:SUS-SRM_M1_DAMP_Y_INMON	913.7	914.0	914.1	914.1	914.2
H1:SUS-SR2_M1_DAMP_P_INMON	597.7	-871.6	2660.2	614.7	566.2
H1:SUS-SR2_M1_DAMP_Y_INMON	1125.3	1179.9	1069.3	1878.8	-72.4
H1:SUS-SR3_M1_DAMP_P_INMON	-290.2	-57.7	-619.4	-297.9	-279.1
H1:SUS-SR3_M1_DAMP_Y_INMON	-411.0	-425.2	-390.4	-256.9	-633.7

Here are the OSEM values, so that Alena can cotinine her 78268 analysis of the OFI beam spot position.

X1 SUS - Comment to BBSS M1 BOSEM Count Drift Over Last Week - Temperature Driven Suspension Sag

ibrahim.abouelfettouh@ligo.org — Mon, 15 Jul 2024 15:04:47 -0700

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 79137

Ibrahim

Taking new trends today shows that while the suspension sag "breathes" and comes back and forth as the temperature fluctuates on a daily basis, the F1 OSEM counts are continuing to trend downwards despite temperature not changing peak to peak over the last few days.
This F1 OSEM has gone down an additional 670 cts in the last 4 days (screenshot 1). Screenshot 2 shows the OSEM counts over the last 11 days. What does this tell us?

What I don't think it is:

It somewhat disproves the idea that the F1 OSEM drift was just due to the temperatures going up, since they have not leveled out as the temperatures have - unless for some reason something is heating up more than usual
A suggestion was that the local cleanroom temperature closer to the walls was hotter but this would have an effect on all OSEMs on this face (F2 and F3), but those OSEMs are not trending downwards in counts.
It is likely not an issue with the OSEM itself since the diagnostic pictures (alog 79079) do show a percieveable shift when there wasn't one during centering, meaning the pitch has definitiely changed, which would show up on the F1 OSEM necessarily.

What it still might be:

The temperature causes the Top Stage and Top Mass blades to sag. These blades are located in front of one another and while the blades are matched, they are not identical. An unlucky matching could mean that either the back top stage blade or two of the back top mass blades could be sagging net more than the other two, causing a pitch instability. Worth check
It is not temperature related at all, but that the sagging is revealing that we still have our hysteresis issue that we thought we fixed 2 weeks ago. This OSEM has been drifting in counts ever since it was centered, but the temperature has also been drastically in that time (50F difference between highs and lows last week).

Next Steps:

I'm going to go set up temperature probes in the cleanroom in order to see if there is indeed some weird differential temperature effect specifically in the cleanroom. Tyler and Eric have confirmed that the Staging Building temperature only really fluctuates between 70 and 72 so I'll attempt to reproduce this. This should give more details about the effect of temperature on the OSEM drift.
See using the individual OSEM counts and their basis DOF matrix transformation values if there's a way to determine that some blades are sagging more than others via seeing if other OSEMs are spotting it.
- Ultimately, we could re-do the blade position tests to difinitively measure the blade height changes at different temperatures. I will look into the feasibility of this.

Images attached to this report

H1 SYS - Comment to Fast shutter is OK

keita.kawabe@ligo.org — Mon, 15 Jul 2024 14:53:26 -0700

Author: keita.kawabe@ligo.org

Report ID: 79131

Seems like ASC-AS_B_DC gain was probably a red herring, important thing is that the beam got uglier/bigger at some point, therefore a part of the beam was not blocked by the fast shutter.

First attachment shows when ASC-AS_B_DC gain switch was flipped from x10 to x1 on Tuesday. You can see that the Fast Shutter has been firing OK until Friday evening.

The rest of the plots show the FAST Shutter test done by the Beckhoff at three different points in time, i.e. the last test before my AS_B_DC change ("happy" Monday July/08 ~16:37 UTC or 9:37 local), the first one after my AS_B_DC change ("happy" July/11 4:43 UTC or Wed July/10 21:43 local time), and the first time the FAST shutter went into the error mode ("sad" Jul/13 4:56 UTC or Fri Jul/12 21:56 local time). The last one is when Sheila and Ryan started having problem.

Important points are:

The two "happy" plots are quite similar. Especially, even though I see bouncing action, the power measured by both AS_A and AS_B more or less settle to the minimum value. Ratio of closed/open is about 230ppm in both cases. This is compatible with the specification of Fast Shutter mirror (which is <1000ppm).
The "sad" plot looks nothing like the happy ones. Right after the shutter was closed the light level went down to less than 1000ppm, but bounced back and eventually settled down to ~6500ppm. You can also see that the light level when FS was open (~2200 cts) is about 50% of what it used to be in happy plots.
ASC-AS_C was pretty close to center in all of the three plots.

From these, my conclusion is that the beam position on the fast shutter mirror was pretty much the same in all three tests, but the beam was pretty ugly for the "sad" plot as was witnessed by many of us on AS port camera. Because of this a part of the lobe was missed by the Fast Shutter. Centering an ugly beam on AS_C might have complicated the matter.

Later, when I forced the test with much lower power, the error was cleared because even though the ugly beam was still there the power went lower than the "shutter closed" threshold of the guardian.

I don't fully understand who did what when during the time the shutter was in the error state (it includes people pressing buttons and power cycling the driver and then pressing buttons again, and I certainly pressed buttons too).

Looking at this, and since Daniel agrees that the Fast Shutter has been working fine, my only concerns about locking the IFO are:

The beam should be properly attenuated by the fast shutter. Not thousands of ppm, but more like ~230ppm.
The beam should not clip on ASC-AS_C. If it's not clipped there, the chances of thing getting clipped downstream is very small.

Images attached to this report

H1 OpsInfo - Added manual_control parameter to lscparams to avoid some automation states when True

thomas.shaffer@ligo.org — Mon, 15 Jul 2024 14:52:08 -0700

Author: thomas.shaffer@ligo.org

Report ID: 79136

Sheila, TJ

In times of heavy commissioning or troubleshooting, we want full control of the IFO and don't want to fight the automation. Sheila suggested that we add a manual_control boolean to lscparams.py that ISC_LOCK will look at to decide whether it will automatically run states like Check_Mich_Fringes, PRMI, Increase_Flashes, etc. When this is set to True, it will avoid these automated state either through a conditional in a state's logic, or by weighting edges to force ISC_LOCK to avoid particular states.

For now we are setting manual_control = True while we troubleshoot the IFO. We will need to remember to return it when we want fully automatic operation again.

ISC

H1 AOS - Comment to April 18 Seismic Timeline, Lock, Lockloss Summary

neil.doerksen@ligo.org — Mon, 15 Jul 2024 13:18:45 -0700

Author: neil.doerksen@ligo.org

Report ID: 79133

Here are some plots around the EQ from April 18, 2024 at which USGS reports coming from Canada at 06:06.

Images attached to this report

H1 AOS - April 18 Seismic Timeline, Lock, Lockloss Summary

neil.doerksen@ligo.org — Mon, 15 Jul 2024 12:29:48 -0700

Author: neil.doerksen@ligo.org

Report ID: 79132

Pre April 18

Camera offsets made : 77139 : April 12
Unknown issues began with FC IR unlocking : 77164 : April 14
- Solved : high winds : April 16
Wind induced lockloss : 77192 : April 15
- Audible creak in Control Room
- 5.9 EQ in S. Korea inbound
Windy making lock difficult : 77188 : April 16
Squeezing having lock issues 77219 : April 16
Lockloss - IX saturation : 77224 : April 16 16:41
Lockloss - reason unknown : 77233 : April 16
Nominal SQZ angle changes lock-to-lock : 77238 : April 17
Feedforward and squeezing problem solving

April 18

Lockloss, unkown cause, Apr 18 01:02 UTC : 77275
USGS reports 4.3 in Gulf of Cal Apr 18 04:22:45 UTC
Lockloss, Apr 18 04:46
USGS reports 5.1 off CDN coast Apr 18, 06:06:04 UTC
Lockloss from EQ @ Apr 18 06:11UTC : 77259
USGS reports 4.3 off CDN coast Apr 18, 06:41:45 UTC
USGS reports 5.2 in Gulf of Cal Apr 18 07:39:39 UTC
Still aligning : EQ 5.3 from Mexico April 18, 2024 07:39:45UTC : 77261
USGS reports 4.1 off CDN coast Apr 18, 08:40:12
USGS reports 3.5 off west coast of Cal Apr 18, 09:33
USGS reports 4.2 in Gulf of Cal Apr 18, 11:14:04
USGS reports 2.7 in N Cal Apr 18, 11:28
USGS reports 2.6 in N Cal Apr 18, 13:21
USGS reports 5.6 in Gulf of Cal Apr 18, 15:12:30
USGS reports 3.6 in W Wyom Apr 18, 15:15
USGS reports 4.3 in S Cal Apr 18, 19:19

Post April 18

Locked and observing for 36 min : 77280
Locked for >8.5 hrs : 77281 : April 19, 08:02
Back to FF and SQZ troubleshooting
Lockloss, errant gain change at 21:56 : 77283
Locked for ~4 hours at Apr 19 20:02
USGS reports 2.9 N of Badger Mt Community Park : Apr 20 04:18:57
- Stayed locked : 77299 : Apr 20, 00:00
Lockloss Apr 2024, 15:23 : 77301
Lockloss at Apr 20, 22:25
2 relocks : Apr 21 23:58 (windy)
Lockloss 26 min after mid-shift entry : 77318
- Claim - 5.4 EQ from Mexico at 13:43 cause lockloss
- USGS reports a 5.4 EQ from 19:05 or a 4.4 EQ at 07:28
- I am probably getting time zones mixed up

Locked and observing for 36 min : 77280
Locked for >8.5 hrs : 77281 : April 19, 08:02
Back to FF and SQZ troubleshooting
Lockloss, errant gain change at 21:56 : 77283
Locked for ~4 hours at Apr 19 20:02
USGS reports 2.9 N of Badger Mt Community Park : Apr 20 04:18:57
- Stayed locked : 77299 : Apr 20, 00:00
Lockloss Apr 20, 15:23 : 77301
Lockloss at Apr 20, 22:25
2 relocks : Apr 21 23:58 (windy)
Lockloss 26 min after mid-shift entry : 77318
- Claim - 5.4 EQ from Mexico or wind at 13:43 cause lockloss
- USGS reports a 5.4 EQ from 19:05 or a 4.4 EQ at 07:28
- I am probably getting time zones mixed up, but these times conflict

SEI

H1 ISC - Translating SQZ beam after IFO alignemt shift

camilla.compton@ligo.org — Mon, 15 Jul 2024 12:25:35 -0700

Author: camilla.compton@ligo.org

Report ID: 79128

Naoki, Sheila, Camilla

To continue investigating alignment changes: 79105. More details in Sheila's 79127.

We put SRM back to before alignment shift, misaligned SRM2 and ITMs, injected 1.28mW of SQZ SEED beam (as measured on SQZT7 OPO_IR_PD_DC_POWERMON). Followed as in 78115. Increased AS_AIR camera exposure form 10,000 to 100,000

Translated SQZ beam around last NLN spot, each direction the throughput was getting better but in both pitch directions we saturated ZM4 before moving far. Were able to get all expected 0.9mW throughput by moving ZM5 Pitch and yaw both positive.

In starting alignment ASC-AS_C measured 0.1 to 0.2mW. Same after we centered beam on AS-C. Time: 2024/07/15 17:52:14 UTC Plot. AS_AIR beam very bad.
Moving ZM5 Positive YAW (recentering with ZM4 +Y), increased AS_C measured to 0.6mW before it started getting worse. Time: 2024/07/15 18:10:56 UTC Plot
From this spot (ZM5 Positive YAW) we moved ZM5 Positive PITCH (recentering with ZM4 -P), increased AS_C measured to 0.9mW before it started getting worse Time: 2024/07/15 18:16:33 UTC Plot AS_AIR beam got better.
We then went back to the original ZM4/5 alignment
Moving ZM5 Negative YAW (recentering with ZM4 -Y) we were improving but saturating ZM4 before we could finish, AS_C measured to 0.44mW, Time: 2024/07/15 18:55:00 UTC Plot
Moving ZM5 Positive PITCH (recentering with ZM4 -P) we were improving but saturating ZM4 before we could finish, AS_C measured to 0.25mW, Time: 2024/07/15 19:01:57 UTC Plot
Moving ZM5 Negative PITCH (recentering with ZM4 +P) we were improving but saturating ZM4 before we could finish, AS_C measured to 0.22mW Time 2024/07/15 19:12:57 UTC Plot
Note the max we got when not aligned on AS_C was around 0.1mW, maybe caused by edge effects as expect 0.7% reflection off SRM HR surface = 0.7 x 1.28mW = 0.896mW MAX.

SQZ

Images attached to this report

H1 AOS - Computing the average relock time for O4a

mattia.emma@ligo.org — Mon, 15 Jul 2024 11:52:08 -0700

Author: mattia.emma@ligo.org

Report ID: 79129

The two attached histograms show the relock times for LIGO Hanford during the whole of O4a.

The average relock time has been 2 hours and 55 minutes, while the median was at 1 hour and 33 minutes.

The relock time was computed as the time for which the ISC_LOCK_N value was consecutively below 580.

The 580 value corresponds to inject squeezing, which is why there were some short "relocks" when we used below 600 for the lockloss threshold, because the squeezer unlocked and relocked automatically.

Link to the github repository with the employed code and data

DetChar

Images attached to this report

H1 SQZ - power estimates for sqz beams

sheila.dwyer@ligo.org — Mon, 15 Jul 2024 11:29:52 -0700

Author: sheila.dwyer@ligo.org

Report ID: 79127

Summary: We think that the sqz beam is seeing similar loss to the IFO beam in th alignment that we were running in until Friday, but double passed. The sqz beam transmission can be restored by moving the beam, and there should be enough power in this beam to be a useful diagnostic in chamber to distinguish between OFI and SRM damage.

Camilla and Noaki have measured 1.3mW coming from HAM7 towards HAM5 with the OPO locked on the seed. SRM transmission is 32.34%, ( 72680 ) so if nothing was damaged we should have something like 0.41mW of the OPO seed beam in transmission of SRM, which we should be able to measure with a power meter.

The table in 79101 indicates that the single bounce transmission to AS_C has dropped to 64% of what it was last Thursday. Here's a table of power predictions for the sqz beam arriving in HAM6:

no damage	sqz seed beam we expect at AS_C
no damage	1.3mW(1-0.3234) 0.99^2 = 0.86mW
single pass of damaged optic with 0.64 transmission	1.3(1-0.3234)0.990.99 0.64 = 0.55mW
double pass of damaged optic with 0.64 tramission	1.3(1-0.3234)0.990.99 0.64 *0.64 = 0.35mW

With the alignment used for observing in recent weeks Naoki and Camilla measured 0.2 mW on AS_C QPD (not well centered on AS_C, but on the diode), which is calibrated into Watts arriving in HAM6. They moved ZM4 +5 to improve the transmission of the sqz beam to 0.9mW when centered on AS_C. These numbers don't agree super well with the predictions above, but they seem to suggest that the SQZ beam sees similar damage to the main IFO beam, but double passed.

IOO, ISC

H1 ISC - Current dip in output arm gain seems more abrupt compared to previous OFI 'burn' on 22nd April

jennifer.wright@ligo.org — Mon, 15 Jul 2024 10:57:10 -0700

Author: jennifer.wright@ligo.org

Report ID: 79125

Sheila asked me to look at the same trends we used to track the damage in the OFI over the 22nd and 23rd of April when it first happened. We think something like this might have happened again on Friday.

Attached you can see the optical gain, coupled cavity pole, range in Mpc, power at the anti-symmetric port, circulating power, and power reflected from the OMC all changed between April the 22nd, when the damage first started affecting the range causing it to decrease slowly.

In the next image we have the same trends from the last couple of days, there doesn't seem to be a slow degradation in range in this case, the range got worse between two locks on the 12th but nothing else seems to have degraded (unlike beforte when we observed a drop in optical gain, circulating power, output power and coupled cavity pole) before we got to the point where we couldn't relock.

Images attached to this report

H1 ISC - Comment to we suspect that something in OFI has changed

ryan.crouch@ligo.org — Mon, 15 Jul 2024 10:21:16 -0700

Author: ryan.crouch@ligo.org

Report ID: 79124

Recreating this trend for our current situation, power drops aren't as big? Comparing a lock before last Thursday (07/11) to our most recent lock.

Images attached to this report

H1 General - Comment to OPS Monday day shift start

david.barker@ligo.org — Mon, 15 Jul 2024 09:36:14 -0700

Author: david.barker@ligo.org

Report ID: 79122

Jonathan's emailed instructions on how to use the local archived DCC (read-only):

1. Goto https://dcc-lho.ligo.org
2. click login
3. Do not click the ligo icon that says fed-proxy (this is what is
broken).
4. Click on the link that says select from a list
5. In the drop down list select the Backup::LHO option
6. Click Ok
7. log on as normal.

This will work. But you cannot push new documents and it is about a
day out of date.

As for other services, if you get a choice of selecting the backup LHO
login will work that way. Anything that does not and just goes
straight to the federated proxy (ie like the dcc, you goto a page that
says cilogon.org to select your institution) will not work right now.

This is being worked on and should clear up later.

LHO VE - Comment to Pressure glitch on PT100B HAM1 at 07:25:15 Sun 14 July 2024

jordan.vanosky@ligo.org — Mon, 15 Jul 2024 09:33:03 -0700

Author: jordan.vanosky@ligo.org

Report ID: 79121

IP13 ion current/pressure spiked up ~1 second before the PT100B gauge did, see attached. Likely argon instability of the pump

Images attached to this report

LHO FMCS - HVAC Fan Vibrometers Check - Weekly

ryan.short@ligo.org — Mon, 15 Jul 2024 09:28:44 -0700

Author: ryan.short@ligo.org

Report ID: 79120

FAMIS 26315, last checked in alog78896

No appreciable change in any fan's noise levels compared to last check.

PEM

Images attached to this report

H1 PSL - PSL 10-Day Trends

ryan.short@ligo.org — Mon, 15 Jul 2024 09:21:56 -0700

Author: ryan.short@ligo.org

Report ID: 79118

FAMIS 21008

No major events of note for these trends except for the PSL work done last Tuesday (seen in almost every trend), and, concerningly, a slight rise in PMC reflected power of about 0.3W over 5 days. It seems to have leveled off in the past day or so, however.

Images attached to this report

H1 CDS - Atomic Clock lost sync with timing-system

david.barker@ligo.org — Mon, 15 Jul 2024 09:16:40 -0700

Author: david.barker@ligo.org

Report ID: 79119

at 08:50 PDT the MSR Atomic Clock lost sync with the timing system, the comparator is reading a steady 0.24S 1PPS difference. This has happened before, it requires a manual resync of the Atomic Clock to the timing-system 1PPS. This was last done 6th May 2024

Images attached to this report

H1 General - Comment to Lockloss at 16:44 UTC

camilla.compton@ligo.org — Mon, 15 Jul 2024 09:05:00 -0700

Author: camilla.compton@ligo.org

Report ID: 79111

We turned on a PRCL FF the day before: 79035. But this 3.6Hz PRCL wobble is normal, it was constant throughout the lock plot and locks before the feedforward was installed (example).

This lockloss looked very normal, AS_A then IMC loosing lock, as usual, plot.

Images attached to this report

H1 General - Comment to Comparisons before/after alignment changes

camilla.compton@ligo.org — Mon, 15 Jul 2024 08:42:55 -0700

Author: camilla.compton@ligo.org

Report ID: 79117

Looking more at the OFI TEC/temperature sensor, it seems like the alignment through the OFI changed, into probably a worse alignment that clipped somewhere (requiring more OFI temperature control as we powered up), at a time between the 2024/07/12 16:45UTC IFO unlock (looked normal, maybe 10-20% larger than normal temp swings) and the Power up at 17:45UTC. See attached. This is before we explicitly changed the SRC alignment.

ISC

Images attached to this report

H1 General - Comment to OPS Monday day shift start

david.barker@ligo.org — Mon, 15 Jul 2024 08:14:48 -0700

Author: david.barker@ligo.org

Report ID: 79116

I'm seeing the same issue from off-site, there is now a message on the page indicating a server issue

Images attached to this report

LHO VE - Mon CP1 Fill

david.barker@ligo.org — Mon, 15 Jul 2024 08:11:57 -0700

Author: david.barker@ligo.org

Report ID: 79115

Mon Jul 15 08:06:22 2024 INFO: Fill completed in 6min 18secs

Jordan confirmed a good fill curbside.

Images attached to this report

H1 General - Comment to OPS Monday day shift start

ryan.crouch@ligo.org — Mon, 15 Jul 2024 08:06:05 -0700

Author: ryan.crouch@ligo.org

Report ID: 79114

I'm getting a 502 proxy error whenever I try to load the DCC

CDS

H1 General - OPS Monday day shift start

ryan.crouch@ligo.org — Mon, 15 Jul 2024 07:44:08 -0700

Author: ryan.crouch@ligo.org

Report ID: 79113

TITLE: 07/15 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 8mph Gusts, 5mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.05 μm/s
QUICK SUMMARY:

The IFO is still down, pending more investigations

LHO VE - Comment to Pressure glitch on PT100B HAM1 at 07:25:15 Sun 14 July 2024

michael.zucker@ligo.org — Mon, 15 Jul 2024 07:41:53 -0700

Author: michael.zucker@ligo.org

Report ID: 79112

Given no lockloss correlation, may not be significant; ~3 orders of magnitude less PV than the June event(s). Maybe an ion pump burped up some argon.

H1 AOS - High Dust Count

neil.doerksen@ligo.org — Sun, 14 Jul 2024 15:36:17 -0700

Author: neil.doerksen@ligo.org

Report ID: 79110

High dust count.

Images attached to this report

LHO VE - Comment to Pressure glitch on PT100B HAM1 at 07:25:15 Sun 14 July 2024

jenne.driggers@ligo.org — Sun, 14 Jul 2024 11:31:11 -0700

Author: jenne.driggers@ligo.org

Report ID: 79109

This is clearly very concerning.

Attached is a trend of the pressure in the top right, along with several other channels (IMC power in, ISC_LOCK guardian state, RM alignments, DC power levels on all 4 PDs in that chamber), and I don't see anything in any of these channels that looks suspicious.

EDIT to add second screenshot, of in-vac tabletop L4C seismic sensors. Again, nothing suspicious-looking to me that I see yet.

Images attached to this report

LHO VE - Pressure glitch on PT100B HAM1 at 07:25:15 Sun 14 July 2024

david.barker@ligo.org — Sun, 14 Jul 2024 09:15:06 -0700

Author: david.barker@ligo.org

Report ID: 79108

There was a pressure glitch on PT100B HAM1 cold-cathode at 07:25:15 PDT this morning. There was no corresponding glitch on any other corner station gauge as far as I can tell.

Pressure increased from 3.3e-08 to 4.0e-08 Torr, recovered in 4 minutes.

Images attached to this report

LHO VE - Sun CP1 Fill

david.barker@ligo.org — Sun, 14 Jul 2024 08:13:51 -0700

Author: david.barker@ligo.org

Report ID: 79107

Sun Jul 14 08:04:08 2024 INFO: Fill completed in 4min 5secs

Images attached to this report

LHO General - Ops Eve Shift Summary

ryan.short@ligo.org — Sun, 14 Jul 2024 01:09:06 -0700

Author: ryan.short@ligo.org

Report ID: 79106

TITLE: 07/14 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
SHIFT SUMMARY: H1 remained down again for this whole shift, and we will not be attempting to lock until further discussion can be had on the recent alignment changes. This is planned for Monday morning, which means H1 will not be observing until at least then.

The main activities of the shift were in pursuit of diagnosing the output alignment changes seen in the IFO recently, including my SR3 moves looking for good power spots (alog79103), further diagnosis of abnormal fast shutter behavior from last night (alog79102), and Sheila's comments on this situation being similar to that of the output change in late April (alog79101).

H1 General - Comparisons before/after alignment changes

ryan.short@ligo.org — Sun, 14 Jul 2024 01:04:59 -0700

Author: ryan.short@ligo.org

Report ID: 79105

I've gathered several trends comparing the before and after of the alignment shift the IFO saw in the past couple of days, likely on Friday the 12th.

On all of these plots, I use y-cursors to indicate the levels each signal was at the appropriate time leading up to the last lock H1 had fully up to Nominal Low Noise, which lasted from 10:14 to 16:45 UTC on July 12th, in order to compare to what was likely the last time H1 was behaving "well."

DRMI signals with OM1/2 OSEMs:

The last DRMI acquisition before NLN, POP18 was around 66 counts while POP90 was around 11 counts.

The most recent DRMI acquisition, POP18 has dropped to 42 counts while POP90 has risen to 16 counts. The only significant alignment difference I see is with OM1 Y being about 70 counts different.

Optic alignments during 10W single-bounce: (times all taken from when ALIGN_IFO was in the state SR2_ALIGN [58])

The last SR2 alignment before NLN (P/SRC optics; large optics), AS_C_NSUM signal was at 0.0227 counts.

The next time in this configuration (P/SRC optics; large optics), AS_C_NSUM signal had dropped to 0.018 counts. The most obvious alignment changes are with PR2Y, PRM, some SR2, and SRM.

The most recent time in this configuration (P/SRC optics; large optics), AS_C_NSUM signal had dropped again to 0.014 counts. The same optics are off in alignment as before, but PRM has now flipped to being off in the opposite directions for both pitch and yaw.

OFI and KAPPA_C:

I recreated trends like Camilla did in alog78399 to check the behavior of the OFI. See attached for multi-day trend, zoomed in on shakes during high-state locklosses, and zoom in on one of these shakes.

HAM6 vacuum pressure:

Out of an abundance of caution, I trended the pressure gauge on HAM6 since we've seen pressure spikes somewhat recently, and I don't see anything that hasn't been previously noted on a 21-day timescale (the pressure rise ~20 days ago was noted to be because of the OM2 thermistor in alog78829).

ISC, OpsInfo

Images attached to this report

H1 SYS - Comment to Fast shutter is OK

keita.kawabe@ligo.org — Sat, 13 Jul 2024 23:35:31 -0700

Author: keita.kawabe@ligo.org

Report ID: 79104

That story agrees with my observation on Friday night when I started looking at the FS after 11:15 PM.

Sheila reported (79092) that the Fast Shutter reopened only after ~50ms or so. It seems that the low voltage drive to keep the shutter closed was not working. 1st attachment shows the last time that happened at around 23:10 local time. Daniel points out that the shutter was in an error state at that time but that was after Ryan power cycled the FS driver. We don't know exactly what kind of state the fast shutter was in here.

The next time the HV firing was tested was at 23:23 local time (2nd attachment), the shutter was kept shut (i.e. low voltage thing was somehow working) but there are two things to note.

H1:ASC-AS_B_NSUM_OUT_DQ was about 25 or so, i.e. larger than the dark offset threshold of +-15. The FS guardian go to failed state.
Bouncing was worse than I've ever seen. It bounced multiple times with 40 to 50ms period and eventually settles to the "closed" posotion. (Before Friday I only saw single bounce and that was it.)

The last FS test I've done was 00:12 local time on Jul/13 when the error was cleared, with the smaller power than nominal (3rd attachment). Bouncing was as bad but the power coming to HAM6 was smaller (see the trigger power at the top left). AS_B_NSUM was somewhat smaller (more like 10).

The reason why AS_B_NSUM is worse is because I reduced the analog DC gain by a factor of 10 and compensated for that by digital gain. The effect of analog offset as well as ADC/electronics noise are 10x worse than AS_A. I adjusted the dark offset while IMC was unlocked but we can probably increase the threshold to 30 or so if it continues to bother us.

Bouncing behavior might be more serious as it could mean that the beam was close to the end of the travel of the FS mirror (and it was bad on Friday because of funny alignment), or low voltage drive was somehow still funny. I suspect the former.

Images attached to this report

H1 ISC - Moved SR3 with SR2 following to find spots with good AS port power

ryan.short@ligo.org — Sat, 13 Jul 2024 21:16:28 -0700

Author: ryan.short@ligo.org

Report ID: 79103

To help characterize the recent issues we've seen with IFO alignment that are familiar to the output change in late April, I moved SR3 in 10W single-bounce configuration with the SRC2 ASC loop on so that SR2 would follow (ALIGN_IFO's SR2_ALIGN state), much like in alog77694. I moved SR3 far enough in each direction (+/- pitch and yaw) to bring the ASC-AS_C_NSUM signal back up to our target value of 0.022, which I was successful doing in every direction except -Y, where I could only bring AS_C up to 0.018. In the three directions where I was successful, the beam spot on the AS AIR camera looked much more like the clear circle we're used to seeing and less like the upside-down apostrophe we have now.

It seems that our old output spot (starting place in TJ's alog) is still bad (-Y direction with SR3 from current place) since that was the only direction where I couldn't get the AS_C power back up.

Slider values of SR3 after each move:

	Start	+P move	-P move	+Y move	-Y move
SR3 P slider	438.7	651.1	138.7	438.7	438.7
SR3 Y slider	122.2	122.2	122.2	322.2	-167.8

Attachment 1 is the starting AS AIR camera image (and our current spot), attachment 2 is after the +P move, attachment 3 is after the -P move, and attachment 4 is after the +Y move.

Images attached to this report

H1 SYS - Fast shutter is OK

daniel.sigg@ligo.org — Sat, 13 Jul 2024 20:27:32 -0700

Author: daniel.sigg@ligo.org

Report ID: 79102

Yesterday, the fast shutter test failed due to dark offsets in the AS WFS DC NSUM channels.

The guardian started running the TwinCAT testing code, which works as intended: it sends a close command to the trigger logic, which in turn fires the fast shutter. The fast shutter works fine as can be seen on the HAM6 geophones. The slow controls readbacks also indicate that both fast shutter and PZT shutter are closed no later than 200ms after the trigger. However 0.5sec after the guardians started the test, it checks the AS WFS DC NSUM outputs and compares them against dark offset limits of ±15. Since the dark offset on WFS B was over 30, the guardian then sent an abort command to the TwinCAT code and reported a failure.

H1 ISC - some symptoms that seem similar to April 22nd/23rd output change

sheila.dwyer@ligo.org — Sat, 13 Jul 2024 17:19:50 -0700

Author: sheila.dwyer@ligo.org

Report ID: 79101

For a summary of links related to the 22nd/23rd incidents, see

The first two screenshots show the AS air camera with 2W out of the PSL in single bounce, reflecting off ITMY. This seems very similar to what happened on April 22nd/23rd, where the as camera image looked bad when we were centered on AS_C, but looks sort of normal when we were way off center.

The single bounce throughput is also low, similar to what we saw on April 23rd. Jenne looked at some times when the initial alignment guardian did SR2 offload, and narrowed down when the single bounce throughput changed to sometime between 9:23 and 23:22 UTC yesterday.

	IMC-PWR_IN (Watts out of PSL)	IMC-IM4_TRANS_NSUM (has had clipping at some of these times) in Watts onto PRM	AS_C NSUM (Watts into HAM6)
April 16 77204	10.01	9.14	0.0229
April 24th 77441	10.005	9.62	0.023
12 July 2024 09:23:57 (SR2 align time Jenne found)	10.0	8.7 (clipped)	0.0227
12 July 23:22:55 (SR2 align time that Jenne found)	10.01	8.7 (clipped)	0.0185
now July 13 23:35:51 UTC	10.0	8.67	0.0148

Images attached to this report

H1 General - OPS Saturday day shift summary

ryan.crouch@ligo.org — Sat, 13 Jul 2024 16:31:01 -0700

Author: ryan.crouch@ligo.org

Report ID: 79095

TITLE: 07/13 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:

16:00 UTC MANUAL_IA I manualed to SRC align in INIT_ALIGN where I got stuck at SRY as I expected, so I misaligned SRM and touched SR2 to maximize the ASC-AS_A_DC signal. I had to move a lot in YAW, I realigned it and started on SRM next. AS AIR looks good now, the ASC SRC loops seem to be making it worse? It degrades it until it loses lock. I adjusted it by hand, went back and finished the IA parts I skipped then tried SRC again to the same result so I went out of IA and went back into locking. I ended up having to make the following moves:
- SR2: P -7.0 --> 62.4, Y 2044.0 --> 2052.0 (Reverted)
- SR3: P 438.8 --> 440.1, Y 122.3 --> 122.8
17:45 UTC The camera spot looked much better now, and DRMI flashes were good, now SRC2 loop killed the lock now (makes sense as I couldn't complete SRC align). Sheila and I then held at DRMI_PREP_ASC and I turned off DC3 & 4 and clicked TEST on the SYS asport_protection medm screen to check the fast shutter to confirm it was ok. alog79097
18:43 UTC Held us in down as the R waves of an earthquake rolled in and we couldn't hold als
19:32 UTC I got to PREP_ASC again and ran the test again
- The DC CENTERING loops killed it?
- No SRC1 loop and I turned off all 4 DC loops as soon as they came on and same behaviour
- PRMI, I adjusted PR2 a little bit and it made the camera better, we stayed locked through the PRMI to DRMI transition now
- Both SRC1 and 2 were False and RF90 still oscillated and lost lock
22:08 UTC another earthquake shaking things up
22:19 UTC dropped to do another IA after touching PR3 a bit (less than a microradian)
- SR2 align made the camera look bad again and put it back into its previous spot, Making the camera better moves us off center on ASC-AS_C?
Into single bounce mode, where we recentered SR2 on ASC-AS_C

Images attached to this report

H1 ISC - Comment to Summary of alogs related to changes in our output arm this week

sheila.dwyer@ligo.org — Sat, 13 Jul 2024 16:08:36 -0700

Author: sheila.dwyer@ligo.org

Report ID: 79100

comparisons of single bounce throughputs: 77441

LHO General - Ops Eve Shift Start

ryan.short@ligo.org — Sat, 13 Jul 2024 15:56:22 -0700

Author: ryan.short@ligo.org

Report ID: 79099

TITLE: 07/13 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 14mph Gusts, 11mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.09 μm/s
QUICK SUMMARY: Work on our IFO alignment issues have continued today; Ryan has just finished an initial alignment and Sheila is supporting remotely.

Wind is low and the last EQ has mostly passed
All other systems appear nominal

H1 General - Comment to OPS Saturday day shift update

ryan.crouch@ligo.org — Sat, 13 Jul 2024 13:54:25 -0700

Author: ryan.crouch@ligo.org

Report ID: 79098

The loops are unstable and keep killing DRMI, SRM is shaking the most.

Images attached to this report

H1 General - Comment to OPS Saturday day shift update

ryan.crouch@ligo.org — Sat, 13 Jul 2024 13:32:18 -0700

Author: ryan.crouch@ligo.org

Report ID: 79097

RyanC, Sheila D.

I got up to PREP_ASC twice and ran the shutter test each time, It looks good.

Images attached to this report

H1 General - OPS Saturday day shift update

ryan.crouch@ligo.org — Sat, 13 Jul 2024 12:02:25 -0700

Author: ryan.crouch@ligo.org

Report ID: 79096

STATE of H1: Corrective Maintenance

We're getting shaken by an earthquake right now, once it calms down I'm trying to lock DRMI in PREP_ASC to check on the AS_SHUTTER. My IA efforts seemed to have helped as AS AIR doesn't look so strange anymore and DRMI flashes looked good. I'm going to try to keep moving up the locking sequence.

H1 General - Comment to OPS Saturday day shift start

ryan.crouch@ligo.org — Sat, 13 Jul 2024 08:59:48 -0700

Author: ryan.crouch@ligo.org

Report ID: 79094

I'm trying to get up to CHECK_AS_SHUTTERS to run the tests, PRMI can lock ok but DRMI struggles. I'm starting an IA at 16:00 UTC, mostly to try to help SRC which is definetly misaligned from DRMIs AS AIR.

LHO VE - Sat CP1 Fill

david.barker@ligo.org — Sat, 13 Jul 2024 08:15:54 -0700

Author: david.barker@ligo.org

Report ID: 79093

Sat Jul 13 08:09:15 2024 INFO: Fill completed in 9min 11secs

Images attached to this report

H1 SYS - AS shutter really wasn't working normally last night

sheila.dwyer@ligo.org — Sat, 13 Jul 2024 08:12:43 -0700

Author: sheila.dwyer@ligo.org

Report ID: 79092

The first attached screenshot shows a time when the fast shutter was tested on July 11th, and seemed to be working fine and passed the tests. In that case, the light on AS_A and AS_B was blocked for almost 10 seconds during the test.

The second screenshot shows one of the many times that we tried to manually hit the run test button last night while sitting in ISC_LOCK state 120 (check as shutter). The light on AS_A and AS_B were only blocked for 10ms.

So, this wasn't a false alarm, something about the shutter really wasn't working correctly last night.

I've placed an ndscope template people can use to check on the fast shutter in /ligo/home/sheila.dwyer/ndscope/SYS/check_fast_shutter.yam The fast shutter full tests only run if it has been 48 hours since it last ran, so I think that Ryan should rerun the test manually this morning and check this template to convince ourselves that the shutter is really working now, before we continue with locking.

Images attached to this report

H1 General - OPS Saturday day shift start

ryan.crouch@ligo.org — Sat, 13 Jul 2024 07:39:27 -0700

Author: ryan.crouch@ligo.org

Report ID: 79091

TITLE: 07/13 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 6mph Gusts, 4mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.08 μm/s
QUICK SUMMARY:

We've been down since last night due various issues

LHO General - Ops Eve Shift Summary

ryan.short@ligo.org — Sat, 13 Jul 2024 01:12:31 -0700

Author: ryan.short@ligo.org

Report ID: 79090

TITLE: 07/13 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
SHIFT SUMMARY: H1 has been down since the lockloss at 16:44 UTC earlier today.

Since midshift, Sheila was working with me on diagnosing locking issues. After locking DRMI, we turned on the DRMI ASC loops one-by-one until we found that turning on the SRC1_P&Y loops caused the buildups and camera image on AS AIR to get worse, so we've commented out using those loops in the ISC_DRMI Guardian. Locking DRMI after that and running DRMI ASC all at once made DRMI look much better and we didn't see the oscillations and saturations that I was seeing earlier. We also commented out the use of SRC1 in ENGAGE_AS_FOR_FULL_IFO since Sheila and Keita saw earlier in the day that it was not minimizing POP90 well at that point.

Once we made it past DRMI, we then encountered our next issue at CHECK_AS_SHUTTERS. The FAST_SHUTTER Guardian ran through its shutter test, then jumped to SHUTTER_FAILURE where it reported "Fast shutter failed tests! Do not power up!" and the AS port protection screen shows the error messages "Protection in fault" and "Power interlock is on". Unsure of what these errors meant, and having been unsuccessful in running the fast shutter test again manually, I called Fil for support. He had me power cycle the fast shutter driver chassis in the ISC rack by HAM6, which showed gibberish on the LCD screen (a previously seen issue), and the screen looked better and showed 255V DC charge. This didn't seem to fix the issue of the test failing, so I also power cycled the shutter logic chassis at the bottom of the rack, but this also didn't solve it.

Later, Keita discovered that the shutter test was failing because when the shutter closes, the signal on ASC-AS_B_DC_NSUM was not low enough, so Guardian didn't think it was closed even though it was. By just pressing the open and close buttons manually on the fast shutter screen, the shutter was able to run through its test and the Beckhoff errors cleared. Nothing was fundamentally changed here to fix this, aside from adding some log statements to the FAST_SHUTTER Guardian, but at least it sounds less like an electronics issue at this point. Keita commented that this perhaps isn't an alignment issue, unless some alignment in HAM6 changed recently.

Even though the issue with the fast shutter seems to be dealt with (but untested during a lock acquisition), there are still IFO locking issues that Sheila and I were unable to fully address this evening due to the fast shutter issues, including the 0.5Hz oscillation and lockloss when going to full power and the strange image on the AS AIR camera, which will need to be addressed in the morning. I'm leaving H1 DOWN for the night until more thorough investigations can be done.

ISC, OpsInfo, SYS

H1 CAL - Comment to testing patched simulines version during next calibration measurement

erik.vonreis@ligo.org — Fri, 12 Jul 2024 22:48:59 -0700

Author: erik.vonreis@ligo.org

Report ID: 79089

The run was probably in done in the 'cds' environment. At LHO, 'cds' and 'cds-testing' are currently identical. I don't know the situation at LLO, but LLO typically runs with an older environment than LHO.

Since it's hard to stay with fixed versions on conda-forge, it's likely several packages are newer at LHO vs. LLO cds environments.

H1 ISC - Comment to Testing CPU Turn Around Time for OMC DCPD 524 kHz IOP model :: Unused High Frequency Notches Removed to Save Computation Time

erik.vonreis@ligo.org — Fri, 12 Jul 2024 22:43:30 -0700

Author: erik.vonreis@ligo.org

Report ID: 79088

Variation of CPU time in that range shouldn't by itself have any effect on the control loops running on that model until they get to a sustained time above 15 us or an individual cycle time somewhat more than 15 us depending on the model.

The effects of a model that's run too long are DAC buffer starvation, i.e. the IOP didn't keep up with the DAC clocks, or IPC communication between models arriving to late.

Both of these errors would appear immediately on the CDS overview MEDM.

H1 SYS - fast shutter did shut in our last lockloss

sheila.dwyer@ligo.org — Fri, 12 Jul 2024 22:25:24 -0700

Author: sheila.dwyer@ligo.org

Report ID: 79087

Ryan S and Filiberto remotely are troubleshooting why the fast shutter isn't working now. I just had a look at our last high power lockloss, from MAX power at around 6:45 local time. In that lockloss the fast shutter did function and protect the AS port.

We've had a strange looking AS air camera all afternoon, screenshot of the camera with DRMI locked is attached. Hopefully the shutter issue is just the controller needing to be reset. In the past when the shutter wires were in the beam path towards OM1, the AS camera image did look strange.

Images attached to this report

H1 General - Ops Eve Mid Shift Report

ryan.short@ligo.org — Fri, 12 Jul 2024 20:39:13 -0700

Author: ryan.short@ligo.org

Report ID: 79086

H1 has been unable to relock so far this shift. After running an initial alignment at the start of the evening, we were able to reach MAX_POWER fairly consistently before a 0.5Hz oscillation started in ASC MICH_P and also smaller in DHARD_P and some SRC_P which would cause a lockloss. After one of these locklosses, when relocking DRMI, there would be many SRM and SR2 saturation callouts after DRMI ASC turned on and DRMI would lose lock. I dropped back down to run another initial alignment, paying closer attention this time, and everything went smoothly aside from SRC alignment, where no matter what SRM and SR2 moves I made, we could not acquire SRY. Trending OSEMs, optics in the SRC are in generally the same place as they are when locking normally, so I'm unsure at this point where the misalignment is (and I'm hesitant to move SR3 at this stage). The AS AIR camera has looked quite bad the whole time as well.

Starting the locking sequence to see what things look like now.

H1 General - OPS Friday day shift summary

ryan.crouch@ligo.org — Fri, 12 Jul 2024 16:33:00 -0700

Author: ryan.crouch@ligo.org

Report ID: 79076

TITLE: 07/12 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY: Currently finishing up with a IA to align SRM.

Lock1:

15:32 to 16:01 UTC dropped Observing to do some range checks alog79069
16:44 UTC lockloss alog79073

Lock2:

18:07 UTC lost it at INJECT_SQUEEZING, we were getting shaken by an earthquake at the time, it looks like MICH rang up?
We kept losing it at PREP_DC_READOUT, while the OMC was trying to lock, at different points of the OMC locking sequence. The ASC signals go crazy, maybe the SRM alignment isn't good enough?
- Tried turning off the DARM offset once we got to PREP_DC
- Sitting in SOFT_LOOPS we turned off ASC and adjusted SRM, lowered the DARM OFFSET, then thesholds were lowered like OMC-DCPD_SUM_OUTPUT from 10 to 7 which got it to lock alog79082
- 22:49 UTC lockloss at TRANSITION_FROM_ETMX
- AS AIR looked much worse now, after losing it at DRMI we dropped to start an initial alignment at 23:12 UTC, skipping green arms
- We had to align SRM by hand as it was too badly misaligned for the GRD
20:15 UTC erroneous signal from the HWUT seismometer on picket-fence

LOG:

Start Time	System	Name	Location	Lazer_Haz	Task	Time End
18:30	ASC	Keita	HAM6	N	Install ADC ASC component	18:36

LHO General - Ops Eve Shift Start

ryan.short@ligo.org — Fri, 12 Jul 2024 16:06:52 -0700

Author: ryan.short@ligo.org

Report ID: 79084

TITLE: 07/12 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 11mph Gusts, 8mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.08 μm/s
QUICK SUMMARY: Sounds like H1 has been having some locking troubles today, which I'll be aware of as we relock. Currently H1 is in the process of locking PRMI.

Wind and microseism low
Just dipped below 100degF on-site
All other systems nominal

X1 SUS - BBSS New Transfer Functions with Temperature Sag Offsets Applied

ibrahim.abouelfettouh@ligo.org — Fri, 12 Jul 2024 15:57:39 -0700

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 79083

Ibrahim

Attached are the new set of BBSS TF measurement screenshots with slightly adjusted offsets applied. These offsets account for the Pitch drift we're seeing in F1 and for any other non-centered OSEM behavior due to temperature sagging. These are also the first TFs since moving the EQ stops back and re-confirming that nothing is rubbing or touching. See alog 79079 for the summary of what was done right before these were taken. Screenshot 1 shows the new offsets that are currently in. Screenshot 2 shows the old offsets that were from the OLV script - if the temperature stops varying to this degree, it may be worth reverting these.

The red trace are results as of today (07-12-2024)

The green results are results from earlier this week 07-10-2024, which were presented in alog 79032 had the issues mentioned in alog 79042 and alog 79036.

The blue results are the results from 01-05-2024, which we were using as a reference for expected coherence.

The new measurements are saved under a new file in the Data folder: /ligo/svncommon/SusSVN/sus/trunk/BBSS/X1/BS/SAGM1/Data/2024-07-12_1400_tfs. I've svn commited this Data file so it should be visible both in the X1 and normal workstations.

Interpretations:

The new transfer functions look much better than the ones from two days ago, meaning that moving the EQ stops back and adjusting for the Temp Driven sag was extremely fruitful. We can clearly see cleaner results, with almost eliminated noise in the previously problematic 1.5-6hz region and the results are markedly more coherent, comprable to the pre-rebuild state from 01-05. Now we can begin interpreting these results seriously with respect to the dynamic model, which I will leave for next week.

Next Steps (Next Week):

Produce the BBSS Model-Result overlay comparison and look for cross coupling, unexpected resonances and other oddities.
Make decisions about how to change the mechanical build or the parametric models for the BBSS
Fix the M3 roll
Discuss, then fix the M1 Pitch issue if it is still there an apparent
If there's time, recenter OSEMs mechanically and set offsets back to their OLV values (since weather is going to be reliably around 100F all next week at the same times)
Place a thermometer/temperature sensor in the staging building to track temp drift more accurately

Images attached to this report

H1 ISC - DARM offset causing locklosses again

sheila.dwyer@ligo.org — Fri, 12 Jul 2024 15:36:06 -0700

Author: sheila.dwyer@ligo.org

Report ID: 79082

Keita, Sheila, Ryan C, Ibrahim

Two previous alogs about this problem: 78332 78258

We tried to open the SRC1 loop and align SRM manually, we were able to decrease POP90 that way but when we again turned on the DARM offset we saw the same drop in powers that preceeds locklosses. (see screenshot, when things recover we manually turned off the DARM offset.)

We were able to move past this for today by lowering the DARM offset used from 9e-5 to 6e-5 in ISC_LOCK DARM_OFFSET. Then to get the OMC guardian to lock we had to lower the threshold on the peak height in OMC_LOCK FIND_CARRIER line 328 to 7. Then we had to lower the check in OMC_LOCKED from 10 to 7, OMC_LOCK line 512.

We manually reengaged the SRC1 ASC loops, and are now powering up.

Images attached to this report

H1 CAL - PCALX :: Long-term High-Frequency Sensing Function TF vs. PCAL Spot Position Systematics Study

jeffrey.kissel@ligo.org — Fri, 12 Jul 2024 15:27:26 -0700

Author: jeffrey.kissel@ligo.org

Report ID: 79081

J. Kissel, L. Dartez, F. Llamas As Louis and I were wracking our brains for all the changes to the IFO that might have impacted the IFO's calibration in 2024 in order to update / maintain T2300297, we realized that Francisco's work -- changing the PCAL "inner beam" spot position of PCALX to explore and validate models of systematic errors the PCAL system -- would potentially impact the measurements that are constantly running on PCALX -- the so-called "roaming lines" which is a really long duration, repeating sweep of the sensing function between 1 and 5 kHz. Because this long-duration sweep relies on nominal low noise data, the speed at which the sweep frequency advances is dependent on the IFO's duty cycle, and is thus irregular; sometimes complete in 6 days, other times taking 10 days or more. To facilitate this research, I plot the trend of the PCALX Oscillator Frequency that's used to define the sweep frequency over time and overlay Francisco, et. al's PCALX spot moves. And just so you can join us in our worry -- while Francisco's work (see latest in LHO:78964) is showing 0.03%-level changes in the comparison between PCALX and PCALY at 283.91 and 284.01 Hz respectively (via plots like this), the roaming lines measurement is only PCALX and the sweep is from 1-5 kHz, where it's been known for a long time that PCAL spot position changes of ~5 [mm] at RX, or ~2.5 [mm] scale at the Test Mass, can cause 1-to-10%-level errors in reported displacement values at 3, 4, 5 kHz (largest at highest frequencies) -- see, for example, figures 2.30 through 2.35 in T2300381. Stay tuned! This aLOG does not suggest we've done the leg work to quantify that we've seen the a problem, but the start of doing such a study.

Images attached to this report

H1 ISC - Comment to DARM Offset step with hot OM2

jennifer.wright@ligo.org — Fri, 12 Jul 2024 14:25:23 -0700

Author: jennifer.wright@ligo.org

Report ID: 79080

Here is the calculated Optical gain vs dcpd power and DARM offset vs optical gain as calculated by ligo/gitcommon/labutils/darm_offset_step/plot_darm_optical_gain_vs_dcpd_sum.py

The contrast defect is calculated from the height of the 410Hz PCAL line at each offset step in the output DCPD, and is 1.014 +/- 0.033 mW.

Non-image files attached to this report

all_plots_plot_darm_optical_gain_vs_dcpd_sum_1404768832_2_5_hours_into_lock__hot_OM2.pdf

X1 SUS - BBSS Post-TF Diagnostic Check-up

ibrahim.abouelfettouh@ligo.org — Fri, 12 Jul 2024 13:17:00 -0700

Author: ibrahim.abouelfettouh@ligo.org

Report ID: 79079

Ibrahim

Today, I went into the staging building to see if there were any visible or otherwise apparent reasons for our L, P, Y M1 Transfer Function incoherence.

Here's what I did:

Backed off all EQ stops, none were too close to touching, but the temp drif sag was higher yesterday than today so I just made sure they were all the way back.
Re-did the foil, double triple checking nothing was touching, rubbing or in contact
Checked every BOSEM Flag

Here's what I found:

M1 is visibly pitched - this is corroborated by the 8-day drift of the F1 OSEM counts (as opposed to the daily sag and rebound of the LF-RT OSEMs. I've copied over the F1 section of the drift screenshot from yesterday's alog 79042 on this topic, since it seems the pitch instability suggestion from that alog was the right hypotehesis (F1 is used in P and is above the M1 center of mass). Image 1 shoes this clearly.
- This could either be differential sagging of the Top Stage blades causing it to pitch in one direction over another.
- It could also be a pitch hysteresis issue we had with the Top Mass during assembly - this was fixed by setting the heights of the blades to a different nominal position.
- It could also be that the the Top Mass blades are differentially sagging too, causing more pull on one side than the other.
- Or a combo, or something else.
The M1 flags are surprisingly not visibly sagging that much - I guess this makes sense since 25 microns (our measured offset from centered position last week) is only 0.025mm.
The M2 flags are much more visibly sagging, which means that the Top Mass blades are more affected by the temperature drift. This sort of corroborates the idea that the Top Mass blade sag is affecting the pitch moreso than the top stage blades or a previous ptich hysteresis blade height problem. Image 2 shoes one of the 4 M2 BOSEMs substantially lower than their centered positon.

Here's what I'm going to to:

I'm going to set an offset in order to "fake" center the BOSEMs and then take data but I expect that with a Pitch instability the results won't be too coherent either. We'll see. We're at least now ensured that there won't be any rubbing or contact.

What I'm going to do later:

We will fix the percieveable Roll issue on Monday when there are more hands on deck. I'll also consider re-centering the OSEMs at noon and measuring the temp at that time so we have some sort of informed reference.

Images attached to this report

H1 General - Comment to Lockloss at 16:44 UTC

ryan.crouch@ligo.org — Fri, 12 Jul 2024 12:55:15 -0700

Author: ryan.crouch@ligo.org

Report ID: 79078

We've lost it at PREP_DC_READOUT twice in a row, during different points of the OMC locking process. Lockloss tool tags ADS_EXCURSION

H1 AOS - Comment to We cannot assess the energy deposited in HAM6 during pressure spike incidents (yet)

keita.kawabe@ligo.org — Fri, 12 Jul 2024 11:49:53 -0700

Author: keita.kawabe@ligo.org

Report ID: 79077

Unfortunately, I forgot that the input range of some of these PEM ADCs are +-2V, and so the signal still railed when the analog output of ASC-AS_SUM didn't (2V happens to be the trigger threshold of the fast shutter), so this was still not good enough.

I installed 1/11 resistive divider (nominally 909Ohm - 9.1k) on the output of the ASC-AS_C analog SUM output on the chassis (not on the input of the PEM patch panel) at around 18:30 UTC on Jul/12 2024 while IFO was out of lock.

H1 PSL - PSL Status Report - Weekly

ryan.crouch@ligo.org — Fri, 12 Jul 2024 10:58:05 -0700

Author: ryan.crouch@ligo.org

Report ID: 79075

Laser Status:
    NPRO output power is 1.821W (nominal ~2W)
    AMP1 output power is 65.27W (nominal ~70W)
    AMP2 output power is 138.5W (nominal 135-140W)
    NPRO watchdog is GREEN
    AMP1 watchdog is GREEN
    AMP2 watchdog is GREEN
    PDWD watchdog is GREEN

PMC:
    It has been locked 2 days, 23 hr 37 minutes
    Reflected power = 17.94W
    Transmitted power = 107.8W
    PowerSum = 125.8W

FSS:
It has been locked for 0 days 0 hr and 49 min
TPD[V] = 0.7949V

ISS:
The diffracted power is around 3.2%
Last saturation event was 0 days 0 hours and 49 minutes ago

Possible Issues:
ISS diffracted power is high, seems like its been high since early morning this past maintenance.

H1 ISC - Comment to Range checks

sheila.dwyer@ligo.org — Fri, 12 Jul 2024 10:35:33 -0700

Author: sheila.dwyer@ligo.org

Report ID: 79072

Attached is a spectrum comparison for a 159 Mpc time from last night to 149 Mpc this morning. The issue is between 30-70 Hz, where Ryan's coherence plot doesn't show much.

The second attachment shows that today's poor range time is similar to what happened after Tuesday maintenance (78954).

We tried the no squeezing time because we've seen in the past that the squeezer was adding noise around this frequency range, 78033 77969 77888 . In these past times were at times when we had moved the spot position on PR2, and we also had the intermittent squeezing problems that we seem not to have this week. We reverted PR2 spot moves because of this suspicion (77895 78012) and the issue seemed to go away, but we thought it might be simply that the intermittent squeezer issue happened to get better.

The third attached screenshot shows a similar trends for the times when we moved PR3 in May to move the spot on PR2 (77949), that was a ~3 times larger move than the one we did last week (78878).

Images attached to this report

H1 ISC - Comment to Range checks

keita.kawabe@ligo.org — Fri, 12 Jul 2024 10:19:02 -0700

Author: keita.kawabe@ligo.org

Report ID: 79074

Quickly checked temperature.

LVEA diurnal temperature swing is larger than usual in the past 4 days or so (e.g. zone 1B 0.33 Celsius pp instead of 0.15) but I don't see correlation with the range drop.

Images attached to this report

H1 General - Lockloss at 16:44 UTC

ryan.crouch@ligo.org — Fri, 12 Jul 2024 09:52:25 -0700

Author: ryan.crouch@ligo.org

Report ID: 79073

16:44 UTC lockloss. PRCL was oscillating at about ~3.6 Hz.

Lockloss

H1 ISC - Range checks

ryan.crouch@ligo.org — Fri, 12 Jul 2024 09:43:04 -0700

Author: ryan.crouch@ligo.org

Report ID: 79069

Ryan C, Sheila D

15:32 to 16:01 UTC we dropped Observing to do some range checks/investigations.

Sheila and I did some range checks following the wiki. Running the coherence check showed high coherence with CHARD and the SQZer BLRMs did not look as good as previous locks, particularly in the 10-20, 20-34, and 60-100 HZ bandwidths. So we decided to drop out of observing since LLO was down to run the SQZ alignment and angle scans and I concurrently ran the A2L_min script (TJs alog78552). After these finished we also took 5 minutes of NO_SQZ time starting at 15:49 UTC which showed that the extra noise is not coming from the SQZer. We gained a few Mpcs from the SQZ scan and A2L_min script.

Coherence comparison before and after SQZ and A2L checks

Images attached to this report

H1 ISC - Comment to Testing CPU Turn Around Time for OMC DCPD 524 kHz IOP model :: Unused High Frequency Notches Removed to Save Computation Time

sheila.dwyer@ligo.org — Fri, 12 Jul 2024 09:42:42 -0700

Author: sheila.dwyer@ligo.org

Report ID: 79071

Keita, Sheila

We were looking for explanations for the drops in range we've seen since Tuesday. Attached is a plot of the CPU meter, it seems that this jumped up shortly after Jeff's plot was made. It is still below 13usec, and doesn't look correlated with our range problems.

Images attached to this report

H1 SEI - Comment to Earthquake Analysis : Similar onsite wave velocities may or may not cause lockloss, why?

neil.doerksen@ligo.org — Fri, 12 Jul 2024 09:14:59 -0700

Author: neil.doerksen@ligo.org

Report ID: 79070

One of the plots (2nd row, 2nd column) has the incorrect x-channel on some of the images (all posted images are correct, by chance). Patterns reported may not be correct, will reanalyze.

SEI

H1 ISC - Comment to Ran A2L P & Y

ryan.crouch@ligo.org — Fri, 12 Jul 2024 09:03:01 -0700

Author: ryan.crouch@ligo.org

Report ID: 79068

SDFed

Images attached to this report

H1 ISC - Ran A2L P & Y

ryan.crouch@ligo.org — Fri, 12 Jul 2024 08:47:57 -0700

Author: ryan.crouch@ligo.org

Report ID: 79066

Ran the (userapps)/isc/h1/scripts/a2l/a2l_min_multi.py script for all four quads in both P and Y to try and help our range. Minimal improvements, this was done in tandem with the SQZ scans which gained us about 2 Mpc.

ETMX P
Initial: 3.12
Final: 3.09
Diff: -0.03

          ETMX Y
Initial: 4.79
Final:    4.81
Diff:     0.02

          ETMY P
Initial: 4.48
Final:    4.49
Diff:     0.01

          ETMY Y
Initial: 1.13
Final:    1.26
Diff:     0.13

          ITMX P
Initial: -1.07
Final:    -1.02
Diff:     0.05

          ITMX Y
Initial: 2.72
Final:    2.79
Diff:     0.07

          ITMY P
Initial: -0.47
Final:    -0.43
Diff:     0.04

          ITMY Y
Initial: -2.3
Final:    -2.36
Diff:     -0.06

H1 CAL - Comment to testing patched simulines version during next calibration measurement

louis.dartez@ligo.org — Fri, 12 Jul 2024 08:34:11 -0700

Author: louis.dartez@ligo.org

Report ID: 79065

right, there was no expectation of this dataset being valid. the IFO was not thermalized and the cal lines remained on. The goal of this exercise was to demonstrate that the patched simulines version at LHO can successfully drive calibration measurements. And to that end the exercise was successful. LHO has recovered simulines functionality and we can lay to rest the scary notion of regressing back to our 3hr-long measurement scheme for now.

H1 General - Comment to OPS Monday day shift start

ryan.crouch@ligo.org — Fri, 12 Jul 2024 08:22:33 -0700

Author: ryan.crouch@ligo.org

Report ID: 79064

Running the coherence low range check, CHARD_P,Y and MICH seem to have high coherence.

Images attached to this report

LHO VE - Fri CP1 Fill

david.barker@ligo.org — Fri, 12 Jul 2024 08:11:16 -0700

Author: david.barker@ligo.org

Report ID: 79063

Fri Jul 12 08:08:03 2024 INFO: Fill completed in 8min 0secs

Jordan confirmed a good fill curbside.

Images attached to this report

H1 CAL - Comment to testing patched simulines version during next calibration measurement

vladimir.bossilkov@ligo.org — Fri, 12 Jul 2024 08:06:03 -0700

Author: vladimir.bossilkov@ligo.org

Report ID: 79062

There's a number of WAY out there data points in this report.

Did you guys also forget to turn off the calibration lines when you ran it?

Not marking this report as valid.

H1 CAL - Comment to testing patched simulines version during next calibration measurement

ryan.crouch@ligo.org — Fri, 12 Jul 2024 08:00:38 -0700

Author: ryan.crouch@ligo.org

Report ID: 79061

Attaching the calibration report

Non-image files attached to this report

H1_calibration_report_20240711T234232Z.pdf

H1 General - OPS Friday day shift start

ryan.crouch@ligo.org — Fri, 12 Jul 2024 07:37:44 -0700

Author: ryan.crouch@ligo.org

Report ID: 79060

TITLE: 07/12 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 147Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 5mph Gusts, 3mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.08 μm/s
QUICK SUMMARY:

We've been locked and observing for 4:20, since 10:17 UTC
- The lockloss overnight (08:24 UTC) was from an earthquake
Lots of earthquakes overnight
Range is ok, SQZing doesn't look the best from the trace on NUC33

Lockloss

H1 CAL - Comment to testing patched simulines version during next calibration measurement

vladimir.bossilkov@ligo.org — Fri, 12 Jul 2024 07:33:54 -0700

Author: vladimir.bossilkov@ligo.org

Report ID: 79059

Here's what I did:

Cloned simulines in my home directory
Copied the currently used ini file to that directory, overwriting default file [cp /ligo/groups/cal/src/simulines/simulines/newDARM_20231221/settings_h1_newDARM_scaled_by_drivealign_20231221_factor_p1.ini /ligo/home/vladimir.bossilkov/gitProjects/simulines/simulines/settings_h1.ini]
reran simulines on the log file [./simuLines.py -i /opt/rtcds/userapps/release/cal/common/scripts/simuLines/logs/H1/20240711T234232Z.log]

No special environment was used. Output:
2024-07-12 14:28:43,692 | WARNING | It is assumed you are parising a log file. Reconstruction of hdf5 files will use current INI file. 2024-07-12 14:28:43,692 | WARNING | If you used a different INI file for the injection you are reconstructing, you need to replace the default INI file. 2024-07-12 14:28:43,692 | WARNING | Fetching data more than a couple of months old might try to fetch from tape. Please use the NDS2_CLIENT_ALLOW_DATA_ON_TAPE=1 environment variable. 2024-07-12 14:28:43,692 | INFO | If you alter the scan parameters (ramp times, cycles run, min seconds per scan, averages), rerun the INI settings generator. DO NOT hand modify the ini file. 2024-07-12 14:28:43,693 | INFO | Parsing Log file for injection start and end timestamps 2024-07-12 14:28:43,701 | INFO | Commencing data processing. 2024-07-12 14:28:55,745 | INFO | File written out to: /ligo/groups/cal/H1/measurements/DARMOLG_SS/DARMOLG_SS_20240711T234232Z.hdf5 2024-07-12 14:29:11,685 | INFO | File written out to: /ligo/groups/cal/H1/measurements/PCALY2DARM_SS/PCALY2DARM_SS_20240711T234232Z.hdf5 2024-07-12 14:29:20,343 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L1_SS/SUSETMX_L1_SS_20240711T234232Z.hdf5 2024-07-12 14:29:29,541 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L2_SS/SUSETMX_L2_SS_20240711T234232Z.hdf5 2024-07-12 14:29:38,634 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L3_SS/SUSETMX_L3_SS_20240711T234232Z.hdf5

Seems good to me. Were you guys accidentally using some conda environment when running simulines yesterday? When running this I was in " cds-testing " (which is the default?!). I have had this error in the past due to borked environments [in particular scipy which is the underlying responsible code for coherence], which is why I implemented the log parsing function.
The fact that the crash was on coherence and not the preceding transfer function calculation rings the alarm bell that scipy is the issue. We experienced this once in LLO with a single bad conda environment that was corrected, though I stubbornly religiously ran with a very old environment for a long time to make sure that error doesn't come up,

I ran this remotely so can't look at PDF if i run 'pydarm report'.
I'll be in touch over teamspeak to get that resolved.

H1 General - Ops Eve Shift End

oli.patane@ligo.org — Fri, 12 Jul 2024 01:09:06 -0700

Author: oli.patane@ligo.org

Report ID: 79058

TITLE: 07/12 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Observing at 148Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY: Mostly uneventful evening, but after the lockloss while trying to relock (hadn't done an IA), DRMI unlocked twice while we were in ENGAGE_DRMI_ASC, and got beamsplitter saturations both times, with the second time also getting a BS ISI saturation (1st time ndscope, 2nd time ndscope). Not sure what that was about, as I ran an IA and when I tried locking again it was fine. After being in Observing for a little bit I decided that I wanted to adjust sqz because it was really bad, and I had some issues after running it the first time (clicked things out of order even though it shouldn't matter??), but eventually I was able to get a lot more squeezing and a much better sensitivity at the higher frequencies.
LOG:

23:00 Relocking and at PARK_ALS_VCO
23:35 NOMINAL_LOW_NOISE
23:42 Started running simulines measurement to check if simulines is working
00:05 Simulines done
00:13 Observing

00:44 Earthquake mode activated due to EQ in El Salvador
01:04 Seismic to CALM

04:32 Lockloss
Relocking
- 17 seconds into ENGAGE_DRMI_ASC, BS saturated and then LL
- BS saturation twice in ENGAGE_DRMI_ASC, then ISI BS saturation, then LL
05:08 Started an initial alignment
05:27 IA done, relocking
06:21 NOMINAL_LOW_NOISE
06:23 Observing

06:38 Out of Observing to try and make sqz better because it's really bad
07:24 Observing

Start Time	System	Name	Location	Lazer_Haz	Task	Time End
23:10	PCAL	Rick, Shango, Dan	PCAL Lab	y(local)	In PCAL Lab	23:54

Images attached to this report

H1 General - Comment to Lockloss

oli.patane@ligo.org — Thu, 11 Jul 2024 23:24:13 -0700

Author: oli.patane@ligo.org

Report ID: 79057

06:23 UTC Observing

X1 SUS - Comment to BBSS Transfer Functions and First Look Observations

oli.patane@ligo.org — Thu, 11 Jul 2024 21:56:01 -0700

Author: oli.patane@ligo.org

Report ID: 79055

We have also now made a transfer function comparison between the dynamical model, the first build (2024/01/05), and following the recent rebuild (2024/07/10). These plots were generated by running $(sussvn)/trunk/BBSS/Common/MatlabTools/plotallbbss_tfs_M1.m for cases 1 and 3 in the table. I've attached the results as a pdf, but the .fig files can also be found in the results directory, $(sussvn)/trunk/BBSS/Common/Results/allbbss_2024-Jan05vJuly10_X1SUSBS_M1/. These results have been committed to svn.

Non-image files attached to this report

allbbss_2024-Jan05vJuly10_X1SUSBS_M1_ALL_TFs.pdf

H1 General - Lockloss

oli.patane@ligo.org — Thu, 11 Jul 2024 21:33:55 -0700

Author: oli.patane@ligo.org

Report ID: 79056

Lockloss @ 07/12 04:32 UTC

Lockloss

LHO VE - Thu CP1 Fill

david.barker@ligo.org — Thu, 11 Jul 2024 20:23:12 -0700

Author: david.barker@ligo.org

Report ID: 79054

Thu Jul 11 08:11:57 2024 INFO: Fill completed in 11min 53secs

late entry from this morning

Images attached to this report

H1 General - Ops Eve Midshift Status

oli.patane@ligo.org — Thu, 11 Jul 2024 20:06:56 -0700

Author: oli.patane@ligo.org

Report ID: 79053

Observing at 157Mpc and have been locked for 3.5 hours. We rode out a 5.2 earthquake from El Salvador earlier. Wind is low and going down.

H1 CDS - Comment to SUSH2A

erik.vonreis@ligo.org — Thu, 11 Jul 2024 18:37:14 -0700

Author: erik.vonreis@ligo.org

Report ID: 79052

The upper bound on sush2a latency for the first three months of O4B is 39 IOP
cycles. At 2^16 cycles per second, that's a maximum of 595
microseconds.

At 1 kHz that's 214 degrees of phase shift.

Normal latency is 3 IOP cycles, 46 microseconds, 16 degrees phase shift
@ 1 kHz.

The minimum latency when sush2a was in error was 4 cycles, 61
microseconds, 23 deg @ 1 KHz.

H1 General - Comment to Lock loss 2154 UTC

oli.patane@ligo.org — Thu, 11 Jul 2024 17:26:22 -0700

Author: oli.patane@ligo.org

Report ID: 79051

07/12 00:13 UTC Observing

There were changes to the PCAL ramp times(PCALX, PCALY) made at 21:48 UTC. At that time we were locked and commissioning.

I have reverted those changes.

CAL

Images attached to this report

H1 CAL - Comment to testing patched simulines version during next calibration measurement

erik.vonreis@ligo.org — Thu, 11 Jul 2024 17:22:57 -0700

Author: erik.vonreis@ligo.org

Report ID: 79050

Also applied the following patch to simuLines.py before the run. The purpose being to extend the sine definition so that discontinuities don't happen if a stop command is executed late. If stop commands are all executed on time (the expected behavior), then this change will have no effect.

diff --git a/simuLines.py b/simuLines.py index 6925cb5..cd2ccc3 100755 --- a/simuLines.py +++ b/simuLines.py @@ -468,7 +468,7 @@ def SignalInjection(resultobj, freqAmp): #TODO: does this command take time to send, that is needed to add to timeWindowStart and fullDuration? #Testing: Yes. Some fraction of a second. adding 0.1 seconds to assure smooth rampDown - drive = awg.Sine(chan = exc_channel, freq = frequency, ampl = amp, duration = fullDuration + rampUp + rampDown + settleTime + 1) + drive = awg.Sine(chan = exc_channel, freq = frequency, ampl = amp, duration = fullDuration + rampUp + rampDown + settleTime + 10) def signal_handler(signal, frame): '''

H1 CAL - Comment to testing patched simulines version during next calibration measurement

erik.vonreis@ligo.org — Thu, 11 Jul 2024 17:18:25 -0700

Author: erik.vonreis@ligo.org

Report ID: 79049

All five excitations looked good on ndscope during the run.

H1 CAL - Comment to testing patched simulines version during next calibration measurement

oli.patane@ligo.org — Thu, 11 Jul 2024 17:16:00 -0700

Author: oli.patane@ligo.org

Report ID: 79048

Simulines was run right after getting back to NOIMINAL_LOW_NOISE. Script ran all the way until after Commencing data processing, where it then gave:

Traceback (most recent call last): File "/ligo/groups/cal/src/simulines/simulines/simuLines.py", line 712, in run(args.inputFile, args.outPath, args.record) File "/ligo/groups/cal/src/simulines/simulines/simuLines.py", line 205, in run digestedObj[scan] = digestData(results[scan], data) File "/ligo/groups/cal/src/simulines/simulines/simuLines.py", line 621, in digestData coh = np.float64( cohArray[index] ) File "/var/opt/conda/base/envs/cds/lib/python3.10/site-packages/gwpy/types/series.py", line 609, in __getitem__ new = super().__getitem__(item) File "/var/opt/conda/base/envs/cds/lib/python3.10/site-packages/gwpy/types/array.py", line 199, in __getitem__ new = super().__getitem__(item) File "/var/opt/conda/base/envs/cds/lib/python3.10/site-packages/astropy/units/quantity.py", line 1302, in __getitem__ out = super().__getitem__(key) IndexError: index 3074 is out of bounds for axis 0 with size 0