Following up on the discovery that the 1.34Hz peak in DARM was injected by the SRCLFF, I implemented a more aggressive high-pass with a 1.34 Hz notch in the SRCLFF path.

The new high-pass and notch completely remove the 1.34 Hz peak from DARM, and improves the DARM noise in the 1-2 Hz region. The reduction in RMS is visible also in the time trace of the DARM error signal.

As expected, the new high-pass introduces a bit of phase rotation at 10-20 Hz, and this limits the amount of subtraction we can get now with the SRCL FF. I also tried to retune the SRCL FF with a new measurement, but I could not improve the subtraction. The reason is that I need to take a new measurement of the SRCLFF to DARM path with the new high-pass and notch. I did not have time to do it today because violin modes were damped and we needed to go back to Observe. But it should be an easy fix (10 minutes measurement + retuning + 10 minues test)

However, looking at the coherence between DARM and SRCL, I don't see a lot of it, so maybe we're fine with this reduced level of SRCL subtraction.

The new SRCLFF combination to be used is FM1 FM2 FM3 (all together). ISC-LOCK has been updated and reloaded

H1:LSC-SRCFF1 SDF Changes were Accepted after speaking with Gabriele.

ScreenShot attatched.

Randy had pointed out a few months ago there was a section of piping coming out of the endstation HEPI pumps that should probably get some extra support. While the control room was relocking today, he went and added a bit of unistrut to the stand for the revervoir, attached phot shows how he attached to the foot of the reservoir stand. This was only needed for the end station pumps, the corner pumps have a much shorter run of pipe to the next support clamp.

Daniel, Camilla 

Following on from Ryan's 71420, we looked at the movement of ITMY (only has ASC control) during DHARD_WFS and see there is now larger spikes during locking than before 29th, see attached plot where H1:SUS-ITMY_L2_OSEMINF_{U,L}{L,R}_INMON spikes during locking are larger after the t-cursor (see bottom purple plot for when violins rang up).

Zooming in, attached plot, we can see when the Input to the DHARD_Y filter is turned on that the output to DHARD_Y filter has a large transient lasting ~1second, not seen in DHARD_Y inmon. The only filter that is on during that time FM6 and although it has a larger step response, there is a 5s TRAMP so Daniel doesn't see an issue with this.

Nothing obvious happened to DHARD on the 29th, only alog around that date shows a change that was reverted 70928.

As reported in alog71528, the ETM cameras were frozen last night, and the camera guardian went to WAIT FOR CAMERA state and came back in 1 min. This process was repeated three times as shown in the attached figure.

Looking back the guardian log, the ETMX camera (PIT2, YAW2) was frozen. I also looked at the VALID channel for ETMX/ETMY cameras mentioned in alog71460 (H1:VID-CAM25_VALID for ETMX, H1:VID-CAM27_VALID for ETMY). The VALID channels of ETMX/ETMY were 1 at that time, which means that the cameras were not actually frozen. So we probably can use this VALID channel to solve the short camera freeze issue for ETMX/ETMY. Unfortunately, the BS camera does not have the VALID channel since the ETMX/ETMY cameras are running on h1digivideo3, while the BS camera is running on h1digivideo2.

We looked at some filter cavity detunings and slightly varied the generated squeezing level, to see if we could at all change the noise around 70-100 Hz. It's not entirely clear what's going on, no huge obvious changes, but perhaps subtle ones. At least, even at low frequencies < 100 Hz, it seems like higher NLG is slightly better, and that moving the FC detuning to -25 Hz is not obviously worse. To start, I just tried moving FC detuning up/down 5 Hz, and rotating sqz angle to see if anything was better, see the trends of the moves here.

From the screenshot, red + cyan are from hot om2 last time on 6/28. Other traces are from the recent heating of OM2; the recent no-sqz seems to have other injections going on at the same time. There may be a slight difference between brown FDS (before moving anything), the yellow FDS (moving FC detuning from -30 Hz to -25 Hz, and slightly increasing NLG), and the dark blue FDS (less NLG, same FC - 25Hz detuning, different sqz angle apparently).

While LLO was down, Genevieve, Lance and I worked on checking for scattered light at a few viewports and shaking at the ITMY cryobaffle in the 10 to 20 Hz region. We may have caused a lockloss when we were checking the top viewport of HAM2 on the +Y side.

First Lockloss of the Day: 18:14 UTC
Lockloss tool failed on analyze of this LockLoss. 
https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi?event=1373912078
Happened while the PEM crew were potentially working around HAM2's sensitive areas. Possibly could have been them.

Screen shot of lockloss select's ndscopes attached.

The DARM error signal RMS is dominated by the large peak at 1.34 Hz. This is coherent with SRCL. I don't know what is the origin of the peak in SRCL, but today I did a quick test by turning off MICHFF and SRCLFF.

When SRCLFF is off, the peak is still present in SRCL, but it disappears from DARM. So the SRCLFF is injecting the peak in DARM.

One way to fix it (until we figure out why SRCL is moving that much at 1.34 Hz) would be to add a notch to the SRCLFF1 path. I have a design that should not change gain or phase of the SRCLFF abvoe 10 Hz in any significant way, while notching the 1.34 Hz peak by 20 db. And also a more aggressive high pass filter, that might work without spling the SRCL subtraction at 10Hz

Derek noticed (via the summary pages) that there seems to potentially be an occasional discrepancy between CALIB_STRAIN_NOLINES and CALIB_STRAIN_CLEAN.

The NonSENS subtraction should be off.  The first attachment shows that indeed the noise estimate output by the front end NonSENS is zeros to the 1e-27 level (these are in the same strain units as GDS-CALIB, and I just didn't bother scrolling in more to see how 'zero' I can make the y-axis) during a time when the discrepancies are visible on the summary page. 

A next step is likely to try to zoom in on the time axis of the spectrograms from the summary page to see when these things are happening, and see what other things might be changing around those times (eg, are those the beginning of Observe segments or going in/out of NLN_CAL_MEAS?).  Since the output of the noise estimate from the front end is zeros, _NOLINES and _CLEAN *should* be identially the same. 

While LLO was relocking this morning, I dropped observing from 13:44 to 13:54 UTC to run Gabriele's MICH FF template with 35 averages.

The measurement is saved in /ligo/home/ryan.short/MICH_excitation_2023_07_20.xml

Last night and this morning, we've turned the OM2 heater back on.  Our range didn't improve, however.  We still had in place the AS36Q yaw offset from last Thursday (alog 71309).  At the start of our Wed commissioning period, I turned that offset off.  This improved our range back to ~140 Mpc according to CAL-DELTAL on the wall, but our range is still not back to where it was the last time we had OM2 hot. We may need to think about more alignment changes / checks.

At 18:50, we were briefly pushed out of observing when the ITMX CO2 laser lost lock, and the resulting SDF differences in H1:TCS-ITMX_CO2_CHILLER_SERVO_GAIN & H1:TCS-ITMX_CO2_PZT_SERVO_GAIN (see attached). The SDF differences disappeared on their own about a minute later as the CO2 laser locked back on, and we observed that the power in H1:TCS-ITMX_CO2_LSRPWR_HD_PD_OUTPUT went down slightly from 46.7W before it lost lock to 46.4W after it locked back on.

At 18:53 we put the detector back into Observing

We conducted a study using data from the Fscan pipeline that shows when the violin modes are rung up, there is a dramatic increase in the number of narrow lines (of the type problematic for CW searches) in the regions around the violin modes and harmonics. This increase in the number of narrow lines is visible both in daily and weekly plots since June 29th when the violin modes and its harmonics are known to have rung up. 

Figure 1 shows how the number of lines jumped significantly in the 300-600Hz and 900-1200Hz bands from June 29th to June 30th, and again from July 9th to July 10th without much change in the other frequency bands. Figures 2 and 3 show the averaged spectrum for the 29th and 30th respectively, demonstrating just how dramatic the increase in the number of lines is. Figure 4 shows how visible and persistent the lines are in the weekly data. 

Figures 5, 6, and 7 show the max amplitude of the violin modes vs number of lines per the square root number of SFTs in three different frequency bands surrounding the violin modes and its harmonics. To remove the effect of the duty cycle on the number of lines counted, the total number of lines counted is divided by the square root of the number of SFTs produced in a day because the number of SFTs produced in a day is directly proportional to the amount of time spent observing on that particular day. From here on out, I will refer to the number of lines per square root of the number of SFTs as the number of lines.

Looking at all the O4 data so far, it is obvious that the number of lines increases with the max amplitude violin modes in the bands surrounding the violin modes and its harmonics. The most dramatic effect is visible in the 300-600Hz band (Figure 5) where the number of lines seems to increase approximately linearly with the max amplitude of the violin modes between amplitudes of 10^-20-10^-18 strain/√Hz. Between 10^-18-10^-17 strain/√Hz, the number of lines increases significantly with no clear pattern. Multiple days after the June 29th ring up appear in this region, showing that these long ring ups can greatly increase the number of lines and decrease CW data quality for a significant period of time. Around the 1000Hz (Figure 6) and 1500Hz (Figure 7) harmonics, the number of lines increases approximately linearly with the max amplitude of the violin modes without displaying the jump present in the 300-600Hz band. This might be because the amplitude of the violin modes has a much smaller range of 10^-19-10^-18 strain/√Hz during the ring up. 

Lockloss after 46h25 in NLN. 1372737707.

No obvious cause, DCPD saturation tag. DARM was the first loop to change attached plot.

NOISE_CLEAN reloaded as requested in 71124.