Building off of 74289, I checked to see how the ALS-X beatnote strength was doing, and then tried to confirm that the issues were actually a result of the EX VEA temperature changes.

Currently, the beatnote strength is still oscillating, and last night it was back to dipping below -10dB when at its minimum (attachment1).

Here is what I noticed when looking to confirm the beatnote-temperature correlations:

ALS-X_FIBR_DEMOD_RFMON is very sensitive to changes in EX VEA temperatures
- Already pretty obvious based on how the temp changes from Friday affected ALS-X_FIBR_DEMOD_RFMON, but even small changes/normal daily fluctuations in EX VEA temperatures can change the beatnote strength by almost 2dB.
- Here are two stretches of time comparing the EX VEA average temperatures and how it affects ALS-X_FIBR_DEMOD_RFMON (attachment2, attachment3).
    - Sometimes the temperature affects the beatnote strength more than other times.

ALS-Y_FIBR_DEMOD_RFMON is NOT very sensitive to changes in EY VEA temperatures
- Comparing ALS-Y_FIBR_A_DEMOD_RFMON and FMC-EY_VEA_AVTEMP_DEGF(attachment4), although VEA temp changes are definitely affecting the beatnote strength (especially during large temperature excursions), the beatnote strength is able to stay very consistant.

It is especially interesting seeing the difference between the amount of variation in ALS-X_FIBR_DEMOD_RFMON as compared to ALS-Y_FIBR_DEMOD_RFMON as temperature changes (attachment5). To be fair, the temperatures at EX do seem to fluctuate more (higher highs and lower lows) than at EY, but the way that the ALS-X beatnote strength reacts still seems to me to be much more exaggerated than it should be (like if ALS-Y was going through the EX temperature changes).

Mon Nov 20 10:24:12 2023 INFO: Fill completed in 24min 7secs

Gerardo confirmed a good fill curbside.

As mentioned in LHO:74264, Sheila and I have been taking a closer look at the DARM loop and our ability to model the PUM crossover in particular in response to LHO:73937. I'm attaching a plot of the DARM OLG with the pyDARM model (blue trace), OLG measurements taken as part of the weekly calibration measurement suite (orange dots), and Sheila's measurements from Friday (LHO:74264) with the Y2L and P2L gains nominal (green circles) and with the Y2L and P2L gains reduced by a factor of two (red crosses).

There's a few interesting things going on that I'll have to elaborate on in a future alog. Briefly, the model clearly doesn't match any of our measurements below about 20Hz (zoomed out version also attached). 

More plots of just the PUM and UIM crossovers are in progress.





Relevant Links:

- Sheila takes OLG measurements with different Y2L and P2L gains (LHO:74264)
- Sheila attempts taking PUM crossover measurements at the L2 LOCK band (LHO:74226)
- Gabriele finds nonlinear high frequency noise coupling in DARM and suggests offloading ETMX L3 (LHO:73937)

FAMIS 20003

Chiller alarm 2 days ago, has been addressed (alog 74276).

The FSS TPD signal has continued to drop over the past 10 days; currently is at 0.72V.

Link to full report

Summary of the report:

• H1 was in observing for 72.9% of the week.
• Squeezer drift issues still present from last week. Commissioners actively investigating. A temporary line at 1.3 kHz was added over the weekend as part of tracking the drift - alog 74256
• Sprinkler blowout on Wednesday (Nov. 15) at about 16:10 UTC (alog 74218). End time is not known, but the work was estimated to last about 30 minutes.
• A few locklosses through the week due to earthquakes.
• Heavy winds on Sunday caused issues locking and reduced sensitivity for part of the day.
• Fscan plots appeared every day, and the number of lines was always within one standard deviation of the mean.

TITLE: 11/20 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 151Mpc
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 9mph Gusts, 7mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.43 μm/s
QUICK SUMMARY:

Looks like we had reached NOMINAL_LOW_NOISE at 15:42UTC and it wasn't able to go into Observing because of the one EXISC sdf from yesterday (ASC-X_FIBR_LOCK_BEAT_RFMIN being -20 instead of -10dB). I reverted the SDF and put us in Observing.

TITLE: 11/20 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 147Mpc
INCOMING OPERATOR: Austin
SHIFT SUMMARY:

5:59 UTC lockloss  see alog: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=74299

ALSX Camera looks stange. Like there is an extra light from somewhere on ALSX cam.
Gaurdian ALX Node in Error. This is what Oli had warned me about.
I changed H1:ALS-X_FIBR_LOCK_BEAT_RFMIN back down to -20 as Oli has done.

I Started Initial Alinment.
Initial Alignment complete.
Running through SDF revert changes H1:ALS-X_FIBR_LOCK_BEAT_RFMIN back to -10. so I changed it back to -20 again.

Lock had to go through AQUIRE_PRMI right after an initial alignment.

Current H1 Status: Locked for 6 minutes and Observing.

LOG:
no log

Lockloss from Nominal Low noise. Wind is not high, MicroSeism is elevated, but no earthquakes rolled through, PI's looked good.
https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi?event=1384452871
The lockloss plots look like they dont have any data, and soem of the data looks glitchy maybe. The ISC_LOCK Log that Lockloss select also brings up looks like it's missing data, the time stamp is several hours ago.

More investigations needed.

 

TITLE: 11/20 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 161Mpc
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 10mph Gusts, 7mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.52 μm/s
QUICK SUMMARY:

I do not know what exactly fixed diaggui, But while I was tinkering with different settings it started to work again on the CDS machines & it works on the FOM too.
Other than that it's been a really queit night.

TITLE: 11/20 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 150Mpc
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 13mph Gusts, 8mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.67 μm/s
QUICK SUMMARY:

H1 had been locked for 22 minutes and OBSERVING when I arrived.
DARM FOM Is still not working properly.

H1:ALS-X_FIBR_LOCK_BEAT_RFMIN is currently set to -20 and I will be keeping an eye on it.

RELOCKING
16:00UTC Detector relocking and at FIND_IR.

16:09 I started an Initial Alignment because we started getting stuck in the ACQUIRE_PRMI -> ACQUIRE_DRMI_1F -> ACQUIRE_PRMI cycle again
16:26 Initial Alignment completed

16:28 Lockloss at LOCKING_ALS
16:34 Lockloss at FIND_IR
16:46 Lockloss at CARM_OFFSET_REDUCTION
    - Starting at DHARD_WFS EX started saturating every few seconds, through DHARD_WFS, PARK_ALS_VCO, SHUTTER_ALS, and CARM_OFFSET_REDUCTION
    - 16:46:53 EY, IX, IY, PRM all also saturated, DRMI lost lock, and we went down.
16:50 Lockloss at FIND_IR
16:55 I took us to DOWN due to high winds and microseism

17:35 Tried relocking again
17:35 In LOCKING_ARMS_GREEN, Guardian ALS_XARM node in error
    
17:38 ALS_XARM trying to get to INCREASE_FLASHES but keeps jumping to FAULT
    - Errors PLL; PDH and Beat note strength
    - Node keeps falling into error
    - Tried lowering beatnote strength threshold but only worked for a bit before we lost lock
    - Happened June 29th (70951)

18:49 Lockloss at CARM_OFFSET_REDUCTION
19:05 Lockloss at CARM_OFFSET_REDUCTION, putting detector in DOWN

20:02 Trying relocking
20:56 NOMINAL_LOW_NOISE
21:01 Observing

21:33 Earthquake mode activated due to earthquake from Japan

21:51 Lockloss

RELOCKING
21:55 Starting an Initial Alignment
22:28 USERMSG: disabled pump iss after 10 locklosses. Reset SQZ-OPO_ISS_LIMITCOUNT to clear message.
    - Followed 73053, minus touching up the OPO temp, to fix and relock sqz
22:47 Initial Alignment finished, relocking
23:38 Observing

Lockloss 11/19 21:51UTC (Lockloss page has been offline for 85 mins as of when I'm writing this so I cannot link the lockloss page for this lockloss)

Not sure of cause. An earthquake was rolling through (crosshairs at lockloss time) but we had gotten through the biggest part fine. Microseism and wind are still high so it's possible that the lockloss was caused by their combination.

Closes FAMIS#26218, last checked 73966

Laser Status:
    NPRO output power is 1.819W (nominal ~2W)
    AMP1 output power is 68.01W (nominal ~70W)
    AMP2 output power is 137.0W (nominal 135-140W)
    NPRO watchdog is GREEN
    AMP1 watchdog is GREEN
    AMP2 watchdog is GREEN

PMC:
    It has been locked 12 days, 3 hr 5 minutes
    Reflected power = 15.94W
    Transmitted power = 109.9W
    PowerSum = 125.8W

FSS:
    It has been locked for 0 days 1 hr and 57 min
    TPD[V] = 0.7357V

ISS:
    The diffracted power is around 2.6%
    Last saturation event was 0 days 1 hours and 57 minutes ago

Possible Issues: None

Everything looking good!

Been having trouble relocking all day. Wind and microseism have been high and increasing, and the ALSX beat note strength being below the minimum has been throwing ALS-XARM into error (74289). Lowering the beat note strength threshold has been working for getting us past LOCKING_ARMS_GREEN, but we have now lost lock at CARM_OFFSET_REDUCTION three times, each time having a lot of EX and other suspensions saturating and causing us to lose lock.

Luckily the wind is starting to lower a bit, so I'll let the detector sit in DOWN for a bit and then hopefully relocking will go much better.

I've been getting Guardian ALS_XARM node in error when trying to lock today, something that Austin also got overnight. It looks like ALS-X_FIBR_A_DEMOD_RFMON has been worming its way down(attachment1, attachment2) towards the minimum beat note strength of -10dB since Nov 17th 16:30ish UTC. I have temporarily lowered the beat note strength minimum to be -20dB so that we can lock, but going off of the last time this happened, 70951, the ALSX laser temp and current are going to need to be adjusted.

FRS ticket already opened on this issue: FRS#28430

Naoki, Camilla, Sheila

We had another look at some signals that we could use to track or servo the SQZ angle.  

We turned the ADF back on, making a line at 1.3kHz in DARM, and tuned the demod phase for the ADF so that the SQZ angle readout was 0 for the angle we've been using for observing in this lock.  Camilla added a bandstop at 1.3kHz in the SQZ BLRMS4, sum and null channels and checked that the ADF isn't dominating those.  We did a sweep of the CLF6 demod phase like this, with the dither amplitude reduced comapred to Wed (0.01 instead of 0.03 CLK gain).  

In the first attachment we are using BLRMS 4 to demodulate for the noise lock signal, the second cursor shows that the zero crossing of the ADF SQZ angle and the noise lock both correspond roughly with the minimum of noise in the BLRM4 (1kHz BLRMs).  This isn't the same phase as the one that minimizes the brown trace, a blrms centered at 350 Hz. This means we have a frequency dependence of the SQZ angle, so we should probably look into things like the SRCL offset that might be causing this. 

Naoki then turned the ADF phase with the SQZ angle set to minize BLRMS3, as shown in the second screenshot from Camilla (at the begining here you can see that Naoki set the ADF phase so SQZ ANG was zero for CLF demod phase of 145).  We repeated the sweep and see that we have lower SNR for the noise lock using this lower frequency BLRMS, and that the ADF as it is now wouldn't make a good error signal for this sqz angle because it doesn't really go negative. 

Detchar:  We are planning to leave the ADF on over the weekend, which will create a line at 1.3kHz.  We are hoping to use this to track changes in the squeezing angle over time.  

Sheila, Naoki, Camilla

We took FDS, Anti-SQZ and Mean SQZ data at different NLGs: 14, 17, 43, 82 and 123. We think we see evidence of frequency noise. We see less squeezing at high NLGs: around 4.5dB of SQZ at NLG < 43.1 but at NLG 81.9 and 122.8, squeezing at 1kHz reduced to 3.6 and 3.1dB. Attached Plot.

Each time we optimized SQZ angle around 1kHz. Because of this, the low frequency increased noise at high NLGs could be due to incorrect rotation, rather than phase noise. 

Data saved in /camilla.compton/Documents/sqz/templates/dtt/20231025_GDS_CALIB_STRAIN.xml

Vicky and Naoki did an NLG sweep on the Homodyne in 73562.