Attaching a set of slides summarizing the on-going efforts to reproduce the beam path from SR3 through the OFI. Two sets of the beam spot position measurement on SR2 and SR3 were used to "calibrate" the slider moves and to simulate the beam position for other alignment configurations using Zemax. The slides are posted to the dcc https://dcc.ligo.org/E2400189-x0
The aLogs with the beam spot position measurements:
beam spot 1 measurement  aLog77443 
beam spot 2 measurement aLog78119

[Jenne, Sheila, Jennie, RyanS, Ibrahim]

The interferometer has remained fussy all day.  I suspect we may have some alignment issue somewhere, since the AS_AIR camera looks a little bit lobe-y and wrong in yaw.  Twice now, if we've gotten past the DARM_OFFSET state and convinced the OMC to lock (which it will do with the autolocker once the darm offset is at the full value of 9e-5), we're able to at least start powering up. 

• One time, as RyanS mentioned, we succeeded by slowly stepping the DARM offset up to the value of 9e-5, after all ASC for full IFO (Including SOFT) loops had converged. I'm not sure why we lost lock on this one, but it's possible that we're more sensitive to motion on the LVEA floor, and Robert was out taking photos.
• One time, that wasn't enough, and we were more successful with an offset of +0.6 in place in the SRC1 yaw loop both during DRMI ASC and re-engaging it during full IFO ASC.  This current lock, with that technique, once the OMC was locked I slowly set that offset back to its nominal value of zero, and we're powering up okay.  More specifics on what I did:
  • With the ISC_LOCK at CHECK_DRMI_ASC, I turned on an offset of +0.6 in SRC1 yaw.  I then let the ASC converge and set ISC_LOCK to PREP_ASC_FOR_FULL_IFO, which turned off the offset (during a time when SRC1 was not engaged).
  • I set ISC_LOCK to ENGAGE_ASC_FOR_FULL_IFO, and it was clear that the alignment was getting a bit worse with the SRC1 yaw offset off, so I re-turned it on again to +0.6. 
  • I set ISC_LOCK to PREP_DC_READOUT_TRANSITION, and the OMC was able to lock, we didn't lose lots of PRG, everything seemed overall fine.
  • I ramped the SRC1Y offset back to 0, and then requested NomLowNoise.  The alignment definitely saw the move, but it didn't seem terribly dramatic, so it seems fine.
• I am hopeful that this was a fluke, and we don't need to actually have a SRC1 yaw offset.

My hypothesis of why we might be needing this alignment offset, is that increasing the DARM offset is letting more carrier light go to the AS port, and the the AS WFS loops are seeing that and responding poorly.  So, this alignment offset helps kepe them in the right spot.  That's not a very satisfying story though.

Earlier, I had tried moving SR3 by +/- 1urad on the slider in yaw (reverting afterward to the previous slider val), but that didn't really help.  I also tried offsets in AS_C and AS_A pointing, but neither of those had the same improvement in the AS camera or POP90.  They would improve momentarily, but after the rest of the ASC caught up, we still had the lobe-y look on the AS camera.

Jennie also noted earlier (and it happened at least both of these last two power ups) that we're saturating the OMC suspension when we power up (before moving spots).  But, after all the ASCs catch up, it's no longer saturated.  So, we're close to the edge of what the OMC suspension can do, but it seems surviveable.

Attached are screenshots at CHECK_DRMI_ASC of the AS air camera, without and then with my hokey offset of 0.6 in SRC1 yaw.  You can see that with the offset in place, the camera image is much more centered and nice looking.

For posterity, here are the mattermost versions of the notes, starting with this message:

TITLE: 06/05 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY: It's been a busy shift of troubleshooting. See alogs 78251, 78258, and 78259 for summaries of the first half of the day's events. Since then, we had trouble locking DRMI where buildups looked very unstable and there were several locklosses at different places in the DRMI acquisition steps. Eventually DRMI was able to lock (Jim reverted some sensor correction-like changes he put in Monday, but unsure if this had any effect) and we made it up to engaging ASC. Sheila had moved the DARM_OFFSET state to between the ENGAGE_ASC and SOFT_LOOPS states, but we found that increasing the DARM offset again caused the PRG to drop as we had been seeing in the morning. I wasn't fast enough to lower the DARM offset, so we lost lock. On the next attempt, we ran through both ENGAGE_ASC and SOFT_LOOPS before running DARM_OFFSET, but we still saw the drop in PRG as before. I was able to turn off the DARM offset in time, then Jenne started to step it up slowly to see where things became unstable. When she reaced the nominal value of 9e-5, things still looked okay and the OMC was able to lock, so we proceeded to DC readout and powered up to 25W. At this point, Robert went into the LVEA to take pictures through a HAM2 viewport, and sometime around then, we lost lock (unsure exactly if he was the cause). We've since relocked and continue to troubleshoot why we canot turn on the DARM offset reliably.

Handing off to Ibrahim as troubleshooting is ongoing.

LOG:

Andrei, Sheila, Naoki

We've observed that the reduction of the observable range is accompanied by the presence of "whistles", high-magnitude frequency-modulated signals in the spectrum in SQZ FC WFC.

Sheila created several plot overlays to demonstrate this dependence. Specifically, we selected the Lock : Range : DMT-SNSL_EFFECTIVE_RANGE_MPC and SQZ : Subsystems : SQZ FC WFC plots and overlaid them to highlight the relation between them.

• overlayjune2.png shows no “whistles,” and the range does not undergo any significant decreases.
• overlayMay18.png, overlayMay21.png and overlaymay30.png datasets exhibit the presence if "whistles" and a corresponding range decrease.
• overlayMay22.png, overlayMay23.png datasets dont have the visible "whistles", but the 10-20 Hz low-frequency noise band increases during the range decrease.

DARM_comparison.png plot contains the 18th of May 2024 DARM during the “whistle” and during its absence. ≈0.5 dB difference is observed.

TITLE: 06/05 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 13mph Gusts, 11mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.38 μm/s
QUICK SUMMARY:

IFO is LOCKING at DARM_OFFSET

We're experiencing issues with the Power Recycling Gain. Jenne, Sheila and Ryan S are troubleshooting it currently. This is preventing us from locking.

Naoki, Sheila

To estimate the FC backscatter noise, we excited the FC IR error signal. The attached figure shows the FC IR error signal and DARM with (red)/without (blue) exciation. Although the FC IR error signal is excited by a factor of more than 30 between 10-100 Hz, the excess noise in DARM is small so the FC backscatter noise should be much smaller than DARM.

Jennie W, Jenne D, Sheila

The other day I designed a new notch to get rid of a bump we had been seeing at 17.75Hz. We think this is the bounce mode of the BS coupling into the MICH loop. The notch for this in LSC-MICH has not been updated since the BRDs were installed I think, and there is no notch for this mode in the ASC-MICH_P and ASC_MICH_Y loops.

Changes made to ISC_DRMI.py

Added Bounce-roll notch (FM8) to MICH2 (line 457) in DRMI_LOCKED state and took out line turning on notch at 15Hz (line 456).

#ezca.get_LIGOFilter('LSC-MICH2').switch_on('FM10') # Turn on the BounceRoll FM10 in LSC FM

ezca.get_LIGOFilter('LSC-MICH2').switch_on('FM8') # Turn on the BounceRoll FM8 in LSC FM 05/06/2024 J Wright

Added bounce-roll notch to PREP_PRMI_ASC in FM MICH_P and MICH_Y in FM9. These were added to lines 690 and 692.

Added these same filters in PREP_DRMI_ASC state in FM MICH_P and MICH_Y in FM9 in lines 920 and 922.

After much time diagnosing locking issues this morning (see alogs 78251 and 78258 for the main events), we eventually reached NLN after damping violin modes in OMC_WHITENING for 40 minutes. Almost two minutes after reaching NLN, Robert attempted to ramp the bias on ITMY, but accidentally changed it immediately, which unlocked the IFO at 19:38 UTC.

H1 is currently relocking. We're scheduled for commissioning time until 22:00 UTC, so if we can get back to low noise, we'll spend time commissioning until then.

Jenne Driggers, Ryan Short, Sheila Dwyer, Jim Warner

In several of the locklosses that TJ and Ryan had early this morning, the PRG would drop before we started to engage ASC, and we would loose lock while the ASC loops came on.  By pausing and going line by line through the guardian we identified that turning on the DARM offset was causing the drop in PRG.  We were able to engage the ASC without issue when the DARM offset was off.  At this point we did a series of OMC scans, attached screenshot, but everything seems normal here.

Our hypothesis is that the DARM offset was too large when the alignment was off (and perhaps some microseism makes things a little less stable), but that things are fine once the ASC is on. 

For this locking attempt I've moved the DARM offset state to after ENGAGE_ASC.  We had initially moved it to before ASC a long time ago to save time by locking the OMC while the ASC was being engaged.  But, if this causes locklosses while we are engaging the ASC this isn't worthwhile, and we can let the OMC lock while we are waiting for the soft loops.  I had a look at the lockloss tool to see how often we've been loosing lock from this state, it say we lost lock from ENGAGE_ASC (429) 8 times since March, but this only includes one lockloss from last night.

I'll leave this guardian change in for now.

While we were looking into this, Jim made changes to sensor correction and CPS DIFF.  This might have been helpful, based on the locking progress it's hard to say because his changes happened as we were understanding that the DARM offset was the issue.

Sheila, Jennie W, Jenne D

Peter F suggested we dither the alignment into the OFI and look at the power fluctuation on our QPDs and diodes in HAM6.

If we are right on top of some trough in OFI throughput thewn we should see a 2*f_dither line in our QPDs and PDs.

If we are on the side we should see f_dither frequency lines, and away from the bad spot on the OFI we should see no modulation at either of these frequencies.

We had trouble finding the centre of the bad spot with this measurement although it did tell us some more information about how steep it edges are.

See the below image for the channels we were monitoring.

We set the sliders to the 'bad spot' ie. our alignment into the OFI that was nominal before April 24th this year shown in the first table column here.

The values we used were:

We turned the dither lines on the third stage down on SR2 (11 Hz for pitch degree of freedom and 13 for yaw).

The channels used were H1:SUS-SR2_M3_TEST_P_EXC and H1:SUS-SR2_M3_TEST_Y_EXC with a sine wave injection of 1000 counts amplitude.

We set the IFO_ALIGN guardian to SR2_ALIGN which feeds back to SR2 on its own to keep the beam centred on AS_C QPD, this means we only had to change the position of DSR3 and SR2 would be stepped by this loop (SRC2).

We also turned on the centering loops, then the OMC ASC, but had to turn the OMC ASC off for some of the measurement period as OMC suspension kept saturating.

17:16:48 UTC went to bad spot mentioned in table and tried dither - did not see 2F modulation in power on all five HAM 6 QPDs or the OMC REFL PD (shown here as dark blue reference).

This time is shown by the position (roughly) of the first cursor in this image.

We eventually saturated the SR2 suspension when driving it in pitch and still don't see 2*f_dither peaks.

Now walking the beam from this position at around 17:34 UTC.

The magenta curve was taken after we had moved in the positive yaw direction to have the SR3 yaw slider at -116 uradians and the SR2 yaw osem to be at155 uradians. It can be seen that the pitch line gets higher and the yaw line lower so this means we are moving away from the centre of the bad spot in yaw and towards it in pitch as we moved the beam in this direction on the OFI.

We ran OMC ASC again to centre on OMC REFL as this dropped but then fully saturated OMC ASC so turned off OMC ASC.

OMC model restarted at 17:52:45 UTC and so the OMC ASC restarted with 0 offsets, since we hadn't cleared history this caused oscillation on QPDs for OMC.

We increased the gain in the SRC2 centering loop by increasing the input matrix sleement in SRC2 loop.

18:45:52 UTC Trying to move to better OFI spot suggested by Alena's calculations but this moves us off AS_C and saturates DC centering so will move us off AS_A and B (if we do this we will need to pico back onto these QPDs).

The light blue curve was taken at our current alignment through the OFI, marked with the second cursor in this image.

The xml file Sheila used for measuring the dither on SR2 is also attached.

Wed Jun 05 10:10:06 2024 INFO: Fill completed in 10min 2secs

Gerardo confirmed a good fill curbside.

I ran a BRUCO (brunt force coherence) from a time segment (400s) starting at GPS 1399898087 when we had high noise in the range BLRMS and this was correlated with high noise on the CO2_ISS_CRTL2 signal, as discussed in this comment by Sheila.

We know our instances of abrupt drops are correlated with the CO2 ISS array and possibly the squeezer and the purpose of this is to check for any other correlations whoch provide an explanation for our range drops.

The BRUCO was run on GDS-CALIB_STRAIN_NOLINES to avoid finding coherences between the calibration lines and various channels.

The BRUCO can be ran following these instructions from Elenna.

TITLE: 06/05 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 14mph Gusts, 10mph 5min avg
    Primary useism: 0.06 μm/s
    Secondary useism: 0.51 μm/s
QUICK SUMMARY: TJ has been working on locking troubles this morning; sounds like locklosses mostly happening around ENGAGE_ASC and the OMC is consistently locking on the wrong mode. Will start by stepping through ENGAGE_ASC slowly.

• Wind is finally low
• Microsiesm has been on the rise

H1 lost lock at 0807UTC (0107PT) for a reason I can't figure out. Since then there have been numerous lock losses from various states in the locking process. It has made it past full power, but other times it won't make it to DC readout. I've ran another initial alignment just to try something, but I'm having the same results. There was also a small earthquake from mexico to slow this all down.

The useism is getting higher, though not terrible, but I'm beginning to wonder if maybe some of the sensor correction changes that have happened (alog78226) aren't playing well with this ground motion. I've changed the SEI configuration to USEISM since the wind is still low and well see how this goes.

TITLE: 06/05 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Observing at 132Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY:

IFO is in NLN and OBSERVING as of 06:09 UTC

IFO was in DOWN for the majority of the shift due to 40mph wind gusts that were preventing locking (primarily ALS). At 04:00 UTC, the wind finally dropped below 30mph. This was the locking process:

04:05 UTC - INITIAL ALIGNMENT - I had to get involved to alter the states for ALSY since it was having issues with the WFs (wind was still around 30mph here. This was the bulk of time spent in initial alignment.

04:47 UTC - SRM WD tripped during SRC Align (during the usual many saturations - have never seen a WD trip before during though), prompting untripping intervention

04:55 UTC - Initial alignment complete, moved into full IFO locking

05:42 UTC - NLN Acquired fully automatically and without going into PRMI (Dying wind helped with this I presume). There was some extra time spent in ENGAGE_ASC_FOR_FULL_IFO due to non-converging signals

06:09 UTC - OBSERVING - it took 30 minutes because I accepted SQZ diffs that I thought were preventing us from going into OBSERVING only to realized that we weren't squeezing for whatever reason (and thus not in truly complete NLN). I went into the SQZ screen and requested Freq. Dep. SQZ and a few minutes later, we were in full NLN. I then went back and re-accepted those diffs that I erroneously accepted. For the sake of book-keeping, I've attached both versions of each. I was briefly thrown off by an ISS Pump message telling me to troubleshoot via an alog but this was just a symptom of there not being any squeezing at all. I do not know why IFO didn't automatically go into squeezing as it usually does. This was my only intervension during locking.

Other:

• Wind gusts have drastically reduced over the last 3 hours (starting 4:50 UTC), hovering at around 15 max
• Secondary microseism has steadily increased, crossing the 90% threashold around one hour ago (6:50 UTC)

LOG:

None

LLO noted a 10 Hz comb that was attributed to the IRIG B monitor channel at the end station connected to the PEM chassis. (https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=71217)

LHO has agreed to remove this signal for a week to see how it impacts our DARM signal.

EX and EY cables have been disconnected.