Lockloss @ 09/14 01:09UTC, a few minutes after going back into Observing

Closes FAMIS#26328, last checked 79826

Corner Station Fans (attachment1)
- All fans are looking normal and within range.

Outbuilding Fans (attachment2)
- All fans are looking normal and within range.

Landed the new cables for the old type of controllers (Dual type) for IP1, IP2, and IP3.  The MEDM screens will need to be updated to reflect the correct visual representation.

TITLE: 09/13 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Oli
SHIFT SUMMARY: Short lock stretches today for some reason. Fortunately, at least relocking has been mostly straightforward.

• 14:30 - H1 locked and observing for 6 hours
• 16:36 - Lockloss - alog80075
  • Taking some time for ALS commissioning
• 18:29 - Observing
• 18:47 - Lockloss - alog80079
  • 19:35 - Lockloss during TRANSITION_FROM_ETMX
• 20:42 - Observing
• 22:13 - Lockloss - alog80082

H1 is currently relocking, so far up to TRANSITION_FROM_ETMX.

LOG:

TITLE: 09/13 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 14mph Gusts, 10mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.08 μm/s
QUICK SUMMARY:

Currently relocking and at MAX_POWER.

Lockloss @ 22:13 UTC - link to lockloss tool

No obvious cause; only locked for about 90 minutes. I don't see the ETMX glitch prior to the lockloss that I've seen in the others today.

FAMIS26008

Nothing looks out of the ordinary and is consistent with the last edition of this famis task (alog79990).

Lockloss @ 18:47 UTC - link to lockloss tool

No obvious cause; were only locked for 20 minutes. ETMX started moving very slightly 43ms before the lockloss, but otherwise I don't see anything else suspicious.

After a lock loss this morning Sheila and I spent some time looking further into the mode spacing for ALS (alog80065). We each took an arm and looked at the mode spacing and peak heights at good alignments and some not as good alignments. This exercise didn't yield any major revelations, but did confirm that our Y arm definitely doesn't have perfect mode matching, X arm seems to be better than Y. 

Attachment 1 - ALS Y 00 mode to 2nd order mode spacing. This showed that yesterday Sheila and Ibrahim missed the small 1st order mode between these and Sheila has sinced updated their FSR and FHOM values (alog80076).

Attachment 2 - An example of ALS Y locking below the max flashes, showing that some of our power is in the higher order modes.

Attachment 3 - ALS X comparison of attachment 1

Attachment 4 & attachment 5  - ALS X 1st order peak spacing and height for an okay alignment (4) vs a better alignment (5).

Attachment 6 & attachment 7 - Same as above but for the 2nd order modes.

Lockloss @ 16:36 UTC - link to lockloss tool

No obvious cause, but there's a small hit on ETMX about 100ms before the lockloss. We've often seen similar motion before locklosses like this.

FAMIS 26295

Laser Status:
    NPRO output power is 1.822W (nominal ~2W)
    AMP1 output power is 64.33W (nominal ~70W)
    AMP2 output power is 137.7W (nominal 135-140W)
    NPRO watchdog is GREEN
    AMP1 watchdog is GREEN
    AMP2 watchdog is GREEN
    PDWD watchdog is GREEN

PMC:
    It has been locked 16 days, 21 hr 59 minutes
    Reflected power = 22.18W
    Transmitted power = 104.8W
    PowerSum = 127.0W

FSS:
    It has been locked for 0 days 8 hr and 13 min
    TPD[V] = 0.8122V

ISS:
    The diffracted power is around 1.5%
    Last saturation event was 0 days 8 hours and 13 minutes ago

Possible Issues: (all are known)
    AMP1 power is low
    PMC reflected power is high
    ISS diffracted power is low

Fri Sep 13 08:12:45 2024 INFO: Fill completed in 12min 41secs

TITLE: 09/13 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 156Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 11mph Gusts, 7mph 3min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.07 μm/s
QUICK SUMMARY: H1 has been locked and observing for just over 6 hours. Looks like one automated relock overnight.

• Wind and ground motion low
• All other systems nominal

I just noticed a chat message from Erik that there was only one hour of Seismic blrms data being displayed in the control room. This has to do with the fact that after h1dmt1 was restarted there was only 1 hour of trend data available in the last 48  hours due to the mis-mounting of the /gds trends directory. The trend files are now available back to yesterday afternoon, so a restart of the blrms monitors should expand the displayed data to at least 16 hours. I tried to contact the control room via teamspeak, but I did not get a reply. I will wait until the control room is manned to restart the blrms monitors. This should have no effect other than to cause the blrms monitor displays to update.

Sheila, Ibrahim

Context: ALSY has been misbehaving (which on its own is not new). Usually, problems with ALS locking pertain to an inability to attain higher magnitude flashes. However, in recent locks we have consistently been able to reach values of 0.8-0.9 cts, which is historically very lockable, but ALSY has not been able to lock in these conditions. As such, Sheila and I investigated the extent of misalignment and mode-mismatching in the ALSY Laser.

Investgiation:

We took two references, a "good" alignment, where ALSY caught swiftly with minimal swinging, and a "bad" alignment where ALSY caught with frequent suspension swinging. We then compared their measured/purported higher order mode widths and magnitudes. The two attached screenshots are from two recent locks (last 24hrs) from which we took this data. We used the known Free Spectral Range and G-factor along with the ndscope measurements to obtain the higher order mode spacing and then compared this to our measurements. While we did not get exact integer values (mode number estimate column), we convinced ourselves that these peaks were indeed our higher-order modes (to a certain extent that will be investigated more). After confirming that our modes were our modes, we then calculated the measured power distribution for these modes.

The data is in the attached table screenshot (copying it was not very readable).

Findings:

• Both alignments were found to be pretty bad, so can't really compare to one another. Evidenced by similarity in fractional magnitude.
• Magnitude distribution in M(0,0) shows that only 70% of our power is going into the main 0,0 mode, implies a bad alignment since we're reading 0.9+ on both of these count values, and these are supposed to be normalized to 1.
• ~18% of our power is going into the first higher order mode, which implies a misalignment
• ~6.5% of our power is going into the second order mode, which implies a mode mismatch (and also some misalignment).

Next:

• Take more data points of actually good alignments that may have a higher distribution of power in its 0,0 mode (as designed). 70% is apparenltly quite bad alignment/mode-match wise
• See if there's a particular difference between different successful ALSY lock counts
• Since we were quite misaligned, do the same investigation for ALSX and guage

Investigation Ongoing

A while ago Oli found that some earthquakes cause IMC splitmon saturations, possibly causing locklosses. I asked Daniel this with respect to the tidal system to see if we could improve the offloading some. After some digging we found that some of the gains in the IMC get lowered to -32db at high power, greatly reducing the effective range of the IMC SPLITMON. He and Sheila decided that the best place to recover this gain was during the LASER_NOISE_SUPPRESSION state (575) so Sheila added some code to that state to redistribute some the gain (lines 5778 -5788): 

            self.gain_increase_counter = 0
        if self.counter ==5 and self.timer['wait']:
            if self.gain_increase_counter <7:  #icrease the imc fast gain by this many dB
                #redistribute gain in IMC servo so that we don't saturate splitmon in earth quakes, JW DS SED
                ezca['IMC-REFL_SERVO_IN1GAIN'] -= 1
                ezca['IMC-REFL_SERVO_IN2GAIN'] -= 1
                ezca['IMC-REFL_SERVO_FASTGAIN'] += 1
                time.sleep(0.1)
                self.gain_increase_counter +=1
            else:
                self.counter +=1

We tried running this, but an error in the code broke the lock. That's fixed now, the lines are commented out in ISC_LOCK and we'll try again in some other day.