bypass will expire:
Tue 29 Aug 2023 09:02:01 PM PDT
For channel(s):
    H0:FMC-EY_CY_H2O_PUMPSTAT
 

Last night at 0827 UTC we dropped out of Observing fir just over a minute while TCSX relocked. This is a known issue (see alog72220 for a quick reference), but I wanted to keep record here in the alog.

TITLE: 08/29 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 13mph Gusts, 10mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.07 μm/s
QUICK SUMMARY:

• VAC/SEI/CDS looks good, PLS anteroom dust alarm
• 14:08 lock time
• The charge measurement is finishing up

TITLE: 08/28 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 159Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY:

- Arrived to a relocking IFO, just got back to NLN @ 23:15

- Lockloss @ 23:20 - 5 minutes later :), no obvious cause

- Had a few locklosses at FIND IR and noticing the ITM CAM error signals are poor for both arms, so will be running an initial alignment

• Had to move SRM a bit in P to catch SRC but otherwise the IA was entirely automated

- 23:46 UTC - had a verbal alert for a GRB-Short (E433366) and a STAND DOWN - believe this came from Fermi, as we are currently running an IA

- Back to low noise @ 0:51/OBSERVE @ 1:56

• Vicky mentioned an issue with the SQZ sensing matrix (alog for more info), so she and Naoki are going to troubleshoot while LLO is down and currently relocking (in COMMISIONING mode from 1:11 - 1:56 UTC, we went back to observing once LLO was relocked and had flipped their intention bit)

- 4:54 - inc 5.9 EQ from Indonesia

• 5:14 - another EQ...this one is a 5.4 from Panama
• 5:27 - EQ mode activated, back to CALM @ 5:37

- Overall a quiet night, leaving H1 to TJ in managed mode

LOG:

No log for this shift.

Naoki, Vicky

After alog72496, we turned on the SQZ ASC, but the ANG P/Y ran away for the first 10 minutes as already reported in alog71217. We remeasured the SQZ ASC sensing matrix and decided the input matrix as shown in the first and second attached figures (first: old, second: new). However, the behavior of ANG P/Y with new input matrix is similar to before (third attached figure). The ANG P/Y ran away and SQZ BLRMS got worse for the first 10 minutes and started to converge after that.

H1 has been locked for just over 2 hours. IFO appears to be behaving nominally, and ground motion looks to have recovered from the 7.1 EQ earlier today.

Naoki, Vicky

At 25W, holding at REDUCE_RF45_MODULATION_DEPTH before MAX_POWER, we did a quick anti-sqz/sqz measurement to see if we could loosely infer sqz losses with the IFO relatively cold. We did not see clearly more squeezing here at lower power / colder ifo, as we have in the past, e.g. 66877. We weren't able to easily engage ASC here (it was before move_spots), but slider values are similar to when IFO was last locked and a little bit of walking alignment didn't make a big difference, so we left ASC off.

Looking at DARM (based on GDS), and the SQZ BLRMS, we can read the sqz levels:

• Anti-sqz:  DARM ~ +13.0 dB,   BLRMS 5,6~ 13 dB
• SQZ:     DARM ~ -3.9 dB,     BLRMS 5,6~ -4 dB

which are surprisingly consistent to what we have at full power, e.g. the recent sqz dataset in 71902 with IFO at 60W and thermalized in full lock.

Thoughts:
  -- I'm not sure what it means that now we have similar amounts of squeezing with the IFO at low and high power.
  -- For this test, we left PSAMS at 200V/200V and didn't try walking PSAMS, though in 66877 we saw better squeezing with a cold IFO and lower PSAMS.
  -- The slope of DARM at high ~kHz frequencies seems to be slightly different between 25W and 60W? From the dtt, compare red dashed = "60W no sqz" & cyan line = "25W FDS". Here, the lock sequence was at REDUCE_RF45_MODULATION_DEPTH, so before lownoise length control and laser noise suppression.

We then measured NLG ~ 11.05. In the sqz dataset 71902 we did not separately measure NLG, but given that the NLG has remained consistently accurate with the Feb 2023 NLG calibration, so we can re-affirm that trusting the calibration is probably okay, assuming opo temperature is well-tuned. For comparison, NLG calibration suggests NLG=11 (gen sqz ~ 14.75dB) with opo_trans=80uW.

• max_ir = 0.0081
• min_ir = 0.00022
• no_pump_ir = 7.1e-4
• dark = -2.5e-5

J. Kissel, J. Warner

Given the lovely "natural experiment" of the Aug 25th 2023 ~3.2 [deg F] / 1.8 [deg C]  temperature increase "impulse" over 4 hours at EY (see LHO:72444 and LHO:72428), I wanted to understand both 
    (a) How the IFO handles it / what caused the lock loss
    (b) Document the levels and time-scales of alignment change that resulted in bad alignment for *several* lock stretches -- indeed *days* after the impulse.

We often wave our hands saying things like "well, the vacuum system acts like a low pass filter, with a time constant of [[insert hand-waivers favorite time-scale on the order of hours]]." I wanted to see if we could quantify that, and if not, add a bit more clarity to how complicated the situation is.

To do so, I looked at the Y End Stations' signals in Z, RZ, PIT, and YAW that are either 
    (1) out of loop, or 
    (2) when in-loop -- the loop's feedback control output using the "classic trick" of approximating G/(1+G) ~ 1, where G >> 1, such that (plant) * CTRL = out-of-loop signal as though the loop wasn't there.

Those sensors include 
    - HPI ST1 ISO OUT (which are the IPS, under DC-coupled feedback control) -- calibrated into nano- meters or radians
    - ISI ST1 ISO OUT (which are the CPS, under DC-coupled feedback control) -- calibrated into nano-
    - ISI ST2 ISO OUT (which are the CPS, under DC-coupled feedback control) -- calibrated into nano- 
    - SUS ETMY M0 LOCK OUT (i.e. the WFS, under DC-coupled global ASC control) -- calibrated into micro- meters or radians
    - SUS ETMY M0 DAMP IN (which are "out-of-loop" because the local damping loops are AC-coupled) -- calibrated into micro-
    - SUS TMSY M1 DAMP IN (equally "out-of-loop")  -- calibrated into micro-
    - ETMY L3 Optical Levers -- calibrated into micro-
(I'm pleasantly surprised at how well they all agree, to the ~0.25 micro- kind of level that I have for these trends.)

I conclude:
    (I) The IFO Yaw is most impacted by the SEI system's RZ motion, due to the Z to RZ cross-coupling of the radially symmetric system of triangular ISI blade springs, as the blades sag from temperature increase
        (i) The total SEI system's yaw swung ~2 [urad] during the excursion dominated by ISI ST1, 
        (ii) The SUS ETMY and SUS TMSY follow this input in common, and 
        (ii) ISI ST1 takes the longest time to recover alignment --  then trending over *days* slowly back to pre-impulse equilibrium -- and still not yet there as of Aug 28
    (II) The IFO Pitch most impacted by the ETMY and TMSY SUS system's expected Pitch and Vertical sag from temperature increase.
        (i) The IFO's global alignment drives the pitch of ETMY, which drifted *down* in pitch over ~10 [urad] before losing lock, presumably from running out of range
        (ii) The ASC signals seem to slowly drive the ETMY off into-the weeds trying to recover the original, pre-impulse, alignment, causing *eventual* subsequent lock losses as it pushes the optic *past* the pre-impulse alignment position
        (iii) The TMS, which does not have global control, pitches a similar amount, ~14 [urad] in the *opposite direction, up* in pitch, and also taking *days* to get back the original value (somewhat alleviated )
        (iv) The fact that the Sus-point blades are the *same* for the QUAD and TMS, they're the biggest blades in either SUS, and the order of pitching is about the same implies to me that the pitching is dominated by the upper, Sus-point blades.

I attached the trends that drive me towards these conclusions. 
Give yourself time -- I've stared at these all afternoon to come to these conclusions. And honestly, I *still* don't think I've looked at enough plots (e.g. I don't show the ETMY alignment sliders that are the operators and/or initial alignment drives trying to make up for the SEI blades yaw and SUS blades pitch).

I also attach a .txt file that goes into more detail about how I calibrated the various CTRL signals, including where I got the transfer function values that are scaling the trends that you see.

Lockloss @ 23:20, DCPD saturation right before, cause unknown.

TITLE: 08/28 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 145Mpc
INCOMING OPERATOR: Austin
SHIFT SUMMARY:
Quiet shift until the noisy earth rolled a Mag7.1 Earthquake to H1.  Made it back to Low Noise (but then it just lost lock).

(Jim discovered LVEA lights were on...probably had been for several days!)

LOG:

• 1500 Vacuuming (w/ Shopvac) in Mechanical Room (to vacuum water out of an air handler drain pan)
• 1506 H1 back to Observing (after 1152 lockloss)
• 1548-1622 Optics Lab cleaning (karen)
• 1659-1931 MX cleaning (cindi)
• 1934 Hanford automated message received about siren testing around ~1pm PT/local time.
• 2002:  M7.1 EQ from indonesia verbal warning
  • 2016 Lockloss around when S-wave passed
• 2040-2052 HAM1 guardian for in-air L4Cs (jim)
• 2052-2113 Installing a new in-air L4C under HAM1 (jim)
  • Lights were left ON in lvea!  (probably since Wed or Tues)
  • Jim did a walk through the LVEA to make sure no one else was out there and then turned OFF the lights in the LVEA & CER (which were also left ON)
• 2101-2108 Grabbing wipes in Cleaning Area (travis)
• 2130 Begin locking post-EQ
  • LOCK#1:
    • X&Y green needed INCREASE FLASHES (did not need to deal with ALSy WFS like Austin had to)
    • took to PRMI & locked (after small BS pitch adjustments)
    • But DRMI did not lock (even with good flashes), so returned to PRMI.
    • DRMI locked on 2nd attempt within 4min!  Woo Woo!  
    • Held at REDUCE RF45 MODULATION DEPTH, so Vicky & Naoki could do Squeezer work in the control room.
    • Handing off to Austin.

TITLE: 08/28 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: SEISMON_ALERT
    Wind: 14mph Gusts, 10mph 5min avg
    Primary useism: 0.06 μm/s
    Secondary useism: 0.12 μm/s
QUICK SUMMARY:

- H1 is holding off at REDUCE RF45 for SQZ work

- SEI looks to have mostly recovered from the EQ earlier

- CDS/DMs ok

After H1's longest lock of O4 thus far (60hr29min Lock from 8/23(wed) 1020amPT -- 8/25(fri) 11pmPT), H1 has has been finicky with locking and also has had a degrading inspiral range. 

The primary issue prior to this lockloss was the EY temperature increase of 3+degF due to FAILURE of Chiller Pump #1 at EY.  From about 10am-5pm the temperatures were NON-nominal (but H1 remained locked this whole time(!) as ASC maintained the pointing for EY during this HUGE temperature swing).

With that said about the temperature drift, and at TJ's request, wanted to also list Commissioning changes which occured during the 2.5days of the lock.

(Times in PT)

Aug23 (Wed)

• 1020amLocal:  H1 60hr lock BEGINS
• HAM1 in-air ground L4C HEPI Feedforward filter + gain change (332pm alog 72393, Jim)
• PCALY 17.1Hz Cal Line Amplitude Reduced (355pm alog 72394, Louis)

Aug24 (Thurs):  Nothing in alog

Aug25 (Fri)

• 10am-5pm EY Chilled Water Pump #1 FAILED & temperature in the LVEA drifts on order of 3 degrees farenheit over the cours of most of the day shift. (530pm alog 72444, tyler)
• MICH Feedforward filter change (2pm alog 72430, gabriele)
• HAM1 in-vac L4Cs ASC FF filters updated (324pm alog 72432, Elenna)
• ~11pmLocal H1 60hr lock ENDS

Aug26 (Sat)

• Lockloss recovery late Fri night & Sat morning (before midnight Fri, alog 72440, Austin, Tony)
  • dither script run 
  • changing sliders to a previous initial alignment of gps time 1376843785 (finally helped with ALSy)
  • turning off ALSy WFS for 5 secs at beginning of lock (alog 72460)

Last Tuesday (Aug 21, 2023) Rick and I went down to the EX station while the IFO was still locked. 
We asked the operator to turn off BRS sensor correction (Sitemap-> ISI_CONFIG ->SEI_CONF ->WINDY_NO_BRSX). This was to allow us to gentley walk to the other side of the beam tube to access the PCAL Transmitter module.

Once there Rick and I opened the Tx enclosure and blocked the Outer ( Lower ) PCAL beam at GPStime: 1376752360
We saw the following motion of the PCAL channel :  H1:CAL-CS_TDEP_PCAL_X_OVER_Y_REL_MAG .
We did expect this to go up and then settle down. We were suprised by how long it took to settle down.
At GPStime 1376753100 we blocked the inner (Upper) beam was blocked.
That seems to last uninteruppted until GPStime: 1376754170

At 1376754420 ISC_LOCK went to from NOMINAL_LOW_NOISE to DOWN and signals the the start of the End station measurments made that day.

More analysis coming soon.

The nominal value of the squeezer laser diode current was change to 1.863 from 1.95. The tolerance is unchanged at 0.1. Looking at trends we sometimes read a value just below 1.85 leading to a failed laser condition which in turn triggers a relocking of the squeezer laser. However, since we are already locked, all we see is the fast and commen gains ramping down and up.

2016 H1 lost lock due to 7.1 south pacific earthquake.

We rode through the P & S waves....the R wave is still about 30+min away and it will be "spicy" ~Jim W.

I have taken H1 to IDLE....while we wait for the R wave to pass & for the planet to calm down after that.

Attached are some screenshots Tony snapped.

Naoki noticed the pump fiber rejected PD (in ham7, rejects pump fiber light that comes out in the wrong polarization) was saturated, so today I re-aligned the pump fiber polarization using sqzt0 pico's (described recently also in 71761).

I'm not sure why pump fiber needs to have its input polarization readjusted so often; I checked the CLF fiber and FCGS fiber, and they both seemed relatively well-aligned despite having not adjusted those in a while. Especially this time, it seems the fiber polarization got misaligned more quickly than before.

J. Kissel, D. Barker

As of today Dave helped me install the new front-end, EPICs controlled oscillators discussed in LHO:71746. Then, after crafting a few new MEDM screens (see comments below), I've turned ON some of those oscillators in order to replace the unstable function of the CAL_AWG_LINES guardian.

So, there're no "new" calibration lines (not since we turned CAL_AWG_LINES back ON last week at 2023-07-25 22:21:15 UTC -- see LHO:71706) -- but they're now driven by front-end, EPICs controlled oscillators rather than by guardian using the python bindings for awg (which was unstable across computer crashes, and other connection interruptions).

This is true as of the first observation segment today: 2023-08-01 22:02 UTC
However, due to a mishap with me misunderstanding the state of the PCALY SDF system (see LHO:71879), I accidentally overwrote the PCALXY comparison line at 284.01 Hz, and we went into observe. Thus, 
The short observation segment between 22:02 - 22:11 UTC is out of nominal configration, because there's no PCALY line contributing to the PCALXY comparison.
 
The was rectified by the second observation segment starting on 2023-08-01 22:16 UTC.

Also, because of these changes the subtraction team should switch their witness channel for the DARM_EXC frequencies to H1:LSC-CAL_LINE_SUM_DQ.
The PCALY witness channel remains the same, H1:CAL-PCALY_EXC_SUM_DQ, as the newly used oscillators sum in to the same channel.
Below, I define which oscillator number is assigned to which frequency.

Here's the latest list of calibration lines:
Freq (Hz)   Actuator                   Purpose                      Channel that defines Freq             Changes Since Last Update (LHO:69736)     
8.825       DARM (via ETMX L1,L2,L3)   Live DARM OLGTFs             H1:LSC-DARMOSC1_OSC_FREQ              Former CAL_AWG_LINE, now driven by FE OSC; THIS aLOG
8.925       PCALY                      Live Sensing Function        H1:CAL-PCALY_PCALOSC5_OSC_FREQ        Former CAL_AWG_LINE, now driven by FE OSC; THIS aLOG
11.475      DARM (via ETMX L1,L2,L3)   Live DARM OLGTFs             H1:LSC-DARMOSC2_OSC_FREQ              Former CAL_AWG_LINE, now driven by FE OSC; THIS aLOG
11.575      PCALY                      Live Sensing Function        H1:CAL-PCALY_PCALOSC6_OSC_FREQ        Former CAL_AWG_LINE, now driven by FE OSC; THIS aLOG
15.175      DARM (via ETMX L1,L2,L3)   Live DARM OLGTFs             H1:LSC-DARMOSC3_OSC_FREQ              Former CAL_AWG_LINE, now driven by FE OSC; THIS aLOG
15.275      PCALY                      Live Sensing Function        H1:CAL-PCALY_PCALOSC7_OSC_FREQ        Former CAL_AWG_LINE, now driven by FE OSC; THIS aLOG
24.400      DARM (via ETMX L1,L2,L3)   Live DARM OLGTFs             H1:LSC-DARMOSC4_OSC_FREQ              Former CAL_AWG_LINE, now driven by FE OSC; THIS aLOG
24.500      PCALY                      Live Sensing Function        H1:CAL-PCALY_PCALOSC8_OSC_FREQ        Former CAL_AWG_LINE, now driven by FE OSC; THIS aLOG
15.6        ETMX UIM (L1) SUS          \kappa_UIM excitation        H1:SUS-ETMY_L1_CAL_LINE_FREQ          No change
16.4        ETMX PUM (L2) SUS          \kappa_PUM excitation        H1:SUS-ETMY_L2_CAL_LINE_FREQ          No change
17.1        PCALY                      actuator kappa reference     H1:CAL-PCALY_PCALOSC1_OSC_FREQ        No change
17.6        ETMX TST (L3) SUS          \kappa_TST excitation        H1:SUS-ETMY_L3_CAL_LINE_FREQ          No change
33.43       PCALX                      Systematic error lines       H1:CAL-PCALX_PCALOSC4_OSC_FREQ        No change
53.67         |                            |                        H1:CAL-PCALX_PCALOSC5_OSC_FREQ        No change
77.73         |                            |                        H1:CAL-PCALX_PCALOSC6_OSC_FREQ        No change
102.13        |                            |                        H1:CAL-PCALX_PCALOSC7_OSC_FREQ        No change
283.91        V                            V                        H1:CAL-PCALX_PCALOSC8_OSC_FREQ        No change
284.01      PCALY                      PCALXY comparison            H1:CAL-PCALY_PCALOSC4_OSC_FREQ        Off briefly between 2023-08-01 22:02 - 22:11 UTC, back on as of 22:16 UTC
410.3       PCALY                      f_cc and kappa_C             H1:CAL-PCALY_PCALOSC2_OSC_FREQ        No Change
1083.7      PCALY                      f_cc and kappa_C monitor     H1:CAL-PCALY_PCALOSC3_OSC_FREQ        No Change
n*500+1.3   PCALX                      Systematic error lines       H1:CAL-PCALX_PCALOSC1_OSC_FREQ        No Change (n=[2,3,4,5,6,7,8])

Evan Goetz noticed that there is a mismatch between the Pcal injected frequency and "matching" demodulator for PCALY line 4.  This line at 24.5 Hz is in place for temporary monitoring of the thermalization effects on calibration, eg, alog 69478.  I don't think that this is detrimentally affecting our calibration at this time, so I'm not going to call Jeff while he's on vacation.  He can take a look Monday morning to make it more clear which frequency is supposed to be where.

I went through the hardware injection screens (CAL_INJ_CONTROL2, PCAL_END_EXC) and cleaned them up based on the way that we're now handling hardware injections

• The INJ_TRANS guardian node was decommissioned, so references to that node were removed from the screen
• The old status bits that were controlled by the INJ_TRANS node were also removed
• The STATUS indicator bits were fixed to reflect their proper
• removed some inaccurate/misleading indicators in the PCAL_END_EXC screen

I then did some test injections and confirmed that all the status bits were working as expected.  Here's an example of the screen with an active "detchar" injection (as labeled by the H1:CAL-INJ_TINJ_TYPE record):

The H1:CAL-INJ_STATUS bit word (purple indicators) behaved correctly, and reset appropriately when the EXC was stopped.