At 17:45UTC the major FMCS alarm went off for low temperatures at MY as read by H0:FMC-MY_VEA_202B_DEGF. At the time it was around 60F, but just started going back up and is at 60.2F. In the attachment you can see how much the temperature has dropped over the past day or so.

Lockloss at 17:55

Sat Aug 05 10:11:02 2023 INFO: Fill completed in 10min 58secs

WP11343

root@h1daqframes-0:~# zpool status
  pool: frames-0
 state: ONLINE
  scan: scrub repaired 0B in 3 days 21:34:56 with 0 errors on Sat Aug  5 05:46:41 2023
 

TITLE: 08/05 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 155Mpc
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 7mph Gusts, 5mph 5min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.05 μm/s
QUICK SUMMARY:

Detector has been Locked for 2 hours. We had just gone from Earthquake back to Calm. Earthquake mode was active between 14:40:40-14:50:49UTC.

If we lose lock, I will reload ISC_LOCK as requested (71970) ISC_LOCK reloaded

TITLE: 08/04 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
SHIFT SUMMARY:

- 3:01 UTC - intention bit flipped to COMMISSIONING (this happened again at 6:32), after checkng DIAG_SDF, it appears the culprit is syscssqz for both instances, though not sure what specifically (when I checked, the DIFF was gone), ss attached - Tagging SQZ

• Back to OBSERVE @ 3:03

- 3:43 - Superevent S230805x - there was NO alert for this on Verbal/Standown medm, but I did receive the automated call

- EX saturation @ 4:05

- Ryan S. is the incoming remote owl operator

LOG:

No log for this shift.

H1 has been observing for 7:45, range has hit 152 Mpc! All systems seem stable, wind is still a bit elevated (~25 mph) but looks to be going down.

Closes 25593

Most fans appear nominal, however there appears to have been a glitch on 8/1 ~9:05 PDT which can be seen on the following vibrometers: MR_FAN5_170_1/2 and MR_FAN6_170_1.

Closes 25736

Attached is the 3-month trend for the SUS HWWDs.

A bit hard to see, but after playing with the scale, ETMx had at least 1 switch, no further action needed.

Vicky, Brina,

Looking at coherences in DARM while SQZ was on and off and the PEM acc sensors in SQZT0 (image 1) and SQZT7 (image 2). We noticed the 120 hz noise is a lot more significant in T7 than T0 and we were wondering if this makes sense, since the two chambers are right next to each other. I guess, the question is do we see the 120 Hz jitter noise at the AS port? because this AS port is closest to ham7/SQZT7 & Ham6 since it is connected and that would mean there is some output jitter? and the reason we most likely don't see the noise in SQZT0 is because it is just too far from the others? (or if the source of the like random 120Hz vibrations is just farther from SQZT0 than it is from the AS port HAM chambers)

I am going to plot another set of coherences with the SQZ on/off on another day to see if we can see whether or not this 120 Hz noise goes up/down between the SQZ being on/off, it is not really clear in these first sets of plots I made.

There is also this peak around 770 Hz that is visible when the SQZ is on, and seems to disappear when the SQZ is off, I will work on getting another coherence pot  to see if the 770 Hz is in HAM7 accelerometers if SQZ_OPO_LR is LOCKED vs DOWN. 

Will follow up with Vicky in the comments.

TITLE: 08/04 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 146Mpc
INCOMING OPERATOR: Austin
SHIFT SUMMARY:

Went out of Observing a couple of times to do some commissioning, and lost lock once. Dust in PSL is currently still high due to wind.

15:00UTC Detector in Observing and Locked for 13hrs

--Commissioning Break--
16:20 Out of Observing, into Commissioning
16:21 To NLN_CAL_MEAS

16:21 Gabriele retuning Mich and SRCL FF
16:22 Ryan S. reloaded ISC_LOCK (71954)
--Commissioning Over--

16:56 Back to NOMINAL_LOW_NOISE
16:56 Observing

18:21 Lock loss (71959)

19:13 Into NOMINAL_LOW_NOISE

19:23 Back to Observing

21:06-21:15 Red dust alarm went off for the Optics Lab for particles >300nm. Went up from 70 -> 3130 pcf and back down to 10pcf

--Commissioning Break--
21:13 Out of Observing, into Commissioning
21:14 To NLN_CAL_MEAS

21:14 Gabriele testing new Mich and SRCL FF filters
--Commissioning Over--

21:24 Back to NOMINAL_LOW_NOISE
21:26 Back into Observing

21:30:26 Momentarily pushed out of Observing by DIAG_SDF for syscssqz
21:30:32 I put us back into Observing

21:32 Yellow PSL dust monitor alarm went off (dust count had slowly been creeping up)

LOG:                                                                                                                                                                                                                                                                  

TITLE: 08/04 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 148Mpc
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 30mph Gusts, 25mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.05 μm/s 
QUICK SUMMARY:

- IFO has been observing for just under 4 hours

-CDS/DMs all ok, though wind is hovering ~30 mph

In an effort to remedy the tug-of-war nature of the ETMX calibration line heights during lock acquisition (discussed by Jeff in alog 71710) and to avoid confusion about how these gains are set (alog 71945 and comments), I've created a dictionary in lscparams.py called cal_line_gains (lines 460-463) which will be the one location for these gain values. This dictionary is referenced and the values within are now used in the locations laid out in Jeff's earlier mentioned alog, keeping the line heights the same throughout the lock acquisition sequence.

Now that the main source of non-stationary noise has been removed (71927), we already have another candidate.

A DARM spectrogram shows bursts of noise below 60 Hz, happening from time to time. A better view is provided by a spectrogram of DARM whitened with the median.

There are two interesting features:

• Wide bumps that appears spaced by 10 Hz at 30, 40, 50 Hz. They look very much like scattered light
• Bursts of noise around the 60 Hz line

This morning between 9:20am and 10:00am I injected some noise to retune the MICH and SRCL feedforward. New filters have been uploaded with name '8-4-23'. Unfortunately the IFO lost lock before I was ready to test them.

We shoudl wait for the IFO to thermalize after relock, and then test the two new filters

J. Kissel, L. Dartez, T. Shaffer, R. Short

Too late to do anything about it now (this observation stretch), but Louis and I discovered that -- as a result of years of guardian code development, SDF settings preservation tactics, and calibration line height changes -- that ISC_LOCK.py plays a big tug-of-war with the amplitude until landing on the current *correct* value (set in LHO:69555) just before observe. It's in consequential to any observation segments, but it sure looks very odd when you're staring at the DELTAL ASD during the lock acquisition sequence.

It's not a priority to solve / remedy immediately, but we can and should fix it, since it's an easy fix.

The following covers the UIM line, and the code that changes it throughout the lock acquisition sequence.

Amplitude = 0
ISC_LOCK.py 
592 thru 595
PREP_FOR_LOCKING
main method
        #turn off calibration lines on ETMs so that they don't cause lockloss when the bias is ramped to 0 with the linearization on
        for sus in ['ETMX', 'ETMY']:
            for stage in ['L1', 'L2', 'L3']:
                ezca['SUS-{}_{}_CAL_LINE_CLKGAIN'.format(sus,stage)] = 0

Amplitude = 75
ISC_LOCK.py
727 thru 731
PREP_FOR_LOCKING
run method
            #make sure cal lines are on. Copied from NomLowNoise, JCD 13May2021
            for g in ['CLK','SIN','COS']:
                ezca['SUS-ETMX_L3_CAL_LINE_%sGAIN'%(g)] = 1
                ezca['SUS-ETMX_L2_CAL_LINE_%sGAIN'%(g)] = 120
                ezca['SUS-ETMX_L1_CAL_LINE_%sGAIN'%(g)] = 75

Amplitude = 20
ISC_LOCK.py
lines 844 thru 855
SDF_REVERT
main method
        self.fem_objs = [cdslib.CDSFrontEnd(model[0], model[1]) for model in lscparams.sdf_revert_models] 
        for fem in self.fem_objs:
            if fem.SDF_TABLE_LOCK:
                log('Pending changes on {}, manually verify'.format(fem.name))
                return 'CHECK_SDF'

        self.snap_file = 'safe'
        for fem in self.fem_objs:
            diff_cnt = fem.SDF_DIFF_CNT
            log('SDF diff count for {}: {}'.format(fem.name, diff_cnt))
            # Currently leaving confirm to False, and we check in run
            fem.sdf_load(self.snap_file, loadto='monitored', confirm=False)

# includes the h1susetmx model's "safe.snap" which is actually 
   /opt/rtcds/userapps/release/sus/h1/burtfiles/
      h1susetmx_down.snap
which has the value set to 20 ct.

Amplitude = 35
ISC_LOCK.py 
lines 5864 thru 5865
NOMINAL_LOW_NOISE
   >> ISC_GEN_STATES.py
   lines 260 thru 263
            for g in ['CLK','SIN','COS']:
                ezca['SUS-ETMX_L3_CAL_LINE_%sGAIN'%(g)] = 0.3
                ezca['SUS-ETMX_L2_CAL_LINE_%sGAIN'%(g)] = 50
                ezca['SUS-ETMX_L1_CAL_LINE_%sGAIN'%(g)] = 35

Will fix this at my next opportune moment. 
There's a prize of a sandwiched-sized container full of fresh-from-the-garden peppers in it for whomever beats me to it.