I ran Jenne's rubbing script using a reference time of us being in DOWN on the 22nd before the fuzziness started on the 23rd around 07:00UTC and a check time of 10:00UTC on the 23rd while also in DOWN, I didn't really see anything noteworthy, some noise increase in the 0.1Hz area (microseism)?

I also ran using the same reference time against a time this morning and there was a couple of suspensions that looked interesting, there were some spikes in the 1.0 - 10 Hz bandwidth for PR2, PR3, PRM, SR3, SR2, MC3, MC1, and OM3. In the 0.0 - 1.0 bandwidth the Quads saw an increase in noise, specifically the ITMs, TMSs, and ETMY. SRM and SR2 have some large spikes just under 1 Hz. The ITMs temperature has slightly increased, ~ 0.25 - 0.3 degrees F but these changes are less than 0.5 degrees so its probably not a worry.

Later in the day I ran the script a 3rd time using a time from yesterday (13:54UTC 11/29) with the same reference from the 22nd and most of the features I saw are no longer there, except for ITMX. Maybe the time I chose earlier was just a bad time?

IFO is locking.

Difficulties with microseism locking persist (haven't been able to lock PRMI yet).

We stayed in idle for a bit to allow people to take some measurements (and also while Robert was near the BRS).

Just started our hopefully valiant second attempt.

Addressed TCS Chillers (Wed [Nov29] 415-430pm local) & CLOSED FAMIS #26162:

• TCSx Chiller:  Had level of 29.0 cm.  Added 330mL for 29.9cm.
• TCSy Chiller:  Had level of 9.4 cm.  Added 170mL for 9.9cm.
• There was no water in the small cup under the slow leak.

As suggested by Richard/Robert to look for electronics issues, I trended the SUS OSEMs before and after start of DARM Range Fuzzy period 2020/11/23 7:00UTC, between 6:00UTC and 8:00UTC and zoomed into [s-trends]. Previous troubleshooting in 74455, 74377, 74383.

Trended ZMs, IMs, RMs, OMC, OFI, OM1,2,3; ETMs, ITMs, @ M0,L2, L3; BS @ M1, L2, L3; MC1,2,3, SMs, PRs, FCs @ M1, M2, M3

I saw nothing change at the 2020/11/23 7:00UTC time. Only thing of note is that OFI, OM2, and ZM4,5,6 has louder periods every ~15 seconds, plots attached of ZMs and OM2. This is not a new feature.

The pressure gauge for X2-8 stopped reporting pressure about 8:00 PM last night, this is a known issue due to the lack of sunlight needed to charge batteries via solar panels. No action is needed at this time.

Several of the low ambient hoods located on the condensing units of the Mitsubishi cooling systems for the server rooms have ambient air sensors which are reading incorrectly. This causes the hoods to keep the dampers open too much which results in lower head pressure which in turn leads to inefficient cooling. I measured the ohm value on these sensors and compared them with the correct ohm value correlating to the correct outdoor air temp and then added resistors to increase the ohms to correlate with the correct air temp. I will replace these sensors once we get a hold of new ones. 

Thu Nov 30 10:04:16 2023 INFO: Fill completed in 4min 13secs

Travis confirmed a good fill curbside. Even with the new trip temps of -80C this one barely sqeaked by, with TC A,B mins of -83.0C and -77.6C respectively.

TITLE: 11/30 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 3mph Gusts, 2mph 5min avg
    Primary useism: 0.10 μm/s
    Secondary useism: 0.94 μm/s
QUICK SUMMARY:

IFO has been trying to lock DRMI (and failing) for about 3 hours (due to microseism).

PSL DUST 102 still acting up per previous alogs (74421).

Attempting to continue locking.

H1 lock losses (caltech.edu)

We haven't been able to relock following this LL, Yarm is very unstable and keeps losing it. Lots of low state LLs, FIND_IR, DRMI(mostly), PRMI, GREEN_ARMS

H1 called for assistance at 14:39UTC, From the DRMI timer expiring after an initial alignment had already been run. The DRMI locking issues are likely from the elevated microseism.

TITLE: 11/30 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 140Mpc
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Struggled early in the shift to get DRMI to lock, but once successful, everything else ran smoothly.

• H1 finishing initial alignment
• 00:16 Starting lock acquisition
  • Took a while to get DRMI locked, see alog 74487 for details
• 02:56 Reached NLN
• 03:10 Started observing

H1 has now been locked and observing for 5 hours. If we lose lock, it may be challenging to relock with the elevated microseism.

LOG:

H1 is back to observing at 140Mpc as of 03:10 UTC.

The biggest struggle with lock acquisition following the challenges of the day came at the DRMI locking steps. The POP18 and POP90 flashes have been consistently low recently, so ISC_LOCK went through PRMI to start, which was successful, but PRMI ASC wan't able to make the signal much better. When moving on to try and lock DRMI, I paused ISC_LOCK to give myself time to try a few things. After a while of adjusting triggering thresholds and LSC servo gains, the settings that finally worked to keep DRMI locked enough for me to adjust alignment and engage DRMI ASC were:

• LSC-MICH2_GAIN: 1.0
• LSC-PRCL2_GAIN: 1.2
• LSC-SRCL2_GAIN: 1.4
• MICH_TRIG (upper): 25.0
• SRCL_TRIG (upper): 25.0

After DRMI ASC was engaged and offloaded, I continued locking by resuming ISC_LOCK. I also set the LSC servo gains back to their nominal value of 1.0. The rest of lock acquisition went smoothly and unaided.

That said, BNS range is still on the low side and Camilla pointed out that the low frequency noise she noted in alog 74473 is back. Also, the secondary microseism is well above the 95th percentile and still rising, so things are unstable.

Naoki, Sheila, Camilla

Sheila/Dave created a new SQZ_ANG_ADJUST guardian yetserday 74454 and today Naoki and I moved the ADJUST_SQZ_ANG_ADF and ADJUST_SQZ_ANG_DITHER states that were added to SQZ_MANAGER in 74428 to it, graph attached.

Now SQZ_MANAGER's nominal is set back to FREQ_DEP_SQZ and within this state, SQZ_ANG_ADJUST is requested to it's nominal ADJUST_SQZ_ANG_ADF as long as sqzparams.use_sqz_angle_adjust = True.

Nothing should be different in the working of the system apart from the nominal guardian states, tagging OpsInfo.

We had to restart SQZ_MANAGER as we deleted some of it's states, I tried to request SQZ_READY_IFO before reopening the medm and actually requested ALIGN_HD which offloaded ZC5,6 ASC and took the ZMs, FCs and PSAMs to Homodyne settings. After reverting these and SQZ FC green locked so alignment should be fine.

Dana, Sheila

I plotted how the DCPD matrix has changed over the last eight years in the attached figure (DCPD_sums_and_nulls_8years). I've also included zoomed in snapshots of all the major changes within this time span with the dates and times shown (changes 1-7 attached below). For each one of these jumps in the matrix values, I checked the alogs for that day and attached any potentially relevant ones explaining what was going on at the time below:

change 1 (1) (2)

changes 2 & 3 (1) (2)

change 4 (1)

change 5 (1) (2)

change 6 (1)

change 7 (1) (2) (3) (4) (5)

TITLE: 11/29 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 0mph Gusts, 0mph 5min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.72 μm/s
QUICK SUMMARY:

H1 is running an initial alignment after hopefully having fixed the locking issues from earlier today. See alogs 74472, 74478, 74480, and 74483 for a rundown of efforts made thus far.

Lockloss @ 01:39 UTC - no obvious cause, online lockloss analysis failed.

Looks like PRCL saw the first motion by a very small margin.