CO2X unlocking has kicked us out of observing 3 times in the last month 71213 each time it relocked within 1m30.

• 06/28 07:11UTC for 1m25s 
• 07/09 07:47UTC for 1m16s 
• 07/19 18:50UTC for 1m15s

Each time this is seen, the average CO2X laser temperature has risen above 24.265degC. 1 month plot attached, look at yellow H1:TCS-ITMX_CO2_LASERTEMPERATURE channel. I seen no slow trend with the PZT but it does quickly fall down towards zero before unlock.

Solutions: We could take a average of temperature over 60 seconds (as this is +/-0.1deg) and if we are in ISC_LOCK DOWN state and if it's over this, have TCS_ITMX_CO2 relock. A better solution would be to tweak chiller setpoint slightly lower, I would do this at the start of Tuesday maintenance so we have time to see any immediate issues.

Other times we've gone out of observing in the past month is for the SQZ SHG to relock for 1m50s on 07/09 71595 and 1m45s on 07/11 for FC IR unlock 71597. Tagging GRD for Automation timers.

After Gabriele made a major improvement on the DHARD Y coupling (alog 71588), since LLO was still working on relocking, I checked the A2L gains for the 3 test masses that we use for ADS during locking.  The summary is that the change seemed to be small enough that it's not worth keeping long term, so I have not modified ISC_LOCK.  I did accept the changed A2L gains for this lock, so next lock, the operator will see SDF diffs and need to accept them. 

Rather than being fancy and using the ASC as a metric for when the decoupling is good, I used 'the old way' of dithering the suspensions using the ADS and changed the A2L gains to zero the error signals. (To see the ADS error signals I had to put the gains H1:ASC-ADS_[PIT,YAW][3,4,5]_DEMOD_I_GAIN to 1, because they are set to zero by the camera servo guardian). 

The first attachment is how the ADS error signals looked before I made any changes to the A2L gains for EX, EY, or IX. This is after Gabriele adjusted IY in alog 71588. 

The second attachment is after I adjusted two of the P2L gains (the only two I ended up changing), to get the ADS error signals closer to zero.

The third attachment is the SDFs that I saved in the Observe.snap, that I did not put into lscparams/ISC_LOCK, so will need to be re-accpeted next lock at the values that guardian will set them to.

The fourth attachment is a comparison of the spectra (Brown before I changed any A2Ls, Red after), where I don't see much of a difference, but maybe Brown is better-ish anyway, which is why I'm planning to leave the A2Ls alone in the guardian.  But, we'll be able to see throughout this lock whether there is a change worth making.

TJ, Naoki

In the camera guardian, the StaticTester is checking if the cameras are frozen or not. TJ told me that there is a static_time in the StaticTester, which is an integer time in seconds to check static. We found that the default static time is 5s, which means if the camera outputs are frozen for more than 5s, the guardian thinks that the cameras are frozen. Since the short camera freeze we have experienced so far is less than 10s, we set the static time as 10s. In the StaticTester of the camera guardian, we added static_time=10 as follows.

self.cam_st = StaticTester(self.camera_chans, static_time=10)

We updated the camera guardian and reloaded it. We hope that this solves the short camera freeze issue. 

Since Vicky suggested to compare -25 Hz and -30 Hz of FC detuning, I compared it as shown in the attached figure. It seems that -25 Hz is slightly better than -30 Hz between 20-40 Hz. The nominal FC detuning is -25 Hz so I just leave it.

We removed squeezing for 10 minutes.

start: 1374004600 (2023/07/21 19:56:22 UTC)
end:  1374005215 (2023/07/21 20:06:37 UTC)

To study the upconversion effect in DARM. Analysis will follow

DHARD_Y 2.6Hz line ampl 10
start   PDT: 2023-07-21 12:29:52.313179 PDT
        UTC: 2023-07-21 19:29:52.313179 UTC
        GPS: 1374003010.313179
stop    PDT: 2023-07-21 12:34:54.886621 PDT
        UTC: 2023-07-21 19:34:54.886621 UTC
        GPS: 1374003312.886621

DHARD_Y 2.6Hz line ampl 5
start   PDT: 2023-07-21 12:35:27.167927 PDT
        UTC: 2023-07-21 19:35:27.167927 UTC
        GPS: 1374003345.167927
stop    PDT: 2023-07-21 12:40:31.112863 PDT
    UTC: 2023-07-21 19:40:31.112863 UTC
    GPS: 1374003649.112863

DHARD_Y 2.6Hz line ampl 20
start   PDT: 2023-07-21 12:40:59.351663 PDT
        UTC: 2023-07-21 19:40:59.351663 UTC
        GPS: 1374003677.351663
stop    PDT: 2023-07-21 12:46:00.319361 PDT
        UTC: 2023-07-21 19:46:00.319361 UTC
        GPS: 1374003978.319361

DHARD_Y 2.6Hz line ampl 40
start   PDT: 2023-07-21 12:46:31.743398 PDT
        UTC: 2023-07-21 19:46:31.743398 UTC
        GPS: 1374004009.743398
stop   PDT: 2023-07-21 12:51:31.519029 PDT
        UTC: 2023-07-21 19:51:31.519029 UTC
        GPS: 1374004309.519029
 
Then ramping amplitude down in 200 seconds

With logging seismic noise keeping L1 down, H1 has been using COMMISSIONING time for PEM scatter & retroreflection measurements in the LVEA by Robert, Genevieve, & Lance.

There have been MAJOR dust alarms for the PSL & Lab areas (we also had winds picking up possibly correlated to this).

There is a taco truck on-site.  #LIGOIPA2023

I retuned the ITMY P2L and Y2L gains, by injecting a DHARD line at 22 Hz in pitch and yaw. This was motivated by the higher coherence between DARM and DHARD_Y

By changing H1:SUS-ITMY_L2_DRIVEALIGN_Y2L_SPOT_GAIN from -0.15 to -1.55 I could reduce the DHARD_Y coupling by 32 db.

To see the 22 Hz line from DHARD_P I had to increase its amplitude by a factor 10 with respect to Y. The coupling was highly non-stationary, so I could not improve much. I changed the gain from -0.10 to -0.15

Coherence between DARM and DHARD_Y is gone

Fri Jul 21 10:10:43 2023 INFO: Fill completed in 10min 39secs

Jordan confirmed a good fill curbside.

I retuned the SRCL feedforward, after the implementation of the high-pass filter. This improves the SRCL noise suppression at frequencies below 40 Hz where it matters.

The new filters are FM2 FM3 FM4 all together. ISC_LOCK guardian updated and reloaded.

Laser Status:
    NPRO output power is 1.832W (nominal ~2W)
    AMP1 output power is 67.15W (nominal ~70W)
    AMP2 output power is 135.1W (nominal 135-140W)
    NPRO watchdog is GREEN
    AMP1 watchdog is GREEN
    AMP2 watchdog is GREEN

PMC:
    It has been locked 2 days, 20 hr 30 minutes
    Reflected power = 16.77W
    Transmitted power = 108.1W
    PowerSum = 124.9W

FSS:
    It has been locked for 0 days 16 hr and 43 min
    TPD[V] = 0.9384V

ISS:
    The diffracted power is around 2.7%
    Last saturation event was 0 days 16 hours and 43 minutes ago

Possible Issues: None

• Currently for Scattering w/ H1@NLN at Input Tube for roughly an hour (Robert)
• Then ~15min noise injection (Gabriele)

Thu Jul 20 10:09:16 2023 INFO: Fill completed in 9min 12secs

Gerardo confirmed a good fill via camera.

TITLE: 07/21 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 146Mpc
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 9mph Gusts, 6mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.04 μm/s
QUICK SUMMARY:

Nicely observing H1 locked for almost 13hrs.

Will keep an eye on graying/crashing cameras on nuc26 (as have been noted lately).

TITLE: 07/21 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 135Mpc
SHIFT SUMMARY:

• We've been locked and observing for 12:55, handing off to Corey
• The camera images on NUC26 keep crashing, particularly ETMX and ETMY.
• Lost connection to the dust monitors for a few seconds at 11:37UTC, white alarm, all the wind channels went down except for the CS. I restarted the wind channels at 12:15 UTC following Dave's alog.
• H0:FMC-CS_WS_RO_ALARM is cycling through active and inactive every minute or so with no alarms on Verbal, trending it shows its been doing this for a while but the cycles have sped up during the past 2 weeks, it was over 30 minutes prevously (Tagging FMCS).

LOG:

No log for this shift

Closes FAMIS 25690

Last checked in alog70511

Lots of glitches, EY pressure seems to be stepping down.

TITLE: 07/11 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC

STATE of H1: Lock Acquisition

CURRENT ENVIRONMENT: SEI_ENV state: CALM

Wind: 12mph Gusts, 9mph 5min avg

Primary useism: 0.02 μm/s

Secondary useism: 0.05 μm/s

QUICK SUMMARY:

15:11 Unlocked detector for Tuesday maintainance

20:30 Locked and Observing

   - Needed to force ALS XARM PLL Crystal Frequency

   - Needed to pause at OMC_WHITENING to damp violins

22:24-22:25 Pushed out of Observing briefly

   - Squeezer lost lock and then relocked without manual intervention

22:29 Lock loss

   - Unknown cause In process of locking back up

   - Needed to force ALS XARM PLL Crystal Frequency again

LOG:

TITLE: 07/09 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 141Mpc
SHIFT SUMMARY:

IFO is LOCKED and OBSERVING - now for 48 hours and 35 minutes and counting, O4 record!

Other:

Squeezer unlocked without unlocking full IFO, which knocked us out of observing briefly, but quickly and automatically recovered within three minutes, putting us back in observing. This resulted in a data gap in GWISTAT during this time (4:34:38 UTC to 4:37:12 UTC), which was communicated on the RRT O4 mattermost channel and on teamspeak upon clarification.

LOG: