After the glitches last night, I went to HAM3 to do the usual fixes. I powered off the corner 2 CPS, unplugged the boards in the satellite chassis at the chamber, put it all back together and powered everything back on. The CPS haven't glitched since then, so seems like maybe things are fine now. 

TITLE: 01/02 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: Corey
SHIFT SUMMARY:

- Light maintanence day, mainly VAC turbo/ion pump work and a HWS card swap from the CDS team

- Ran an initial alignment after ISC went through CHECK MICH FRINGES thrice - ran without issue

Note that there was a large tow truck on site towing a car from 22:16 - 22:25 UTC. Tagging DetChar in case this shows up on their end as noise.

• Back to NLN @ 22:12/OBSERVE @ 22:16

- Lockloss @ 23:10 - cause unknown

- H1 is currently relocking at TRANSITION FROM ETMX

LOG:                                                                                                                                                                                                                         

Both EY and EX now have Laptop charging stations in the garb rooms which contains 2 CDS laptops. These Laptops are actively being charged while they are plugged into the charging station. 
Note: these laptops will not let you login if the wifi is turned off at the End station so be sure to let the opperator know that you need wifi at the end station.

The Laptops at EX are:
cdsdell425
cdsdell437

The laptops are EY are :
cdsdell431
cdsdell285

Lockloss @ 23:10, looks like DARM IN1 LSC saw the motion first - cause unknown.

FAMIS tasks 24918 and 24942

Procedure checklist for both stations completed.  No issues were identified at this time.

MY: Scroll pump hours: 75.5

       Turbo pump hours: 210

       Crash bearings: 100%

EY: Scroll pump hours: 73.3

       Turbo pump hours: 1277

       Crash bearings: 100%

I swept the LVEA after maintenance activities completed. Everything was nominal except for the CER lights were left on.

As part of maintenance I updated the x509 certificate for lhocds, it is good for another year.

H1 is still down for maintanence work, but should begin relocking in the next 15 minutes.

Dust monitors running smoothly.

Tue Jan 02 10:06:20 2024 INFO: Fill completed in 6min 17secs

Travis confirmed a good fill curbside.

Jonathan, Erik, Dave:

Erik is replacing the failed h1hwsmsr (ITMX) V1 computer with a spare unit this morning. I have stopped the temporary HWS-ITMX IOC (started on cdsioc0 yesterday) to prevent duplication of EPICS channels. This is the cause of the EDC and SDF disconnect channel counts.

TITLE: 01/02 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 2mph Gusts, 1mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.28 μm/s 
QUICK SUMMARY:

- H1 down for maintanence day, looks like a relatively light day today

- CDS/SEI/DMs ok

Workstations were updated and rebooted.  This was an OS update. Conda packages were not updated.

TITLE: 01/02 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Tony
SHIFT SUMMARY: We lost lock once during the shift from the HAM3 ISI watchdog tripping from the CPS sensor. It then kept tripping the ISI so we're holding in down with the IMC offline and HAM3 in damped until the problem can be addressed in the morning during maintenance.

02:20 Lockloss alog75114

03:20 Back into observing, pretty quick automated relock

06:19 lockloss alog75119

06:24 the HAM3 ISI tripped from a CPS glitch it seems? Then 10 minutes later it tripped with a larger glitch at 06:34

DIAG_MAIN reports

•  SEI_STATE: ISI ['HAM3'] WD saturation count greater than 75% of max which is from the CPS sensors, and later these CPS sensors on HAM3 tripped the ISI. 
• HWS ITMY code has stopped

Lockloss

We've been locked just under an hour

We lost lock at 02:20UTC, relocking was straightforward, back to NLN at 03:14 in observing at 03:20. 

Lockloss at 02:20UTC, HAM3 ISI watchdog trip from the cps sensors? Seismic motion was increasing at the time. CPS_H2 sees a big spike which presumably was the trip.

TITLE: 01/01 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 150Mpc
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 6mph Gusts, 4mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.38 μm/s 
QUICK SUMMARY:

• We've been locked for 4.5 hours, in observing since 19:31UTC
• 23:48UTC DIAG_MAIN reports that SEI_STATE: ISI ['HAM3'] WD saturation count greater than 75% of max which is from the CPS sensors