Yesterday Jim and Fil peeked into the BRSX thermal enclosure and since then the BRS has drifted out of range (see attached). Based on the thermocouple readbacks, it looks like the enclosure has cooled down. Jim thinks that he might not have closed up the enclosure well enough and the heater is already at 9V out of its max of 10V, so he cant add too much more heat. The current plan is to recenter the BRS during our commissioning break today and maybe try to get the heater in a more central voltage location to allow for more future adjustment.

In the mean time, we will have no BRS sensor corrected signal for EX. DIAG_MAIN and the SEI guardians have notifications from this automated process, but we might be more vulnerable to earthquakes during this time.

TITLE: 09/06 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 146Mpc
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 1mph Gusts, 0mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.11 μm/s
QUICK SUMMARY: Locked for 6.5 hours. There was a lock loss and an automatic relock during the last shift, nice.

CDS Overview OK, Dust monitors OK, FOMS OK

TITLE: 09/06 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 150Mpc
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY:

One lockloss at 04:43 UTC

1. While the cause is unknown, I it happened as soon as a DCPD glitch happened (appeared on DARM). Three similar glitches happened before this but did not cause a lockloss.
2. Unlike yesterday, the lock acquisition was not fully automatic:
   1. Had to move EY due to ALSY not catching after watching it for a bit. I moved YAW significantly (about 9-12 cts) and PIT about 3-6 counts while looking at the camera
   2. DRMI did not catch and we went to PRMI, which then had difficulty and went to check mich fringes.
   3. PRMI was very unstable once locked and would cycle between locking and losing lock. To tackle this, I moved the BS YAW by 9 counts (3 taps of 3) when it was locked since I could see the two resolved beams not well overlayed on the AS AIR Camera. This worked.
3. After this, DRMI locked smoothly and locking was fully automatic.
4. NLN Achieved at 06:10 UTC and OBSERVING at 06:24 UTC (with 14 minutes to wait for servo, consistent with last few lock acquisitons - alogs show 10-25 minutes generally).

Nothing else of significance to report

LOG:

None

DRMI unlocked causing a lockloss - cause of that is unknown so far but it seems to have correlated with a DCPD glitch (3 so far have happened during shift but this one may have caused a lockloss)

IFO is in NLN and OBSERVING as of Sep 5 22:37 UTC (same lock as before - now at 4:42 hrs).

Nothing else to report.

Output mode cleaner tube turbo station;
     Scroll pump hours: 5559.7
     Turbo pump hours: 5603.4
     Crash bearing life is at 100%

X beam manifold turbo station;
     Scroll pump hours: 778.0
     Turbo pump hours: 782.1
     Crash bearing life is at 100%

Y beam manifold turbo station;
     Scroll pump hours: 1877.0
     Turbo pump hours: 595.4
     Crash bearing life is at 100%

FAMIS tasks 23522, 23642 and 23594.

WP11409 HPI HAM1 fix model path

Jim, Dave:

A new h1hpiham1 model was installed. No DAQ restart was required.

Fix h1digivideo3 memory leak

Patrick

Patrick installed new video server code on h1digivideo3. At the time of writing the memory leak has been fixed.

Checked wind fences this morning during maintenance, nothing new to report.

X2-8
   Solar panels are mostly clear, all cables appear good.

   Each battery was checked with a multimeter:
      Right battery 13.58 VDC
      Left battery 13.70 VDC

   The LCD readout at the charge controller showed for both batteries 27.3 VDC, measured with clear skies.

Y2-8
   Solar panels are mostly clear, all cables are good.

   Each battery was checked with a multimeter:
      Top battery 13.16 VDC
      Bottom battery 12.99 VDC

   The LCD readout at the charge controller showed for both batteries 26.1 VDC, measured with clear skies.

FAMIS task 22075.

TITLE: 09/05 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 147Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 9mph Gusts, 4mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.11 μm/s
QUICK SUMMARY:

IFO is in NLN and OBSERVING as of 22:37 UTC (1 hr lock)

TITLE: 09/05 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 147Mpc
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY: Lighter maintenance day activities, but some ITMY CPS glitching and card replacement kept us from continuous observing. Relocks have been autonomous.
LOG:

• 2057 Observing
• 2110 Out of observing from ITMY stage 2 trip (more CPS woes)
• 2130 Lock loss, intentional to fix ITMX CPS
• 2237 Observing

Picket fence update:

TL;DR,

-Latency is down and we're using pickets that should be more reliable for early warning.

-The picket fence now downloads some data from the PNSN server and therefore will only work properly from opslogin0.

-We added proposed heartbeat EPICS channels H1:SEI-USGS_SERVER_GPS and H1:SEI-USGS_SERVER_START_GPS

Full details are in the SEI log 2275  git pull to get the latest version and contact me if something does not work.

Back after another CPS card swap.

Closes WP 11403.

Updated to 0.1.11. This is a patch to free memory allocations that were previously overlooked until spotted by Jonathan and EJ. So far the memory usage on h1digivideo3 appears much steadier.

The HAM1 model was restarted this morning to get fix for the issue I found on Friday. After bringing the HEPI back up I tested some configurations and adding the 3dl4c ff path to the twist sum reduces the measured low frequency bending motion cause by the ff drive. Attached plot compares a segment of time from last night (red and blue) before the model restart and this morning after (green and brown) while TJ was relocking. The blue trace shows the excess noise caused by running the 3d ff around .1-.5hz. This excess is gone in the green, and is basically the same running without the ff now. This isn't really an issue for the other HEPI, but it would probably be a good idea to extend this fix from the friday alog to all the other HEPI models.

Back to observing at 2057UTC.

Maintenance recovery was slowed by ITMY CPS glitching (alog72683) that Jim fixed by replacing a CPS card. This was also the cause of some lock losses last night.

J. Kissel, J. Warner

More details to come, but after trending around this morning, Jim found that Ryan's issues with lock losses last night were coincident with H1:ISI-ITMY_ST1 and ST2 capacitive position sensor (CPS) glitching as loud as ~4 micron jumps in 65-100 Hz band limited RMS.

Attached are trends of the new-ish band-limited RMS channels installed in April 2023 (see CPS BLRMS ECR installation mentioned in LHO aLOG 68798, and design in SEI aLOGs 1849 and 1867) compared against the ISC_LOCK_STATE_N, where 600 in NOMINAL_LOW_NOISE.

The 2nd row of both plots show the ST1 and ST2, H2 and V2 CPS BLRMS trends for both the 65-100 and 130-200 Hz bands. One can see the loudest of glitching in these sensors, where the y-axis is in nano meters.

These 4 sensors are all in the corner two "rack," but each sensor has its own readout card. 

The loud glitches correspond to times of ITMY watchdog trips too, which is what Ryan saw in LHO aLOGs 72671 and 72666.

Jim is swapping out some of the cards as I type, and will aLOG accordingly.

I've written a python program to generate a H1CDS_PICKET_FENCE.adl MEDM (see attached). This can be opened from the SITEMAP as the last entry in the SEI pull-down.

All the non-string PVs can be trended using the DAQ.