TITLE: 07/03 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 145Mpc
CURRENT ENVIRONMENT:
    SEI_ENV state: SEISMON_ALERT
    Wind: 18mph Gusts, 14mph 5min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.10 μm/s
QUICK SUMMARY:

Looking like we are riding through the Alaska M4.6 EQ (Austin was worried we were going to have another lockloss at a transition with our recently-ailing H1), but we are good (Seismic went to CALM as I'm typing!).  Winds recently touching 20mph, but other than that it's all par for the course.

TITLE: 07/03 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 146Mpc
SHIFT SUMMARY:

- Arrived with H1 in observing as of 4:37

- EX saturation @ 8:09

- 14:51 - inc 4.7 EQ from Alaska

Quiet night for H1, all appears stable. Leaving H1 in observing to Corey, going on 10.5 hours.
LOG:

No log for this shift.

H1 has been observing for just under 6.5 hours. Ground motion and wind speeds are low, violins have been damping well, and the IFO appears stable.

TITLE: 07/03 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 144Mpc
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 9mph Gusts, 7mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.09 μm/s
QUICK SUMMARY:

- H1 has been locked and in observing for just under 2.5 hours

- CDS/SEI/DMs ok

TITLE: 07/03 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 146Mpc
SHIFT SUMMARY:

• Violin ringups causing trouble again
• Issues with the BS/ Aligning mich fringes
• Locked and Observing for 2:23 as of 07:00UTC

Lock#1:

Lockloss at 23:06

Lock#2:

During DRMI_1F ISC_LOCK went into error again, from the same bit of code, so I again reloaded the guardian and it was fine. Guardian went to check_mich next but lost lock during it.

Lock#3-4:

Couldn't get DRMI or PRMI again, went to check _mich again where guardian complained about the BS alignment, tried twice and it lost lock during the 2nd attempt. I'm decided to do an initial alignment, which went well until I couldn't get Mich_bright to align, the spot look fine and had good symmetrical flashes. I cut my loses and went back into regular locking after some tapping of the BS to try and maximize the ASC_AS_A_DC_NSUM_OUT signal, the flashes looked much better now and PRMI was able to lock but DRMI was still having trouble. During the second attempt at DRMI I tapped on the BS a little bit in Pitch and it was able to lock (finally). Once we got to OMC_WHITENING I had to damp violins for an hour, once we got to NLN and I went into Observing, we lost lock 34 seconds later with a DCPD saturation at 01:58.

Lock#4-5:

• Lost lock at LOCKING_ALS
• No issues this time, straight through ALS, IR, DRMI. Held up at whitening again, this time after the threshold was reached I set a timer for 30 minutes then I was able to bring us back to NLN and Observing at 04:37 UTC.
• A few minutes before Vicky noticed the SQZer was complaining about AS2 so I did a graceful clear history from the SQZ_ASC_IFO screen at her direction and it fixed the issue. Possible related to alog 70890

LOG:

No log for this shift

We've had two locklosses from NLN this shift, I'm currently stuck at OMC_WHITENING damping violins again, I'm going to give it an extra half hour after they reach the threshold just to be safe.

We lost lock less than a minute after reaquiring NLN and Observing after about an hour of violin damping at 01:58UTC, very sad. DCPD saturation a second before the LL. Maybe I went into NLN too quickly after turning on the whitening? Guardian was requested to NLN so it went as soon as it was able to, I'll change the request to OMC_WHITENING so I can give the IFO another half hour or so to chill out on this next aquisition.

Lockloss like 5 minutes after I sat down, EX saturation 2 seconds before the lockloss at 23:06 UTC. No obvious cause, ground motion isn't very high from the EQ.

TITLE: 07/02 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 134Mpc
CURRENT ENVIRONMENT:
    SEI_ENV state: SEISMON_ALERT
    Wind: 8mph Gusts, 5mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.08 μm/s
QUICK SUMMARY:

• Currently getting hit by a small EQ from Alaska
• CDS/Vac/Dust/Violins looks good

TITLE: 07/02 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
SHIFT SUMMARY:

Other than the ETMy camera issue at the beginning of the shift (see earlier alogs), it has been the type of shif you want.  There was even a new GW Candidate (S230702an).  As I type and was talking with Ryan C, we did get notice of a Canadian/Alaska EQ (but doesn't look too bad **knock on wood**). 

Rahul did TeamSpeak in to say we can text/email him if there's a lockloss and we want violin damping assistance.

LOG:

• 1541 Worked with dave regarding ETMy camera
  • 1557 ETMy camera restored & Camera Servo guardian was finally able to get to the CAMERA_SERVO_ON state (making the ISC LOCK happy to go to NOMINAL LOW NOISE)
• 1601 Back to Observing.

Sun Jul 02 10:05:17 2023 INFO: Fill completed in 5min 16secs

At 03:34 PDT this morning the image of h1cam27 went blue-screen and its centroids froze.

This camera required a full power cycle recovery, the process was:

Initial state: camera is pingable but there is no image.

Attempt to use monit to restart the camera process on h1digivideo2, no image

As root, kill -9 <cam27pid>, get monit to restart the process, no image

Log onto EY POE switch (sw-ey-aux) and power cycle h1cam27 by bringing port 0/9 to shutdown, wait 1 minute, bring it back. 

When h1cam27 is pingable again, check image: no  image

On h1digivideo2 kill -9 the process, get monit to restart it, image restored.

Posting more info via the CDS/H1VID_DIGITAL_OVERVIEW medm. 

• ETMY camera = h1cam27
• h1cam27 located at h1digivideo2
• There are web monitoring pages and they say it's "running".  And there are also buttons to "restart service" and other choices, but I am going to leave addressing this to others since I've not found instructions on this.
• Calling Dave B now.

TITLE: 07/02 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 18mph Gusts, 13mph 5min avg
    Primary useism: 0.06 μm/s
    Secondary useism: 0.07 μm/s
QUICK SUMMARY:

Just received status from Austin, and as he noted, we are currently stuck at NLN (not for violins) due to the CAMERA SERVO guardian having an issue (which is related to the ETMy camera crashing (we have a blue screen on the FOM----attached is a screenshot showing a pile of relevant windows to this).  I'll probably do a quick search to learn more, but will probably be making phone calls soon to address this. 

Oh and it looks like winds have started picking up in the last hour.

TITLE: 07/02 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Corrective Maintanence
SHIFT SUMMARY:

- 10:22 - Went into Commissioning, CAMERA SERVO guardian went into WAIT FOR CAMERA due to the ASC CAM PIT/YAW 3 cameras freezing - sdfs attached

• Also noticed that the ETMY camera is now down, probably related since ETMY feeds in PIT/YAW 3 - ETMY camera does appear to be on and running, but showing up blue screnned on nuc26
• Tried letting the CAMERA SERVO guardian go through WAIT FOR CAMERA a few times before intervening
• Tried running through INIT to try and rerun through the code again - no dice - Tagging CDS
• IFO lost lock from EQ before I could find a solution

- 10:34 - inc 6.7 EQ from Tonga

• 10:39 EQ mode activtated
• Lockloss @ 10:40

- Holding in DOWN to ride out the ground motion

• INCREASE FLASHES failed 2x for ALS Y so I manually intervened during IA
• Same problem as Corey, I could not get PRC or SRC to align after multiple attempts trying to move BS and SRM
• Going to try skipping and going straight up ISC and see what happens
• Locking went surprisingly smooth - holding off at OMC WHITENING to damp violins

- Acquired NLN @ 14:15, but the ASC CAM PIT/YAW 3 on the CAMERA SERVO are still stuck due to ETMY cam failure and we cannot go into observing - will investigate

• On the camera process overview monit, I tried restarting the process, which failed, stating "Execution failed"

- Unfortunately not able to figure out how to restart the camera in time for the operator transition, though I believe it might be a quick fix by contacting Dave

LOG:

No log for this shift.

Lockloss @ 10:40 due to 6.7 EQ from Tonga.

Plot 1 shows the dark noise in LSC-REFL_A_LF and REFL_B_LF (yellow/black), with 10W laser input with ISS second loop (red/blue) and w/o ISS second loop (magenta/cyan). The photocurrent in REFL_A and REFL_B was about 16mW each for the 10W measurements. It will turn to about 10mW in full lock. So, we are now about a factor 4 above dark noise.

Plot 2 shows the RIN of the 2 REFL PDs and their average, together with the ISS second loop sensors. The ISS inner and outer sensors have about 7-8mW of light each, so the measurements are limited by the shot noise of the inner PD. In full lock these sensors see about 60mW.