(Dave Barker, Fil Clara, Jenne Driggers, Corey Gray, Gerardo Moreno, plus those who signed work permit and whom I phoned last night)

Last Night Recap:
Last night at 959pmPT, there was an audible alarm for the HAM6 vacuum gauge (PT110).  Gerardo went out to check to see if it was dead.  He phoned Fil and they did a power cycle of the gauge to see if this would improve the situation, but it did not.  After this, H1 was taken to IDLE for the night (OWL shift cancelled).  (Should have taken Observatory Mode to CORRECTIVE MAINTENANCE at this point, but only remembered to do this this morning.)

Work Permit 12260 generated.  Fil arrived on-site around 9amPT (17utc).

This Morning's Summary of Fil's Work:

The HAM6 Interlock (for operations of the Fast Shutter + PZT) is now Using the BSC3 Interlock (or VAC gauge from BSC3 since it was decided to BYPASS the HAM6 VAC gauge which failed last night [alog 82005]).  This means that the BSC3 Interlock is no longer being used for the ITM ESDs (which sounds like it's fine because Jenne/Fil said we don't use the ESDs for the ITMs).

Fil noted that he confirmed that the Fast Shutter Chassis in the LVEA is OPERATIONAL.  Fil then left the site after giving me this summary.

As far as a test of this new HAM6 Interlock configuration, I intentionally ran a test of the AS Port Protection system (sitemap/SYS/ AS Port Protection....this was my first time doing this).  After clicking the RUN (test) button, after a few seconds received an "OK" for the (1) Test & (2) Fault.  After this test and the signed-off Work Permit, we are OK to GO for High Power locking.

Currently running an Initial Alignment (note:  ISC_LOCK initially went into ERROR [red box] when I selected Initial Alignment (screenshot of error message attached).  LOAD-ing ISC_LOCK (suggested in the Log) cleared this up.

ALSO: 

• Attaching a screenshot of the AS Port Protection medm
• CDS Alarm box on the CDS Overview remains (related to the lingering issue of the failed PT110 gauge).

TITLE: 12/27 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Aligning
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 10mph Gusts, 8mph 3min avg
    Primary useism: 0.07 μm/s
    Secondary useism: 0.69 μm/s
QUICK SUMMARY:

Filibert has been on-site this morning to work on bypassing the failed vacuum gauge at HAM6 to get the Fast Shutter interlock operational again (just received a summary of his work).

Next step is to wait for the Work Permit to get signed for high power operations and then return to locking.

Microseism is still high but only light winds (but basically this is the same state we were in last night where H1 had troubles getting past TRANSITION FROM ETMX.  SEI CONFIG is in the USEISM state.

Since the HAM6 gauge which failed at 10pm local time is used as part of the interlock system for the Fast Shutter, Jenne made the call to have the H1 down for the night (Owl Shift Canceled)

• H1 in IDLE with the IMC Locked.

Tomorrow we will need to figure out the plan forward for H1 operations.  Whatever the fix will require a Work Permit and will need to figure out availability for who can do this work.  

TITLE: 12/27 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Wind
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY:

Bit of a frustrating shift with H1 losing lock most of the time at the same point late in the ISC_LOCKing scheme (alog 82000).  Plus dealing with high seismic motion (microseism is still around 1um/sec).  No alignments were run or needed this shift.  With locking, made it fairly repeatedly all the way up to around TRANSITION_FROM_ETMX / LOWNOISE_ESD_ETMX for majority of shift's locklosses.  For relocking, DRMI generally looks horrible, but PRMI does the trick for continuing on beyond DRMI---hence no Initial Alignments have been run this evening.

And shift ends with a M5.9 mid-Atlantic earthquake rolling through.
LOG:

• 0048-0115+ Site Walk thru (tony)
  • Tony noted that in the H2 Room there was a Test Stand computer which was beeping & had a red light flashing (hadn't heard this on previous walk-thrus).
• 0055 LOCKLOSS at LOWNOISE_ESD
• 0314 Observing
  • 0325 LOCKLOSS
• 0424 LOCKLOSS @TRANSITION_FROM_ETMX
• 0534 LOCKLOSS @LOWNOISE_ESD_ETMX
• 0546 Incoming Earthquake (M5.9 mid altlantic )
• Various ALS & DRMI locklosses to end the shift.
• 0559 VACUUM Alarm Handler alarm for LVEA's PT110.
  • Looking at CDS Overview:  red VAC & CDS Alarm blocks
  • Dave contacted me via Teamspeak to let me know about the above HAM6 alarm PLUS an alarm for BSC3 (which is not as clear and Dave is still investigating)
  • I contacted Janos to apprise him of these alarms.  (he thought we lost the gauge for HAM6 and said that BSC3 might be a glitch (since no other gauges were alarming).  At any rate, he was going to contact Dave.
  • Coincidentally, Gerardo just happened to arrive within 20min of these alarms (he was in the area because he needed to pick up a Yubikey!  So now he can take a look at these gauges and chat with Dave!
  • Also been in contact with TJ, Jenne, and Fernando.

At 21:58 VACSTAT issued a BSC3 alert, which looks like it could be a sensor spike. 1 minute later PT110 (HAM6) sensor went offline to flat 0.0Torr.

Can not look at the:

• H1 Lockloss page (https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/)
• H1 Summary Page (https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/)
• H1 Cal page (https://ldas-jobs.ligo-wa.caltech.edu/~cal/)

I have been able to look at these pages for L1 with no problem, but for the above, they just time out after a few minutes of trying to load their webpages.

Noticing that all of these are ldas-jobs.ligo-wa, I looked for that and found something via monitor.ligo.org, which led to the CUBE visualization and then clicked on Problems, and under here are several Host & Service problems.  For "ldas-jobs-lho" it has had issues for the last 19hrs (or since midnight last night.

As the winds have finally become manageable, H1 has 2-locklosses in a row at beginning (within 2secs) of starting the LOWNOISE_ESD_ETMX state.  (this is after h1 being down for most of the day due to winds [which are now lower], AND with high microseism which still is high)  Alignment looks good after the afternoon alignment run by Tony toward the end of his shift.  DRMI has looked a bit ugly, but PRMI has repeatedly locked (albeit ugly-looking at first) and had ISC_LOCK with smooth sailing....until....LOWNOISE_ESD_ETMX.  Will pause at TRANSITION_FROM_ETMX for this next lock attempt.

TITLE: 12/27 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Wind
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 20mph Gusts, 15mph 3min avg
    Primary useism: 0.07 μm/s
    Secondary useism: 0.78 μm/s
QUICK SUMMARY:

Drove in to some very strong winds (see attached), but they have since calmed down such that (after an alignnment), Tony has gotten H1's ISC_LOCK beyond PRMI!  (currently at ENGAGE ASC).  In addition to the very windy last-9-hrs, the secondary µseism took a turn for the worse in the last 24hrs (but if you squint you might see it drift down over the last 6hrs).  With all this said (lower winds and µseism), here's hoping H1 and the earth can conspire for a return to Observing!

[H1 currently powering up (woo woo!), AND violins are looking fine---not rung up for this first look for H1 since the last lockloss 10hrs ago.]

TITLE: 12/27 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Wind
INCOMING OPERATOR: Corey
SHIFT SUMMARY:

Wind storm started early in the morning with a high microseism.
I held the IFO in Down until the wind stayed below 35 for 10 minutes. Then tried to lock.

I did take Shiela's advise and change the H1:ASC-POP_A {PIT & YAW} OFFSETs and I saved them in the both the Safe and Observe.snaps to try to get better stability through some locking states.

Seems like we are on the back end of this wind storm, Wind fence is still intact.
GRB-Short E538969 @ 22:34 UTC

 

Thu Dec 26 10:12:57 2024 INFO: Fill completed in 12min 53secs

TCmins [-141C, -72C*] OAT [+11C, 50F]

* TB-B shown because it did go below trip-temp of -65C.

TITLE: 12/26 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Aligning
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 36mph Gusts, 26mph 3min avg
    Primary useism: 0.14 μm/s
    Secondary useism: 1.05 μm/s
QUICK SUMMARY:
H1: Unlocked
When I arrived ISC_LOCK was in Initial_alignment.
Winds are gusting above 40MPH.
Once Initial_alignmnet is finished, I'll hold in down for an hour as this wind storm passes. I'll wait for the tumbleweeds to get to where they are going.

Useism has climbed to above 1um/s and winds are above 20mph. We have been unable to keep green arms locked for long enough to complete an initial alignment for the past 1.5 hours...but then it just suddenly worked. We've made it to engaging ASC for full IFO, I'll setup H1_MANAGER again.

TITLE: 12/26 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 161Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY:

Eve Shift:  Christmas Evening Edition
After the busy day shift, the evening was pretty calm.  H1's been locked most of the shift (currently at 6.25hrs)

Sadly, the secondary microseism is now above the 95th percentile at about 0.8um/s.
LOG:

• 0051 Gerardo arrives on site for VAC investigations (Tony letting him through gate)
  • ~0217 this is when Tony signed off and mentioned Gerardo leaving site as well.
• 0145-0148 Sqz bumped H1 from Observing to relock.

TITLE: 12/25 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY:

Christmas Day Lockloss #1:
Happened on the OWL Shift last night. Very pecilar, Small-ish Vacuum pressure increase at EY happened seconds after the lockloss. Pressure increase seen at Mid Y as well.
Persons involved: Dave, Daniel, Gerardo, Sheila.
Seems like the Ion pump failure caused the lockloss and the time delay between the lockloss and the increase in pressure was just the speed of the gasses moving through the chambers.
Gerardo came out to site to check the Vacuum pressure & Ion pump at EY. I've also eneded up coming out here as well as a safety buddy.

Christmas Day Lockloss # 2:

Christmas Day Lockloss # 3
 

Relocking was fully automated except my request for an inital_alignment after the Earthquake.

Sheila, has asked me to try to update the H1:ASC-POP_A_YAW_OFFSET & PIT, for better stability in the while locking in CARM_TO_REFL & Resonance regions. I will try to do this tomorrow during the next lock. Since we are already locked now.

LOG:
Gerardo heading to EY and is still out there as I write this alog, I'll be onsite until Gerardo leaves.