Since the HAM6 gauge which failed at 10pm local time is used as part of the interlock system for the Fast Shutter, Jenne made the call to have the H1 down for the night (Owl Shift Canceled)

• H1 in IDLE with the IMC Locked.

Tomorrow we will need to figure out the plan forward for H1 operations.  Whatever the fix will require a Work Permit and will need to figure out availability for who can do this work.  

TITLE: 12/27 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Wind
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY:

Bit of a frustrating shift with H1 losing lock most of the time at the same point late in the ISC_LOCKing scheme (alog 82000).  Plus dealing with high seismic motion (microseism is still around 1um/sec).  No alignments were run or needed this shift.  With locking, made it fairly repeatedly all the way up to around TRANSITION_FROM_ETMX / LOWNOISE_ESD_ETMX for majority of shift's locklosses.  For relocking, DRMI generally looks horrible, but PRMI does the trick for continuing on beyond DRMI---hence no Initial Alignments have been run this evening.

And shift ends with a M5.9 mid-Atlantic earthquake rolling through.
LOG:

• 0048-0115+ Site Walk thru (tony)
  • Tony noted that in the H2 Room there was a Test Stand computer which was beeping & had a red light flashing (hadn't heard this on previous walk-thrus).
• 0055 LOCKLOSS at LOWNOISE_ESD
• 0314 Observing
  • 0325 LOCKLOSS
• 0424 LOCKLOSS @TRANSITION_FROM_ETMX
• 0534 LOCKLOSS @LOWNOISE_ESD_ETMX
• 0546 Incoming Earthquake (M5.9 mid altlantic )
• Various ALS & DRMI locklosses to end the shift.
• 0559 VACUUM Alarm Handler alarm for LVEA's PT110.
  • Looking at CDS Overview:  red VAC & CDS Alarm blocks
  • Dave contacted me via Teamspeak to let me know about the above HAM6 alarm PLUS an alarm for BSC3 (which is not as clear and Dave is still investigating)
  • I contacted Janos to apprise him of these alarms.  (he thought we lost the gauge for HAM6 and said that BSC3 might be a glitch (since no other gauges were alarming).  At any rate, he was going to contact Dave.
  • Coincidentally, Gerardo just happened to arrive within 20min of these alarms (he was in the area because he needed to pick up a Yubikey!  So now he can take a look at these gauges and chat with Dave!
  • Also been in contact with TJ, Jenne, and Fernando.

At 21:58 VACSTAT issued a BSC3 alert, which looks like it could be a sensor spike. 1 minute later PT110 (HAM6) sensor went offline to flat 0.0Torr.

Can not look at the:

• H1 Lockloss page (https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/)
• H1 Summary Page (https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/)
• H1 Cal page (https://ldas-jobs.ligo-wa.caltech.edu/~cal/)

I have been able to look at these pages for L1 with no problem, but for the above, they just time out after a few minutes of trying to load their webpages.

Noticing that all of these are ldas-jobs.ligo-wa, I looked for that and found something via monitor.ligo.org, which led to the CUBE visualization and then clicked on Problems, and under here are several Host & Service problems.  For "ldas-jobs-lho" it has had issues for the last 19hrs (or since midnight last night.

As the winds have finally become manageable, H1 has 2-locklosses in a row at beginning (within 2secs) of starting the LOWNOISE_ESD_ETMX state.  (this is after h1 being down for most of the day due to winds [which are now lower], AND with high microseism which still is high)  Alignment looks good after the afternoon alignment run by Tony toward the end of his shift.  DRMI has looked a bit ugly, but PRMI has repeatedly locked (albeit ugly-looking at first) and had ISC_LOCK with smooth sailing....until....LOWNOISE_ESD_ETMX.  Will pause at TRANSITION_FROM_ETMX for this next lock attempt.

TITLE: 12/27 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Wind
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 20mph Gusts, 15mph 3min avg
    Primary useism: 0.07 μm/s
    Secondary useism: 0.78 μm/s
QUICK SUMMARY:

Drove in to some very strong winds (see attached), but they have since calmed down such that (after an alignnment), Tony has gotten H1's ISC_LOCK beyond PRMI!  (currently at ENGAGE ASC).  In addition to the very windy last-9-hrs, the secondary µseism took a turn for the worse in the last 24hrs (but if you squint you might see it drift down over the last 6hrs).  With all this said (lower winds and µseism), here's hoping H1 and the earth can conspire for a return to Observing!

[H1 currently powering up (woo woo!), AND violins are looking fine---not rung up for this first look for H1 since the last lockloss 10hrs ago.]

TITLE: 12/27 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Wind
INCOMING OPERATOR: Corey
SHIFT SUMMARY:

Wind storm started early in the morning with a high microseism.
I held the IFO in Down until the wind stayed below 35 for 10 minutes. Then tried to lock.

I did take Shiela's advise and change the H1:ASC-POP_A {PIT & YAW} OFFSETs and I saved them in the both the Safe and Observe.snaps to try to get better stability through some locking states.

Seems like we are on the back end of this wind storm, Wind fence is still intact.
GRB-Short E538969 @ 22:34 UTC

 

Thu Dec 26 10:12:57 2024 INFO: Fill completed in 12min 53secs

TCmins [-141C, -72C*] OAT [+11C, 50F]

* TB-B shown because it did go below trip-temp of -65C.

TITLE: 12/26 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Aligning
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 36mph Gusts, 26mph 3min avg
    Primary useism: 0.14 μm/s
    Secondary useism: 1.05 μm/s
QUICK SUMMARY:
H1: Unlocked
When I arrived ISC_LOCK was in Initial_alignment.
Winds are gusting above 40MPH.
Once Initial_alignmnet is finished, I'll hold in down for an hour as this wind storm passes. I'll wait for the tumbleweeds to get to where they are going.

Useism has climbed to above 1um/s and winds are above 20mph. We have been unable to keep green arms locked for long enough to complete an initial alignment for the past 1.5 hours...but then it just suddenly worked. We've made it to engaging ASC for full IFO, I'll setup H1_MANAGER again.

TITLE: 12/26 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 161Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY:

Eve Shift:  Christmas Evening Edition
After the busy day shift, the evening was pretty calm.  H1's been locked most of the shift (currently at 6.25hrs)

Sadly, the secondary microseism is now above the 95th percentile at about 0.8um/s.
LOG:

• 0051 Gerardo arrives on site for VAC investigations (Tony letting him through gate)
  • ~0217 this is when Tony signed off and mentioned Gerardo leaving site as well.
• 0145-0148 Sqz bumped H1 from Observing to relock.

TITLE: 12/25 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY:

Christmas Day Lockloss #1:
Happened on the OWL Shift last night. Very pecilar, Small-ish Vacuum pressure increase at EY happened seconds after the lockloss. Pressure increase seen at Mid Y as well.
Persons involved: Dave, Daniel, Gerardo, Sheila.
Seems like the Ion pump failure caused the lockloss and the time delay between the lockloss and the increase in pressure was just the speed of the gasses moving through the chambers.
Gerardo came out to site to check the Vacuum pressure & Ion pump at EY. I've also eneded up coming out here as well as a safety buddy.

Christmas Day Lockloss # 2:

Christmas Day Lockloss # 3
 

Relocking was fully automated except my request for an inital_alignment after the Earthquake.

Sheila, has asked me to try to update the H1:ASC-POP_A_YAW_OFFSET & PIT, for better stability in the while locking in CARM_TO_REFL & Resonance regions. I will try to do this tomorrow during the next lock. Since we are already locked now.

LOG:
Gerardo heading to EY and is still out there as I write this alog, I'll be onsite until Gerardo leaves.
 

Christmas Day Lockloss # 3
This lockloss was caused by a 4.9 Mag EarthQuake out of Port McNeill Canada.
H1 was unlocked very quickly, then a few EX saturations and Earthquake mode was activated.

Christmas Day Lockloss # 2:
We had am Unknown lockloss from Nominal_Low_Noise but not Observing while we waited for the ADS camers to come on.
Erik had restarted the BS H1cam26 on sw-lvea-aux on port 35 and were watching the ADS cameras try and start working and waiting. ( They did not fully start working) Erik's restart did not cause the lockloss, and Thankfully this lockloss did not come with a strange spike in pressure either.

Related alog: https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi?event=1419179528

TITLE: 12/25 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 15mph Gusts, 11mph 3min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.43 μm/s
QUICK SUMMARY:

H1 Is currently Locked and in Nominal Low Noise but one of the ADS cameras is not working currectly.
Erik has joined the Teamspeak to help me restart it.


Info Ive collected  so far.
ADS Camera Servo for the Beam splitter is glitched out again and needs to be restarted.
H1cam26 hosted on h1digivideo2.
nslookup returns 10.106.0.46

Pinging h1cam26 returns data though.
Pinging 10.106.0.46 also returns data.