TITLE: 10/16 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 156Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 10mph Gusts, 8mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.39 μm/s
QUICK SUMMARY:
ALS_DIFF had stepped over a resonant offset by about 35. I'm not yet sure if this offset is changing by this much each lock, or if there is something else going on.
TITLE: 10/16 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 161Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY: We've now been Locked for 2hrs 15mins and everything is looking good besides some wind at around 20mph.
Relocking from the 10/15 03:18 earthquake lockloss, I needed to touch both green arms, and IR DIFF could not be found so I needed to adjust the DARM offset, but lost lock while doing that, and during the next relock I had to help with DIFF again.
LOG:
23:00UTC Detector Observing and has been Locked for 11hours 35mins
00:00:02 Verbal said WAP on in LVEA, EX, EY (dropped to a value of 0 for 5s starting at 00:00:00)
- Rarely shows up (unless we actually forgot the wifi on) but when it does it's always at 00:00UTC
03:18 Lockloss due to nearby Canadian earthquake (73488)
- Relocking: needed to go to GREEN_ARMS_MANUAL for both arms, and needed to pause at CHECK_IR and touch DIFF Offset
03:55 Lockloss from CHECK_IR
04:46 Into NOMINAL_LOW_NOISE
04:57 Observing
Lockloss @ 10/16 03:18UTC due to a sudden local earthquakeearthquake from west Canadian coast. Peakmon jumped up to 2200 counts very quickly.
03:18:40 DIAG_MAIN message: SERVO_BOARDS: IMC_REFL_SERVO_SPLITMON > 8V
03:18:45 SRM saturation
03:18:49 Earthquake mode activated
03:18:54 SRM saturation
03:18:56 SRM saturation
and Lockloss
These past couple of days we have had a lot of sudden ground motion for some reason.
EDIT: USGS actually did measure this one as coming from off the west coast of Canada, so makes sense why we did not get notified in time. The local earthquakes from previous days had not shown up on USGS however so were definitely from much closer than this one was.
04:46 Reached NOMINAL_LOW_NOISE
04:57 Observing
We're Observing at 155Mpc and have been locked for nearly 16hours now.
At 00:00:02UTC we did get the 'WAP on in LVEA, EX, EY' notification on Verbals, which happens every once in a while at 00:00UTC when the values go from 2 down to 0 and then back to 2 5s later. Nothing else to note.
TITLE: 10/15 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 159Mpc
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 18mph Gusts, 13mph 5min avg
Primary useism: 0.06 μm/s
Secondary useism: 0.30 μm/s
QUICK SUMMARY:
Detector Observing and has been Locked for almost 12hours now. Everything is looking good. Wind has picked up a bit in the last few hours.
TITLE: 10/15 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 165Mpc
INCOMING OPERATOR: Oli
SHIFT SUMMARY: Quiet day, EY chiller issue thats seems to have been resolved. The wind has been picking up over the past 3 hours. We've been locked for 11:35 as of 23:00 UTC.
LOG:
No log
We've been locked for 7:55, everythings stable. The EY water temperature has recovered it seems.
I was just going to put in an alarm bypass for the EY chiller supply water temperature and noticed that it has returned to its operational temperature in the past 30 mins. 6 hour trend shown.
Sun Oct 15 10:08:06 2023 INFO: Fill completed in 8min 2secs
TITLE: 10/15 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 158Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 4mph Gusts, 2mph 5min avg
Primary useism: 0.08 μm/s
Secondary useism: 0.32 μm/s
QUICK SUMMARY:
Richard called to let me know there was an EY chiller alarm, the VEA temperatures look stable but the H1:FMC-EY_CY_H20_SUP_DEGF channels temperature is rising (Chiller water temperature).
That temperature channel seems to be stabilizing, although at a higher than nominal temperature.
The temperature has come back down, I'll keep watching it though.
After a 1hr59min lock, H1 had a lockloss and H1 was stuck at FIND_IR with ALS DIFF needing "Search for IR Resonance by hand" at 947utc/247amPT.
Unfortunately, my computer has been monumentally slow so. I have rebooted my computer and it is a little bit easier, but still very hard to work on H1 because of the slow NoMachine interface. I was able to FIND IR (very slowly due to my computer) by moving the DIFF OFFSET only a few clicks.
ISC LOCK continued.
Although I had requested NLN for ISC_LOCK, on my (slow) computer it was "stuck" at the ACQUIRE_DRMI_1F state (it was hard for me to tell why it was stuck here....it should continue on to DRMI_LOCKED_PREP_ASC, etc. etc., but ISC_LOCK was stuck at ACQUIRE_DRMI_1F.
QUES/NOTE: --Could this be due to GRD_IFO being in the Automatic Operation state?
I then requested ISC_LOCK to go to the next step: DRMI_LOCKED_PREP_ASC
It immediately went there with no problem. I then selcted one-step-at-a-time on ISC_LOCK & it proceed fine to each step.
On ISC_LOCK, I then requested a bigger step of going to PREP_ASC_FOR_FULL_IFO, and ISC_LOCK continued on its own to PREP_ASC (as normal!). (around this time, since I had time to NOT babysit, ISC_LOCK, I took GRD_IFO to MANUAL operation...in case that could be an issue.)
While waiting at the ENGAGE ASC state....I requested ISC_LOCK to go to CHECK_VIOLINS_BEFORE_POWERUP.
ADDENDUM: It his hard for me to determine if there was an actual problem here, because my computer has been very slow....so all troubleshooting has been painful to perform. But since ISC_LOCK appears to be returning to normalcy (as seen from my SLOW computer), I am going to take GRD_IFO to AUTOMATIC operation and let H1 continue as it has done (from what I'm told, has been 3+ weeks with no issues...up until this morning's issue....which is either (1) a legit problem with H1, (2) a problem with my extremely slow computer, or (3) a combination of 1 & 2.
It is 1058utc (358amPT), and ISC_LOCK is at Check VIOLINS (just made it to this state), so I then requested NLN on ISC_LOCK. I am hoping ISC_LOCK can continue on its way, as normal.
At 1101utc/401amPT, I am taking GRD_IFO to AUTOMATIC operation....the hope is H1 will be back up to OBSERVING within the next 30-45min.
[Crossing fingers]
After taking GRD_IFO to AUTOMATIC operation (via OPS_Overview medm), I returned to the H1_MANAGER guardian node, because it had been having a notification from the beginning about requesting an INIT to clear it's notification (I remember trying an INIT several times, but this notification would NOT clear.).
However this time (about an hour later), I was able to clear this notification when requesting an INIT.
As of 1107utc/407amPT, ISC_LOCK is at MOVE_SPOTS.
Additional Notes:
NoMachine interface & my computer appear to be much better, although still not great/normal. I do have a new wireless mouse that I'm using and it's operation is dodgy, so I only use it as a last resort.
Another note with my laptop is that in the evening, I log into NoMachine and leave it logged in overnight (so if action is needed, I can jump into my session and start working on H1.) Perhaps this is a bad procedure? I noticed that although my laptop is plugged into power, it's battery was very low when I first logged in to work on H1 this morning. (not sure why this is the case since it is plugged into power) After working on H1 the last 90+min, my laptop battery has increased from a low percentage to 52%. Maybe I should not keep my laptop logged into NoMachine overnight? (I only did this so I could save time and could hit the ground running....but maybe my computer (and I!) can not handle this.)
1114utc/411amPT & ISC_LOCK is at TRANSITION FROM ETMX.
Will NOW let H1 continue on its way.
1126utc: Will continue to baby sit H1 until it is back to OBSERVING. Currently waiting for CAMERA SERVO.
1134utc/434amPT: H1 Back To Observing AUTOMATICALLY
In the hopes I do not have an issue with my computer/NoMachine, I am going to completely log out of NoMachine, close my laptop for the night, and keep it plugged into power tonight/this morning.....just....in ....case.
Back to Observing at 1443 after an earthquake took us out. H1_MANAGER waited ~1.5 hours while the ground motion calmed down and then tried relocking.