Mon Dec 04 10:03:51 2023 INFO: Fill completed in 3min 49secs
Gerardo confirmed a good fill curbside.
Now that it has warmed up outside I have lowered the trip temps to -80C.
FAMIS 20005
There was a slight increase in temperature inside the enclosure over the past week (only of about 0.5 degF) which seems to correspond to an increase in NPRO output power (5 mW), PMC transmitted, and PMC reflected (3 mW each). The PMC trans signal also appears to get noisier starting about 6 days ago; not sure if that's anything.
TITLE: 12/04 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 131Mpc
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 5mph Gusts, 4mph 5min avg
Primary useism: 0.06 μm/s
Secondary useism: 0.60 μm/s
QUICK SUMMARY:
H1 is Locked in NLN and Observing for 4 hours.
TSC ITMY Coolant flow rate is still steadyright where I left it yesterday, so thats good.
CDS Overview has some indicators that ar red for some reason, I will start to investigate:
Looks they have been red for about 12 hours.
H1:DAQ-DC1_H1IOPSUSH2B_CRC_SUM
H1:DAQ-DC1_H1SUSIM_CRC_SUM
H1:DAQ-DC1_H1SUSHTTS_CRC_SUM
PSL Dust Monitor is still not changing in value. I imagine that someone will have to go into the PSL and poke it.
*H1:PEM-CS_DUST_PSL102 WARNING: dust counts did not change, please investigate
Full DQ report here
Summary
TITLE: 12/04 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Oli
SHIFT SUMMARY:
Decent night with H1 up for most of the shift (w/ range down to ~144Mpc). In the last few hours microseism is inching up slowly, and there were many Phillipines aftershocks. The fog is rolling in.
LOG:
h1's been at NLN for almost 3.25hrs. Violin modes are rung up on this lock (It took over an hour to complete OMC WHITENING). Have not had any SQZ ASC issues during this lock. H1 range hovers around 140Mpc.
All looks well for the last week for all site HVAC fans (see attached).
This evening as soon as Corey started his shift he found several of the violin modes were rung up (see screenshot of the darm), which is when he texted me to look into it. Most of the modes were coming down slowly with nominal settings, however I had to tweak the settings of the following two modes, the details are given below,
ITMY05/06: FM1+FM8+FM10 Gain -0.01 (will ramp it up later)
ITMY08: FM1+FM5+FM10 Gain= +0.1 (earlier I tried a phase of +60degree and that seems to be making this mode worse).
The above settings are not in the lscparams yet, will test them out for few locks.
The latest settings for IY05/06 worked fine (both narrow and broad filters shows the mode is decreasing) all night as shown in the screenshot attached below. The decline was slow, hence the gain needs to be increased.
However, when I applied the same setting this morning (after IFO rel-ocked) then I was not very sure about the above settings still working (Iy05 increasing and IY06 decreasing- see ndscope plot here). Hence, I need to keep on eye on this and if required further tweak the phase/gain.
TITLE: 12/04 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Corey
SHIFT SUMMARY:
locking notes:
locloss @ Find_IR
locloss @ Find_IR
Went back to Initial Alignment while we were ringing down.
*H1:PEM-CS_DUST_PSL102 WARNING: dust counts did not change, please investigate this screenshot.
I think that the Dust monitor in the PSL may need to be looked at as it's reporting hasnt changed in 12 days even though we have had 40 MPH winds in the last 2 days.
Update on TCS Chiller water level:
Went out to the mechanical room and saw that the water level is where I had left it this morning.
I grabs a picture of the water
Although we are out of water in the refill jug. Im not sure if thats distilled water or just tap water, or special CO2 cooling water, with nano lemon flavored spark, so i'll just leave it until tomorrow.
LOG:
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 19:05 | TCS | Tony | Mechanical Rm | N | Y CO2 Chiller check, went @ 9am returned by 9:30 | 19:25 |
| 19:06 | TCS | Tony | LVEA | N | Checking for TCS Coolant leaks, Start 930 back at 10 | 19:36 |
| 19:07 | PEM | Robert | LVEA | N | Setting up shaker. | 20:07 |
| 20:10 | PEM | Robert | EX & EY | N | Looking for Stands | 20:58 |
| 23:56 | PEM | Robert | LEVA | N | Turning off lights | 00:16 |
TITLE: 12/04 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 3mph Gusts, 2mph 5min avg
Primary useism: 0.06 μm/s
Secondary useism: 0.40 μm/s
QUICK SUMMARY:
Similar to yesterday, Tony handing off an H1 close to Nominal Low Noise (NLN) after his day of TCS chillers & more big earthquakes.
He was able to fully complete several Initial Alignments during his shift (I hadn't been able to do an Input Align on Fri.).
Currently at NLN and can not go to OBSERVING due to violin modes which are higher (on nuc30 they were behind the Legend, so Tony made it smaller---peak is above 10^-15). These rung up violins is a new feature from last few days.
Also currently have the recent-ailment of H1 having the low frequency bump on DARM (10-50Hz), but it does look a little lower. (but our range is still around 140Mpc)
Other NOTES:
Famis 26220
Laser Status:
NPRO output power is 1.82W (nominal ~2W)
AMP1 output power is 68.31W (nominal ~70W)
AMP2 output power is 138.4W (nominal 135-140W)
NPRO watchdog is GREEN
AMP1 watchdog is GREEN
AMP2 watchdog is GREEN
PMC:
It has been locked 4 days, 1 hr 56 minutes
Reflected power = 16.33W
Transmitted power = 110.0W
PowerSum = 126.3W
FSS:
It has been locked for 0 days 0 hr and 18 min
TPD[V] = 0.891V
ISS:
The diffracted power is around 2.1%
Last saturation event was 0 days 0 hours and 18 minutes ago
Possible Issues: None
Tony reported that the CO2Y laser had tripped off from a flow error and when he trended the flow it had been falling for ~12 hours (see his alog74560 and alog 74562). He had to add 4.5L of water to the chiller and after starting it back up and running for a bit, the flow seems to be stable. Tony walked the lines and did not see any signs of a leak. This concerning large amount of water definitely makes me think that there is a leak somewhere, so I've asked hom to check on the chiller level throughout the day and if it continues to drop then we may have to go on table or open up the chiller itself.
Trending a handful of different CO2 channels, I don't anything unusual before the flow started to go down. Once it started going down we see more change in the power, temp, and pzt as expected with a real reduced flow. We will need to keep a very close eye on this.
I took some pictures of my adventures!
The warning on the chiller it's self was the first thing I noticed.
The water level was WAT TOO LOW.
All of these areas were dry when I looked earlier today.
From the Chillers to the pipes on the mezzanine, dry.
across the Mez, and into the wall, dry
Out of the wall into the LVEA was dry.
The Cat walk over to the output arm was dry.
I took pictures of any of the places where elbows and inline devices/ adapters or other points of failure id expect.
All of those places were dry, and I didn't see any puddles.
I saw no sign of an active leak.
But, I still have not check the TCS ITMY CO2 LASER table for a leak on the inside.
Current Water Level now.
The flow rate seems to be steady now.
TITLE: 12/03 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake
CURRENT ENVIRONMENT:
SEI_ENV state: EARTHQUAKE
Wind: 8mph Gusts, 6mph 5min avg
Primary useism: 0.68 μm/s
Secondary useism: 0.66 μm/s
QUICK SUMMARY:
16:20 UTC Started Initial Alignment, Got past ACQUIRE_XAM_IR.
Canceled the Intital alignment because I noticed that there was something wrong with the TCS ITMY chiller
After trending some TCS ndscopes I saw that the flow rate had dropped on ITMY CO2 laser chiller, and the TCS laser had shut it's self off.
I ended up going to the Mechanical room and Putting in 4.5 Liters of water into the ITMY Chiller.
This raised up the flow rate back to what is expected. Screenshot1 & Screenshot2.
Went out on the LVEA floor to look for coolant leaks and saw no drips or puddles.
But I did not open the ITMY TCS laser table. I did listen for drips or leaks and I couldn't hear any.
I then double checked the flow rate was steady, and then turned the TCS CO2 LASER back on.
Started Initial Alignment again and Verbals Immediately started telling me IFO_OUT for OM1 and OM2 Saturations.
I then took ISC_LOCK to IDLE and was surprised to see that the Saturations hadn't stopped.
Spoke to TJ about it and he said I should restore the sliders to a known good value.
Canceled that Initial Alignment.
I then chose GPStime 1388596244 ( End of an Inital_Alignment that was before a recent lock), restored all Optics except for SQZ and that did make OM1&2 stop saturating.
Started Initial_Alignment again, to find that X arm didn't have enough light on the ALSX.
Canceled that Initial_Alignment to take ISC_LOCK to Manual Green Arms to get good flashes on X arm.
Got some good flashes back on ALSX and sent ISC_LOCK back to Initial_Alignment.
Initial_Alignment ran and completed just fine with a fair amount of saturations.
Locking notes:
Comm beat note is around 0dBm
and the Diffbeatnote is about -8dBm
Lockloss @ FIND_IR
Lockloss @ FIND_IR
Doing another Initial Alignment. I.A. Completed Again, this time with less saturations.
Locking notes:
Got past Find_IR !!! yay!
Incoming 7.0 Mag earthquake from Philipines....
Lockloss @ SHUTTER_ALS Due to Earthquake....
ISC_LOCK is currently IDLE
Sun Dec 03 10:04:55 2023 INFO: Fill completed in 4min 52secs
TITLE: 12/03 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
SEI_ENV state: EARTHQUAKE
Wind: 4mph Gusts, 3mph 5min avg
Primary useism: 0.10 μm/s
Secondary useism: 0.41 μm/s
QUICK SUMMARY:
When I walked in H1 was Unlocked and in Earthquake mode due to an earthquake near the Philpines.
H1:TCS-ITMY_CO2_INTRLK_FLOW_ALRM is alarming tagging TCS.
TCS_LASER :Y Flow alarm flag is raised and it seems like the flow rate has dropped according to H1:TCS-ITMY_CHILLER_FLOW1.
Recieved a call at 10:38UTC regarding the detector having issues relocking. It was stuck in an Initial Alignment at Green Arms, and right when it started the initial alignment, almost every suspension saturated, including the RMs and SQZ suspensions. We were also getting constant IFO OUT and MC2 saturations while in green arms. At 10:39UTC the ISI BS Stage 2 WD had tripped after many BS saturations.
We also have multiple large earthquakes coming in from the Phillipines and so the first thing I did was take the detector to DOWN. The R-waves of the earthquakes are now hitting us, and ITMY Stages 1 and 2 WDs have also tripped due to a very sudden and large spike in ground motion. At 11:36 everything is calmer in the chambers so I reset the watchdogs on both ITMY stages and BS stage 2.
13:23UTC Ground motion has calmed down enough that we can start relocking. Starting an Initial Alignment.
13:23UTC Started an Initial Alignment
I was getting the same issue as I had last time - once we get to ACQUIRE_XARM_IR, we aren't able to lock the XARM in IR, and just get a lot of MC2 and IFO_OUT saturations, and the IMC keeps unlocking.
I was going to try running the dark offsets script since that is what worked(ish) for Jenne and Ibrahim (74472), but partway into setting that up (I haven't run one yet and was going to go off of 69650), I decided (also with previous permission from TJ) that the best thing to do would just be to put the detector in DOWN until commissioners can help with relocking during the DAY shift, especially because if the issue is similar to the one I had a few nights ago, I do not know exactly what they did or what to look at. Hopefully relocking in a couple of hours goes well!
