Sat Nov 02 10:10:55 2024 INFO: Fill completed in 10min 51secs
Janos, Dave:
VACSTAT version 1.10
Following several recent false positive VACSTAT alarms due to noise in a single gauge (usually BSC3 PT132) we have made a change to the code to mitigate this.
The new alarm logic is:
The alarm system was modified to alert the vacuum team if the channel H1:CDS-VAC_STAT_GLITCH_NUM_GAUGES is 2 or greater.
My cell phone alarm will continue to sound on a single gauge trip, because tripped gauges are latched out from the sytem and we need to reset any false positives to restore the nominal functionality.
The new code was installed Friday afternoon (15:40 01nov2024 PDT).
Design reasoning:
In the event of an in-chamber gas release the local gauge will see a large signal. Gas migration to neighboring chambers will result in smaller signals there. Increasing their trip sensitivities will increase the chance of seeing a multi-chamber event.
A new enumerated PV has been added, H1:CDS-VAC_STAT_STATE.
It can have the following values
VACSTAT_STATES = {
"MONITORING": 0,
"SINGLE": 1,
"MULTIPLE": 2,
}
MONITORING = nominal state.
SINGLE = one chamber has tripped, all the others have been set to their "sensitive" trip settings.
MULTIPLE = number of chambers tripped >= 2, alarms have been sent.
Lockloss @ 11/02 17:13UTC after only 9 minutes locked
18:08 UTC Observing
Lockloss @ 11/02 15:17 UTC after 4 hours locked
17:05 UTC Observing
TITLE: 11/02 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 158Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 21mph Gusts, 16mph 3min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.26 μm/s
QUICK SUMMARY:
In Observing and have been Locked for 3hours 15mins. Secondary microseism is coming down and wind is just over 20mph
TITLE: 11/02 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: TJ
SHIFT SUMMARY: The IMC has been very unhappy and the FSS has been glitching frequently this shift. HLID picket fence seismometer has also been glitching out this afternoon (Tagging SEI), after the locklosses the IMC would lose it again and relock after reaching READY this kept happening this evening. We just locked PRMI as of 05:00 UTC.
The left display of the Operator WS is flickering, unplugging and plugged it back in fixes the issue but only for like an hour.
LOG: No log
Lock#1:
04:17 UTC, the IMC lost lock less than 100ms after ASC_AS_A, no IMC tag though and the FSS didn't really look glitchy.
Closes FAMIS26509, last checked in alog79454. It looks as expected, the lack of switching in the recent past was the vent I believe.
IMC and ASC_AS lost it at the same time, looks like some kind of fast ringup in the FSS_FAST channel. FSS_OSCILLATION and IMC tag on the lockloss tool.
TITLE: 11/01 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock acquisition
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Ran a handful of tests today to try to narrow down further the FSS glitches that we've been seeing (80993, 80990 ). We recently started relocking with only minor changes, hopefully they will help.
LOG:
LVEA zone 1B has been slowly getting warmer over the past 2 days. Not a worry yet, but something we should keep an eye on.
Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
---|---|---|---|---|---|---|
15:34 | ISC | Sheila | LVEA | n | Unplug cable for IMC test | 15:49 |
17:50 | ISC/PSL | Sheila, Vicky | LVEA | n | FSS tfs | 18:10 |
21:39 | ISC/PSL | Sheila, Marc | LVEA | n | FSS meas. | 21:40 |
21:40 | SAF | Ryan C | LVEA | N -> Y | Laser HAZARD transition | 21:49 |
21:40 | ISC | Keita | LVEA | YES | HAM6 viewport camera housing PD | 22:41 |
21:41 | SAF | Laser | LVEA | YES | LVEA is laser HAZARD | 08:21 |
TITLE: 11/01 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 19mph Gusts, 10mph 3min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.39 μm/s
QUICK SUMMARY:
00:25 UTC Observing
DQ Shift summary:
More daily details can be found at: https://wiki.ligo.org/DetChar/DataQuality/DQShiftLHO20241014
We measured the TTFSS OLG, I didn't save the data but will attach several photos here. There is a feature that looks like it could be unstable at 1.68MHz, there are about 3 untiy gain crossings around 500 kHz. The FSS has a gain between the test points that mean that unity gain is at -10dB in this measurement.
67253 Craig mentions that the FSS OLG was 350kHz, we'd need to reduce the gain by 3dB to get back to a 350 kHz ugf.
Scans from the FSS after reducing gain.
In Marc's files above the A one should be magnitude for 100kHz to a few MHz frequency span, B should be phase for that measurement. C should be magntiude zoomed in on the 1.7MHz region, D should be phase for that measurement.
We lost lock at some higher states in the relocking process due to the laser glitching* issue that we have been having. At this point we decided to pause locking for now to run some tests that were discussed during a meeting yesterday. The current plan is to hold at 2W with the ISS, FSS, and IMC locked. If this configuration is found to be unstable or we see excessive glitching still, then we will start to remove just the ISS or FSS, etc to narrow down the problem.
During this time, other, approved, light maintenance activities can take place.
*The issue may not be the laser itself, but I'm unsure what else to call this at the moment.
We sat in this configuration from 15:34:37 UTC Nov 1 until the IMC lost lock at 16:45:34, 1 hour and 10 minutes. The attached screenshot shows some channels at the time of the IMC lockloss, FSS fast mon and PC mon show some growing oscialltion until the time that the reference cavity transmission drops, the ISS sees a jump, and the IMC unlocks.
We've been sitting with the IMC in offline (MC2 is misaligned so the IMC won't flash), with the PMC, FSS and ISS locked since 17:09 UTC.
Summary of today's tests:
Yesterday we went back to the higher range CARM configuration, in which the fast/IMC locklosses first appeared 80938. Overnight we had two observing locklosses tagged with the IMC, neither of which had glitches visible in the FSS fast mon channel, 1414411965 (1st attached screenshot), and 1414381791.
This morning I changed the gains by 6 dB in the other direction, so we further limit the range available to the frequency actuator. (CARM sliders and ndscope) Our microseism is around 5um/second right now, slightly above the 90th percentile, but there was no wind and no earthquakes this morning. At times the IMC splitmon was reaching -8V, it's range is +/- 10V so this is about as far as we can go in this direction, and might be kind of marginal.
We've since had one lockloss 1414426971, this is tagged as commissioning because we were in commissioning time at the moment, but no one was actively doing anything at the time so this wasn't caused by commissioners. There was a glitch in the FSS fast mon for this lockloss, although it was below the threshold for the tag to be applied, and around the time of that glitch the drive to the ISS AOM becomes noisy. We were not saturating the IMC splitmon at the time of the lockloss so that doesn't seem to be the cause.
I've removed this, so now we will be back to the gains we've been with for the last several weeks, and most of O4.