FAMIS 26301

Laser Status:
    NPRO output power is 1.828W (nominal ~2W)
    AMP1 output power is 64.52W (nominal ~70W)
    AMP2 output power is 136.7W (nominal 135-140W)
    NPRO watchdog is GREEN
    AMP1 watchdog is GREEN
    AMP2 watchdog is GREEN
    PDWD watchdog is GREEN

PMC:
    It has been locked 0 days, 11 hr 34 minutes
    Reflected power = 22.33W
    Transmitted power = 104.0W
    PowerSum = 126.3W

FSS:
    It has been locked for 0 days 1 hr and 1 min
    TPD[V] = 0.7807V

ISS:
    The diffracted power is around 2.4%
    Last saturation event was 0 days 1 hours and 2 minutes ago

Possible Issues: (both are known)
    AMP1 power is low
    PMC reflected power is high

Sat Sep 28 08:13:16 2024 INFO: Fill completed in 13min 12secs

TITLE: 09/28 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 7mph Gusts, 6mph 3min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.27 μm/s
QUICK SUMMARY: H1 just lost lock at 14:16 UTC, 8:34 into the lock stretch. I don't notice anything obvious as to the cause except maybe some ETMX motion immediately prior, but this doesn't look like one of the ETMX "glitches" like we've seen before. Have been trying to lock green arms for 30 minutes, so I'll start touching them up by hand.

• Wind is low, microseism is on its way down
• All other systems nominal

TITLE: 09/28 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY:

Currently relocking and at ACQUIRE_DRMI_1F. We were cleared to go into Observing by the CAL team 45 minutes ago when we were still Locked and got in 10 minutes of Observation time before losing lock from an FSS oscillation.
Before that, troubleshooting was being done by the calibration team for the entirety of my shift, so I had just been trying to keep us locked and by running broadband measurements when they needed them. Current status of the calibration work is that they tried multiple things with ultimately no success. The things they tried have been reverted, and they will be writing up their own alog and continuing work on Monday.

While trying to relock after the first lockloss of my shift and this last lockloss, we got stuck in READY due to IMC_LOCK being in FAULT with the DIAG_MAIN messages: "PSL_FSS: PZT MON is high, may be oscillating" and "PSL_FSS_TPD: RefCav transmission low, fix alignment". I tried taking the IMC to DOWN and then back to LOCKED and tried toggling the FSS loop automation OFF and then back ON, but neither seemed to help. But both times the IMC was eventually able to lock, but I'm not sure that it had anything to do with my toggling. Once the IMC was locked, relocking went smoothly for that first relock and so far is going well this relock.
So the FSS oscillations are not only causing locklosses, but now are also starting to delay relocking. Tagging ISC

LOG:

22:16UTC Lockloss
    22:19 Lockloss from LOCKING_ARMS_GREEN
    22:20 Lockloss from LOCKING_ARMS_GREEN
    22:21 Lockloss from LOCKING_ARMS_GREEN
    IMC in FAULT with the messages "PSL_FSS: PZT MON is high, may be oscillating" and "PSL_FSS_TPD: RefCav transmission low, fix alignment"
        - Taking IMC_LOCK to DOWN+INIT and then back to LOCKED didn't work
        - Toggled FSS loop automation autolock OFF/ON, eventually IMC locked a couple minutes after doing this so not sure if it helped
23:29 NOMINAL_LOW_NOISE

23:29 NLN_CAL_MEAS
23:30 Started broadband calibration measurement
23:36 measurement done (PCALY2DARM_BB_20240927T233009Z.xml), back to NOMINAL_LOW_NOISE
23:41 NLN_CAL_MEAS
23:42 Started broadband calibratoin measurement #2
23:48 Measurement done, (PCALY2DARM_BB_20240927T234226Z.xml) back to NOMINAL_LOW_NOISE

00:11 Lockloss
    00:34 Started an initial alignment
    00:52 Initial alignment done, relocking
01:37 NOMINAL_LOW_NOISE

01:43 NLN_CAL_MEAS
    01:44 Started broadband calibration measurement #3
    01:50 measurements done (PCALY2DARM_BB_20240928T014428Z.xml)
01:50 NOMINAL_LOW_NOISE
03:00 NLN_CAL_MEAS
    03:01 Started broadband calibration measurement
    03:08 Measurement done, back to NOMINAL_LOW_NOISE
03:08 Lockloss due to earthquake https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi/event/1411528136
04:05 NOMINAL_LOW_NOISE    
04:16 Observing

04:22 Lockloss due to FSS oscillation starting 30 before LL
    FSS oscillating and won't lock - same DIAG_MAIN messages as earlier
    04:39 Got it locked, starting an initial alignment
    04:56 Initial alignment done, relocking
                                                                                                             

Currently relocking and at MOVE_SPOTS - we lost lock about 30 minutes ago from a nearby earthquake.

The CAL team has been spending the last several hours troubleshooting the issues with the H1 calibration. We've had two other locklosses during my shift besides this latest earthquake one, both were easy to relock from, but have slightly slowed down the calibration troubleshooting.

Note: There is an issue with the flag that decides whether or not we have clean data for H1:CDS-SENSEMON_CAL_SNSC_EFFECTIVE_RANGE_MPC. Since 09/27 20:25UTC we have not been Observing (so the past 5 hours), but diaggui and ndscope are both showing data for the CLEAN channel.

TITLE: 09/27 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 13mph Gusts, 8mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.52 μm/s
QUICK SUMMARY:

Currently relocking at LOWNOISE_LENGTH_CONTROL. Once we get to NLN we will be running a broadband calibration sweep as part of our continuation of updating/fixing the calibration.

TITLE: 09/27 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Calibration
OUTGOING OPERATOR:  n/a
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 10mph Gusts, 4mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.54 μm/s
QUICK SUMMARY:

• Tagged in to help cover the Day shift from TJ
• 2025 Out of OBSERVING for SRCL change by TJ and then went right into a simulines calibration measurement for Louis.
• 2115 had to run a 2nd simulines calibration (Louis)
• 2216 Lockloss while calibrators were working on calibration.
• 2244-2246 Returning CDS laptop to charger near CER (keita)
• H1 currently relocking as Oli has been taking over for the Evening shift.

Ran simulines calibration measurement for Louis and then stayed out of Observing a little extra while they worked on calibration results.

Measurement NOTES:

• Following instructions (link)
• H1 at NLN for for only ~1hr
• Calibration_Monitor medm screenshot is attached
• ..... Out of Observing, ISC LOCK to NLN_CAL_MEAS & Observatory Mode to CALIBRATION
• 1326-1352 Simullines measurement
  • gps START time:  1411504005.650423
  • gps STOP time: ............
  • Files writtten to subfolders here:  /ligo/groups/cal/H1/measurements
• "pydarm report" was then run by me, but then canceled.....then Louis and Joe B were heard working on Teamspeak.

There was some commissioning time this morning to get the SRCL offset that we needed before running a calibration measurement that we desperately needed. We then lost lock when running simulines. Relocked automatically and we are now running another simulines.

• 1515 Commissoining
• 1628 Observing
• 1759 Calibration
• 1810 Lock loss
• 1630 Observing
• 1325 Calibration

Once Sheila pushed a new SRC detuning, I ran a broadband measurement, then started simulines. We lost lock a little over a minute after starting, not sure of the cause yet.

Simulines start:

PDT: 2024-09-27 11:08:44.473513 PDT
UTC: 2024-09-27 18:08:44.473513 UTC
GPS: 1411495742.473513
 

Lock loss 1810UTC

Lockloss 1411472854

The lock loss tool tagged FSS oscillation. Trending H1:PSL-FSS_FAST_MON_OUT_DQ shows that something started to get noisy 35seconds before lock loss. I didn't see any other strangeness in our usual LSC, ASC, ETMX, DARM signals.

Took another data set of FIS with different SRCL offsets, to try to set the SRCL detuning for calibration measurement, similar to 79903. 

• no squeezing: 1411485337 (120 seconds) (ref 4)
• FIS, ran scan alignment, ressults here , ran scan sqz ang with the blrms changed to 6 (we normally use 3, to minimize the 300 Hz noise, but in this case we want to min 1.7kHz sqz)  SRCL at -290 start at 1411486541 ref 5  (SRCL offset we have been observing with)
• FIS, SRCL at -190 start at 1411487041 ref 6
• FIS, SRCL at -90 start at 1411487385 ref 7
• FIS, SRCL at -340 start at 1411487954 ref 8 (manual sqz ang adjust)
• FIS, SRCL at -390 start at 1411488251 ref 9
• FIS SRCL at -490 start at  1411488585  ref10  (22.2k counts on POP A 45I ADC)

Fri Sep 27 08:14:05 2024 INFO: Fill completed in 14min 1secs

TITLE: 09/27 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 145Mpc
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 4mph Gusts, 2mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.44 μm/s
QUICK SUMMARY: Locked for one and a half hours, some shorter locks overnight as well. Verbal didn't mention any PIs for the last lock loss, but the length of the lock is suspicious. More investigation needed. Our range isn't optimal and the squeezing looks poor in the higher frequencies based on the nuc33 FOM.

TITLE: 09/27 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY:

IFO is LOCKING at ENGAGE_ASC_FOR_FULL_IFO. Fully auto so far...

Shift was mostly quiet. Despite locking issues today, IFO locked pretty automatically following Tony's initial alignment. We did lose lock during LOWNOISE_ESD_ETMX (558). Though, guardian brought it to NLN shortly after and we were in observing swiftly.

In terms of PI modes, there was one very harsh ringup 23 minutes into NLN (or 34 mins after MAX_POWER). This gave 3 verbal PI 24 alarms but the damping, even though at maximum, was able to bring it down. No other ringups.

There was one lockloss 04:08 UTC. probably attributed to the environment and rising secondary microseism/over 35mph wind combo - alog 80323.

TCS Work from today has 2 SDF Diffs (screenshot attached).
LOG:

Lockloss during Post-Event Standdown. Unknown cause not enviuronmental. Relocking now buyt having difficulties with both ALSY and ALSX.