TITLE: 11/01 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
INCOMING OPERATOR: Oli
SHIFT SUMMARY:

IFO is in NLN and OBSERVING since 02:36 UTC

Mostly a quiet and uneventful shift, with the exception of the squeezer failing to lock for abbout 50 mins (until we lost lock - Lockloss Alog 80985). Vicky reverted some changes to SQZ that were made today (comment alog 80984) I accepted those SDF changes (attached).

Microseism is still high and rising, but doesn't seem to be too much of an issue.

LOG:

None

Lockloss while troubleshooting SQZer not locking (before going into OBSERVING). No IMC tags on the Lockloss Tool.

[Jenne and Betsy in PSL enclosure; Jason logged in remotely; Marc, Fil, Rick, Fernando as EE support; consults with Daniel]

At today's commissioning meeting, we decided that one candidate thing to try to help mitigate the lockloss-causing glitches would be to swap in one of the spare TTFSS boxes.  We also noted that if it was going to happen, it had to happen today, or wait until Mon/Tues next week.  So, we tried swapping in a new FSS box.  We were unable to lock the RefCav with either spare without significant oscillations, so ended up putting back in the original box, and that's how we'll leave things at least for the weekend.

Each time we changed out a box, we followed the steps 1-5 of our work permit 12182 (listed also below).  In particular, before unplugging the 37 pin connector on the back, we had one of the support folks turn off the high voltage in the mezzanine of the CER (I think it is a pair of Kepcos, one +150V, one -150V).  Then, once we had the 37 pin connector secured in the new box, one of the support folks went back to the mezzanine and turned back on the high voltage. 

For both spares, the autolocker was able to find a ref cav resonance and hold it there.  However, even with both the FAST and COMMON gain sliders set to the minimum values of -10dB we saw significant oscillation of the loop.  An idea I had later (have not yet checked with anyone else to see if it's crazy or not) is to try one of the spare boxes, but with an attenuator on one of the cables, to see if it really is just a loop gain thing, or if there is something else going on. This could potentially be tried next week.

We first tried the 'blue box' spare, since it is supposed to be the same (including form factor) as the in-service unit.  We noted that this box has had trouble in the past, but Marc said he'd taken a look and made some fixes, so it should work.  Since that one caused oscillations and had previously been known to be challenging to work with, we tried also the 'silver box' spare, which I think has also been referred to as 'Rev-C'. 

The silver box spare is a different form factor and different front/back panel than the blue box style, but it's supposed to be a drop-in replacement.  There is no space to mount the water-cooled cold block, so if we had left the silver box TTFSS in place, we were just going to leave the cold block hanging around on top of the box, but not mounted to anything.  Also, on the blue boxes, there is a SLOW output on the back panel that goes to a RAMP IN on the front panel.  However, on the silver box that SLOW output is on the front panel, and it needs to be connected to the FAST OUT2 also on the front panel.  Finally, the spigot labeled "PC" on the blue boxes is labeled "EOM" on the silver box.  To install the silver box, you'll also need 2 adapters to go from TNC (on the silver box) to BNC (what the cables are, and what the blue box is), so bring those into the PSL enclosure with you. 

Steps from the work permit submitted by Jason:

1)Unlock FSS RefCav.
2)Turn off High Voltage power supply in CER mezzanine.
3)Enter PSL enclosure and disconnect in-service TTFSS box.
4)Install spare TTFSS, hooking the same cables to the same ports on the TTFSS box; turn HV supply back ON.
5)Try to lock RefCav.

If it locks:

6)Measure TF to set common gain for ~500kHz UGF.
7)Measure crossover (use SR785, take a spectrum of IN1 port, crossover is at ~20kHz), set Fast gain to minimize hump in spectrum.

If it will not lock, then follow the same steps above to re-install the in-service TTFSS box.

TITLE: 10/31 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY: Currently relocking and at DARM_TO_RF. Spent most of the day unlocked, with two locklosses and multiple hours of downtime while trying to figure out the FSS issues. Relocking since that work finished has gone great so far.
A couple minutes after Jenne and Betsy left the PSL laser room(and maybe also antetroom), we did notice the FSS PZT have the huge oscillations for about two minutes(attachment), similar to those we had been seeing on the backup FSS boxes(probably alog to come). Jenne is not sure what that means in the case of this FSS box, but mentioned that it might have been due to turning off the air conditioning or something else related to them being in there.
LOG:

14:30 Close to end of an initial alignment
14:40 Initial alignment done, relocking
15:25 NOMINAL_LOW_NOISE
15:32 Observing

16:01 Dropping Observing for Commissioning

16:22 Lockloss
    17:17 Took to DOWN and started manual initial alignment due to DRMI being bad and PRMI ASC having trouble
    17:31 manual IA done, started relocking
18:13 NOMINAL_LOW_NOISE

18:35 Lockloss from little earthquake+low CARM gain actuation range
    18:38 Started an initial alignment
    FSS unlocked due to measurement cable being plugged in and was unlocked for an hour
    20:05 FSS relocked, started manual initial alignment for PRC, MICH, SRC
    20:21 Left manual initial alignment for IDLE for PSL FSS box swap
    22:59 PSL incursion done, started an initial alignment
    23:17 Initial alignment done, relocking                                                                                                                                                                                                                                               

TITLE: 10/31 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 18mph Gusts, 14mph 3min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.80 μm/s
QUICK SUMMARY:

IFO is LOCKING and recovering from CORRECTIVE MAINTENANCE.

To solve some of the PSL/IMC issues, IFO went into maintenance in order to swap out the FSS box. Unfortunately, both spares had PZT oscillations so we had to revert the swap. Attempting to get observing through the weekend until further potential action.

Vicky was concerned that the SQZ_SHG stays trying and failing to lock even when the IMC is unlocked. Driving the PZTs unnecessarily and maybe injecting excess light into the fibers.

We edited ISC_LOCK's DOWN to now take SQZ_MANAGER to DOWN and then ISC_LOCK's READY (had to main a main state) to take SQZ_MANAGER to FDS_READY_IFO in.  Previously requested SQZ_MANAGER to FDS_READY_IFO in DOWN.

Also edited SQZ_MANGER's DOWN to take SQZ_SHG and SQZ_PMC to DOWN. Reloaded all guardian. Previously didn't touch them.

Vicky and I plugged in the 785 to the IMC common mode board.   The first attachment was taken while Oli was trying to lock DRMI, with 10W into the IMC for CHECK_MICH_FRINGES, ugf just over 4kHz.  The second attachment was taken at full lock, nominal low noise, a nearly 80kHz UGF.  This is somewhat higher than in March 76753

Detector is currently in IDLE while the FSS box gets swapped in an effort to solve our PSL/FSS issues.

Lockloss @ 10/31 18:35UTC due to small earthquake, high microseism, and low CARM actuation range(80969)

Thu Oct 31 10:07:19 2024 INFO: Fill completed in 7min 16secs

Jordan confirmed a good fill curbside.

We re-enabled FC beam spot control, by enabling inputs to the H1:SQZ-FC_ASC_INJ_ANG_{P,Y} filter banks. Seemed to just work (see attached screenshot) - moves ZM3 to control the spot on the green QPD in filter cavity transmission (SQZ-FC_TRANS_A on SQZT8). Have not yet checked the centering, last done in May 2024 (see 77942 , 69276 , 69374 , 73777).

QPD offsets are set in the FC_ASC_INMATRIX filter banks, last column and row.

I set the flag in sqzparams.py (use_fc_beamspot_control = True), and reloaded SQZ_FC guardian so it will be used in the next locks.

Yesterday we went back to the higher range CARM configuration, in which the fast/IMC locklosses first appeared  80938.  Overnight we had two observing locklosses tagged with the IMC, neither of which had glitches visible in the FSS fast mon channel, 1414411965 (1st attached screenshot), and 1414381791. 

This morning I changed the gains by 6 dB in the other direction, so we further limit the range available to the frequency actuator. (CARM sliders and ndscope) Our microseism is around 5um/second right now, slightly above the 90th percentile, but there was no wind and no earthquakes this morning.  At times the IMC splitmon was reaching -8V, it's range is +/- 10V so this is about as far as we can go in this direction, and might be kind of marginal.

We've since had one lockloss 1414426971, this is tagged as commissioning because we were in commissioning time at the moment, but no one was actively doing anything at the time so this wasn't caused by commissioners.  There was a glitch in the FSS fast mon for this lockloss, although it was below the threshold for the tag to be applied, and around the time of that glitch the drive to the ISS AOM becomes noisy.  We were not saturating the IMC splitmon at the time of the lockloss so that doesn't seem to be the cause.

Lockloss @ 10/31 16:01 UTC after almost an hour locked

All models on h1seih45 glitched at 07:09:46 PDT (14:09:46 UTC). This appears to have broken the lock and caused guardian to start an initial alignment.

Symptoms are:

• All IPC receivers, Dolphin and SHMEM, on all h1seih45 models reported a single receive error
• DAQ (both DC0 and DC1) reported a single CRC error for all h1seih45 models
• dmesg, DAQ and central loggers show no logs at this time
• No evidence of a timing glitch at this time

The models appeared to have resumed normal operation following the glitch, no restarts were needed. At the time of writing H1 is locked with a range of 160MPc.

Reran all O4b locklosses on the locklost tool with the most up to date code, which includes the "IMC" tag.

I used the lockloss tool data since the emergency OFI vent (back online ~23rd August) until today and did some excel wizardry (attached) to make the two attached plots, showing the number of locklosses per day tagged with IMC and without tag IMC. I made this plots for just locklosses from Observing and all NLN locklosses (can include commissioning/maintenance times), including:

• No IMC locklosses August to September 13th (Iain checked prior to August G2401576)
• Setpeber 12th: IMC gain redistribution 79998
• October 9th: IMC gain redistribution reverted 80566
• October 15th: NPRO current increased 80687
• Ocotber 18th: NPRO current reverted 80746

J. Freed

I followed Step 1 of the double mixer test plan in T2400327 under "1. Characterize frequency references." 

PNSPIDACTestStand.pdf Shows the results in dBc/Hz. And PNSPIDACTestStandrad.pdf shows the result in rad/sqrt(Hz).

Besides the 80MHz standard reference there were 2 devices under test, the IFR 2023A and the SRS SG382. These two devices went under 3 tests involving different combinations of

1. Timing to a 10MHz frequency standard produced by the 80MHz Standard. (Time Standard)

2. PLL locking to a different 80MHz frequency ref through the tune in port. (Lock)

During this process it was assumed that the 80MHz standard and the added 80MHz ref would have a simmilar noise profile as they come from a simmialar OCXO.

A 4th test was added to the Time Standard/No Lock test which used the SR785s internal high pass filter (-3dB at 0.16Hz) to remove some of the DC components from the test. This is because the results of the initial test were inconclusive as the noise floor of the SR785 was too high. The noise floor was too high because there was a DC signal that caused the input range of the SR785 to be about 20dB. The high pass filter, removed the DC signal, lowering the noise floor at the cost of signals below about 0.2Hz not being accurate. We believe this is caused by the fact that while the PLL locking can be controled (tries to lock signals to destructivly interfere so to lower output power for Phase noise tests), there was no control by the Time Standard on phase differences between oscilators. As Time Standard/ No lock, showed the best results above 0.06Hz and is closest to the SPI set up design, this fact may be important when doing futher tests.

Between IFR and SRS, the SRS showed better phase noise performance below 7kHz. While IFR showed better performance above 7kHz. As such, SRS shows more promise in SPIs phase noise range of interest 0-4096Hz.

Continuing on from Jenne's observation that there are still glitches in the new NPRO, I've tried to make a plot we can use to compare the glitch rate in the new and old NPROs, using the NPRO_PWR channel instead of the FSS channel which isn't available for the new NPRO. 

I've used a 3 hour stretch of observing time for the old NPRO, and a three hour stretch before the time when Jenne made a plot in 80837.  The ISS is not on for the new NPRO time, which is probably why the intensity is flcutuating and making it harder to see the small steps in power that are the glitches we are looking for.  In the second panel, I've plotted the data high passed with a 0.05 Hz butterworth, this helps to show the glitches, although not perfectly (in either case).  Based on this, the glitches look to be happening at a roughly similar rate, although somewhat less with the new NPRO.

This script in in sheila.dwyer/DutyCycle)4/dutycycleplots/PSL_glitches.py

Sheila, Vicky, Camilla

We have turned back on the SQZ angle servo using the ADF at 322Hz. Last briefly tried while testing ADS alignment in ADS in 80194.  Turned on ADF and used 'python setADF.py -f 322'. Then set H1:SQZ-ADF_OMC_TRANS_PHASE to get H1:SQZ-ADF_OMC_TRANS_SQZ_ANG close to zero and checked by stepping the SQZ angle that there is a zero crossing in the ADF measured SQZ angle, plot attached.

• Accepted attached sdfs.
• Edited SQZ_ANG_ADJUST nominal state to ADJUST_SQZ_ANG_ADF and reloaded.
• Changed sqzparams flag use_sqz_angle_adjust = True and reloaded SQZ_MANGER GRD.
• Made to edit SQZ_MANGER to turn on SQZ_ANG_ADJUST once we get to FREQ_DEP_SQZ and to turn it off at the start of SCAN_SQZ_ANG and SCAN_ALIGNMENT. We tested by going though scan sqz ang.

The servo adjusts the SQZ angle (H1:SQZ-CLF_REFL_RF6_PHASE_PHASEDEG) via keeping the ADF measured angle (H1:SQZ-ADF_OMC_TRANS_SQZ_ANG) at zero. Setpoint can be adjusted using the ADF phase (H1:SQZ-ADF_OMC_TRANS_PHASE).

Tagging Detchar:  ADF is now on at 322Hz. It was turned all the way off in 79573 by Alan. We can adjust the frequency 50-500Hz if there is a better place for a line.

Note to operators: if you want to run SCAN_SQZANG, the ADF servo will now overwrite the sqz angle. So BEFORE going back to FREQ_DEP_SQZ  you'll want to tweak H1:SQZ-ADF_OMC_TRANS_PHASE (via sqz overview > ADF) to make H1:SQZ-ADF_OMC_TRANS_SQZ_ANG close to zero. Or you can tweak H1:SQZ-ADF_OMC_TRANS_PHASE (via sqz overview > ADF) until the SQZ BLRMs/ DARM is best.