Comparing POPAIR B now to the last good lock in July (before the damage), we only have 54% as much power in POP18 in full lock, in DRMI we have 71%.  At full power, the power on POPAIR B is 78% of what it was before the damage, in DRMI we have 86%.  This might indicate we have an alignment problem, which could perhaps contribute to our PI problems.  Other indicators suggest that our alignment is mostly OK: the arm power, PRG, coupled cavity pole, and kappa C all seem to be similar or slightly better than before the vent. 

I tried moving the BS camera pitch offsets, during the thermalization (our locks are short enough that we have to do things during the thermalization we might rather do later.)  This did seem to improve the arm power (more than the thermalization trend), but if anything it made POP18 slightly worse.  Screenshot attached.  We lost lock before being able to try yaw. 

For future reference, the ndscope I used for this is in userapps/asc/h1/templates/ndscope/move_camera_offset/move_camera_offsets_BS.yaml

LVEA was swept. Lights, WAP, paging system all off. Found nothing extra plugged in apart from usual PEM equipment.

There are a few unconnected BNCs connected to the PSL/SQZ racks that aren't being used. Some tools left on HAM5 pier -X+Y pier (didn't remove as in the middle of locking). SQZ bay crane is a few meters off it's parking spot. Farro is now plugged in near SQZ racks. Large clean-room in the +Y bay is plugged in but off (doubt this has changed).Still long extension cord to HAM3 dust monitor.

I am taking another look at what steps in lock acquisition take a very long time. See Georgia's previous alog on this: 76398

I noticed some of the power up steps seem to take longer than they should. I found some timers that are longer than necessary, so I am shortening them.

• Reduced 30 sec timer to 10 sec in "Power 10 W" > this is a timer to wait on ASC boost engagements that have a 1 and 6 second ramp time respectively
• loosened move spots ADS convergence thresholds from 0.6 to 1.2 > I tightened these thresholds by a factor of 10 previously (from 6 to 0.6)
• I combined a few steps of "lownoise ASC": changing the MICH gains with CHARD P, and doing all DHARD P and Y controls in one step reducing a few of the timer waits that were 5 seconds each
• I noticed the AS A yaw offset tRamp is set to 20 seconds in lownoise ASC, but then the step only waits 10 seconds before moving on, so now the tramp is set to 10 seconds
• I increased the Max Power ADS convergence threshold, was 0.001 is now 0.002
• I reduced the thermal waits in Max Power from 30 to 20

Just a note about many of the longer waits and tight convergence thresholds: powering up to 75 W was sometimes a very shaky process that required a much tighter convergence in the soft loops, otherwise runaway instabilities would often cause locklosses. We have previously relaxed some of these tight constraints to speed up lock but we didn't catch them all. This should take care of more of them (but maybe not all).

Naoki, Vicky, Camilla

After the vent, we can reach NLN, but we have locklosses from PI24 at 10.431 kHz. The SUS_PI guardian tries to damp, but it does not help. Vicky checked that the frequency of this PI is 10431.03 Hz as shown in the attachment. This is within the input bandpass filter (428.5-433.5 Hz after downconversion by 10kHz). 

We tried to excite the PI24 with ETMY. We used awggui and sent a sine wave at 431 Hz to H1:SUS-PI_PROC_COMPUTE_MODE24_BP_EXC. We could excite/damp the PI24 RMSMON by changing the damping phase so PI damping should work. We increased the damping gain for PI24 from 1000 to 10000, which is set by SUS_PI guardian.

Fri Aug 23 08:15:20 2024 INFO: Fill completed in 15min 15secs

TITLE: 08/23 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 3mph Gusts, 2mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.10 μm/s
QUICK SUMMARY:

IFO is in DOWN and MAINTENANCE

Continguing our work on tuning the IFO for observing following the recent vent

TITLE: 08/23 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Tony
SHIFT SUMMARY:
Took over from Tony at 6pm local (1utc) who was covering the first part of my shift. 

Elenna was getting H1 to NLN, well to the Inject Squeezing state, so she could hand off to Naoki for Squeezer optimization. 

H1 eventually made it to NLN and range was approaching 140Mpc, but then PI Mode 24 rung up againand had a lockloss (even with PI DAMPING for SUSPI). 

At any rate, with the lockloss, took the opportunity to run an Intial Alignment to capture the green references that Elenna got earlier today.  (I did have touch up SR2+SRM alignment by hand for SRC_ALIGN_OFFLOADED.)

Oh, also made the edit to ISC_LOCK to 'not' skip the SHUTTER ALS step.

The first attachment shows the DARM with/without squeezing today (blue: with squeezing, orange: without squeezing). The range is ~140 Mpc with squeezing and ~124 Mpc without squeezing. 

Above 1 kHz, DARM with squeezing is as good as the start of O4b and we have 4.8 dB squeezing at 2 kHz. The DARM below 1 kHz is worse and we will need to optimize FF and fix the calibration. We may also need to optimize PSAMS to improve the bucket sensitivity.

I ran SCAN_ALIGNMENT_FDS three times and accepted SDF as shown in the second attachment.
 

Both the X and Y arm green references have been set! Please run a full initial alignment at the next lockloss.

The X arm green references have been set several times and have stayed fairly the same; only the Y arm references have been an issue.

Procedural notes:

• after DRMI locked, we went to the DHARD WFS state, which is close to where the arm alignment control switches from the green WFS to the red (DHARD) WFS
• I started both "setEndGreenQPDOffsets_XARM.py" and "setEndGreenQPDOffsets_YARM.py" which are scripts found in "/opt/rtcds/userapps/release/als/h1/scripts"
• This changes the transmon QPD offsets that are in-loop with the PZTs on the table, controlling the pointing of the green light into vacuum. Therefore, while the arm alignment on red changes, the green alignment can follow and keep the cavity locked and co-aligned with green during these changes.
• Next, we stopped at "prep asc for full ifo" so I could once again step through the "engage asc" state by hand
• I had to walk some loops before turning them on, and I did similar SRC ASC walking as described in last night's recovery alog
• remember, "move_arms_dev.py", found in "/opt/rtcds/userapps/release/asc/h1/scripts" is a useful script for stepping arm alignment, such as CHARD
• Once all the alignment converged in "engage soft loops", I set the green references for both arms, following the steps I outline next

Green references steps:

• run the aforementioned green references scripts while waiting for the signals to converge
• Use "scope for green QPDs" found in sitemap>ALS>End X/Y overview>QPD offsets>!reset references drop down menu
• watch the bottom two scope plots for convergence of X and Y signals respectively.
• Once these signals converge, ctrl-C the scripts to stop them
• SDF green offsets in ALS X/Y
• Go to CAM ITM page on End X/Y overview screen
• average the Pitch and Yaw "position" signals for 30 seconds (command line: z avg -30 [channel name])
• Enter that new value into the "nominal field"
• SDF on ALS X/Y screen

SDFs for ALS X and Y are attached. The ALS X values have stayed fairly similar, and the ITMX CAM Yaw value I measured was exactly the same as the previous, so it did not appear as an SDF diff.

Final notes:

• I turned back on the PRC2 and INP1 DRMI ASC in the ISC_DRMI guardian. SRC DRMI ASC is still off, as is PRC1 (we should check the POP A offsets before turning this back on).
• If the initial alignment proceeds well, the next lock should not require any SRC1 offsets to lock. I am commenting out the SRC1 offsets in the "engage ASC for full IFO" state.
• We should add "shutter ALS" back into the path (I forgot to shutter ALS by hand and BOY does that cause a bunch of extra darm noise)

Today's activities:

- HAM7 turbo was valved out, and switched off. This pump needs ~3 hrs to spin down
- Immediately after this, FCV-2 was opened, so HAM7 was connected with the main volume via BSC3
- Then, RV1 (relay-tube remaining valve), FCV-4 and FCV-3 (valves towards HAM8 along the FCT) have been also opened

- Later, after the HAM7 turbo spun down, the backing supersucker cart was disconnected from it
- During the day, HAM6 RGA bakeout was ramped down. This RGA tree was also valved together with a small Ion-pump, which always keeps it under UHV.
- This RGA's filaments were also fired up. RGA scans of its small volume will be done in the upcoming days, then it will be valved together with the main volume (HAM6), and RGA scans will be done of HAM6, and these scans will be compared with the Corner RGA scan results

So, the interferometer is now fully open. The exact pumpdown time (from the start of the roughing to the opening of large GVs) was 14 days 17.25 hours. This is by far the shortest pumpdown so far. In dcc T2400085 the pumpdown will be more thoroughly analyzed, and will be compared with past pumpdowns, especially with the last one during February of 2024.

The pressures:
Corner (PT-120 and PT131): 6.3E-8 Torr
HAM6: 2.4E-7 Torr
HAM7: 1.25E-7 Torr
CP1: ~2.5E-8 Torr
CP2: ~5E-8 Torr

I reverted the Changes to ALS_ARM.py that were put in today because the DOF2 WFS were not working.

TITLE: 08/23 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 16mph Gusts, 11mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.08 μm/s
QUICK SUMMARY:

Slowly going Locking states so commisioners can fine tune full IFO Y Arm alignment , by stepping through PREP_ASC_FORFULL_IFO[429] manually. This is to set the Y arm Green reference & document settings.

Naoki, Vicky, Camilla, Sheila. Copied instructions from 78115.

Today we couldn't find the SQZ beam while the IFO was locked using  SQZ-OMC_TRANS_RF3_DEMOD_RFMON as we'd been ale to do in 78088. When the IFO unlocked we used the SQZ SEED beam in a SQZ single bounce (much higher power so easier to see) to  improve the SQZ alignment. Expect when we're locked will still need to touch alignment, may get away with a few rounds of SQZ_MANAGER's SCAN_ALIGNMENT_FDS state (wiki notes). Notes below.

• Aligned SRM, misaligned SR2/3  (ITMs being used for ALS troubleshooting). FC misaligned.
• Increased seed from 1mW to 75mW
• SQZ Manager LOCKED SEED DITHER (need to revert the hysteritis offset Naoki made yesterday 79630... don't know why)
• HAM6/7 WD untripped
• First tried to align just to AS_C: kept centered on AS_C with ZM4 while moving ZM5 to increase power on AS_C.
• This didn't go well and Vicky/Naoki/Shiela suggested I should really care about AS_A/B.
• Aligned OM1,2,3,SRM to their alignments in today's full lock. Opened fast shutter.
• Then we walked, centering AS_C with ZM4 and AS_A with ZM6. Then we touched ZM5 to bring  AS_B closer to center and ZM6 to bring back AS_A . At this point we could also see the beam be un-clipped on AS AIR camera. We centered on this and mostly centered on the OMC QPDs.
• Similar/better powers than 78115:  Got peaks of 900e-6 on AS_C_SUM, 70 of AS_A and AS_B and beams were mostly centered (at least within 0.4). as AS_A/B/C, AS_AIR and OMC QPDs. Plot attached of end alignment.
• Accepted in ZM4/5/6 alignments in sdf and brought the seed power back to 0.8mW.

TITLE: 08/22 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Corey
SHIFT SUMMARY: Another busy day of IFO recovery. See Elenna's alog for a summary of the morning's activities.

Since then, H1 lost lock from a ringup of PI mode 24. Naoki, Camilla, and the SUS_PI Guardian all tried to damp it down to no avail; why do we now have a PI that we cannot damp?

After the lockloss, we started back with troubleshooting ALS issues while team SQZ worked on recovering their alignment. TJ and I tried to find an arm alignment that both had the arm power and NSUM counts on ALSY WFS_B high, but we weren't successful. Eventually, Keita was able to help us recenter on both WFS A and B by adjusting the biases for both PZTs on the ISCEY table (found on the secret ALS-Y_WFS Auto Centering medm screen) after we locked the arm in green. These new bias settings ahve been accepted in the ALSEY SAFE.snap SDF table (screenshot attached). It turns out that with the centering being so off on the ALSY WFS, the auto-centering servo was railing, which didn't allow the ALS WFS to engage without us forcing them to. After centering on the WFS, we took the ALS_YARM Guardian to LOCKED_SLOW_WFS_ETM_TMS and the ALSY WFS engaged properly.

The next steps will be to lock the full IFO keeping ALSY locked to set the green camera references.

LOG:

The transition from the piston driven (non-dried) instrument air compressor to the new (inline dried) scroll style compressor was finalized last week. The compressor is presently left at its factory set pressure of 120psi. TJ raised the high limit threshold for alarms to coincide with this change in pressure. The new system has been delivering the facility with air without issue since last week.

Amidst this transition, I elected to pressurize the compressed air line between the chiller yard and mechanical room and remove any feed/demands from it as noted in my previous alog. There were no significant pressure losses over an 8-hour period and as such I have no reason to believe that the line feeding the mechanical room is in any way compromised.

Not shown in the pictures: Insulation and heat trace on lines that require it and weatherproof outlet box

E. Otterman C. Soike T. Guidry

Just a quick report now that we are back to NLN!

• After getting through various alignment details at 2W, getting to NLN is very smooth and requires no intervention
• The problem with the PI guardian last night was because I manualed over the "inject squeezing" state which also engages the PI guardian. PI damping engaged properly today!
• Naoki is working on SQZ alignment, reverting the ZMs to the July alignment
• Some visible excess noise in DARM below 60 Hz. A quick coherence measurement shows that a portion of this noise at least is LSC. We can retune the FF soon
• I ran a quick broadband PCal measurement that shows our calibration is off, unsurprisingly. Louis can help us recover the calibration over the next few days.
• We have still not set the y-arm green references, due to issues getting the ALS Y locked properly when in fully alignment. We can fix this on the next relocking
• We still have not solved the ALS Y WFS problem, and we still don't know what's wrong. However, we have improved the counts on ALS Y WFS B, which might help DOF2 run better when the loops are engaged.
• The current (not-well-calibrated and unsqueezed) range is reading at 123 Mpc

h1cam26 (BS) stopped updating around 22:22 PDT Wed 21aug2024.

I power cycled the camera via sw-lvea-aux POE and restarted the process on h1digivideo2. 

h1cam26 is on switch port 0/35

The offload of the past 6 month's of raw minute trend files from h1daqtw1's SSD-RAID to permanent archival on spinning media is complete. TW1's SSD-RAID disk usage has been reduced from 92% to 2%.

First NDS1 restart: 09:56 Tue 20aug2024

Start file copy: 10:53 Tue 20aug2024

File copy completed: 12:07 Wed 21aug2024 (25hrs 14mins)

Second NDS1 restart: 15:47 Wed 21aug2024

Start file deletion on TW1: 09:32 Thu 22aug2024

File deletion complete: 11:40 Thu 22aug2024 (2hrs 8mins)

I wanted to check the last lock loss concerning the power upstream/downstream of the fast shutter, but quickly remembered that the upstream signal, i.e. the non-whitened analog version of the ASC-ASC_SUM, was disconnected from the fast PEM ADC (H1:PEM-CS_ADC_5_19_2K_OUT_DQ) before the last vent to fix OFI, and it was not reconnected since.

I went to the floor and restored the connection.

Original alogs about this connection: 78968 and 79077.

TITLE: 08/21 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY:

LOG:

• 2320-2338 Going out to center OMC trans camera (tj, camilla)
• 1650 Started Initial Alignment & then went into locking
• 0407 PI Mode 24 alerts came up.  I went to the PI Overview screen, found the SUS PI guardian, and saw it was in the IFO DOWN state!  So I transitioned it to its nominal state of PI Damping.  But looks like PI24 is railing around 2000 on nuc25.