At 21:13:58 we had a crash of h1susb123 (ITMX, ITMY, BS) which in turn caused a SWWD DACKILL of the seismic front ends for these chambers.

Unlike the h1susb123 crash Sunday 11 May 2025 in which lscpi was slow but showed good cards, in this case lspci is responsive and shows a possible problem with the 2nd 18bit-DAC card.

Like the Sunday crash, dmesg does not give much information (listing below).

First thing to try would be a power cycle of the IO Chassis to see if the 18bit DAC is OK. Given the lateness of the hour we could delay this until the morning.

[Tue May 20 21:13:56 2025] rts_cpu_isolator: LIGO code is done, calling regular shutdown code
[Tue May 20 21:13:56 2025] h1iopsusb123: ERROR - An ADC timeout error has been detected, waiting for an exit signal.
[Tue May 20 21:13:57 2025] h1susitmpi: ERROR - An ADC timeout error has been detected, waiting for an exit signal.
[Tue May 20 21:13:57 2025] h1susbs: ERROR - An ADC timeout error has been detected, waiting for an exit signal.
[Tue May 20 21:13:57 2025] h1susitmy: ERROR - An ADC timeout error has been detected, waiting for an exit signal.
[Tue May 20 21:13:57 2025] h1susitmx: ERROR - An ADC timeout error has been detected, waiting for an exit signal.
 

HAM1 pressure has already gone below the threshold that we consider the suspensions to stop floating, so I've taken in vac transfer functions for RM1, RM2, and PM1. All three suspensions are looking healthy.

Earlier today, Fil completed the coil driver swap for PM1 (84490), so I updated the COILOUTF filters(84499), and now all filters (physical and digital) match the ones for the RM coil drivers (84499).

Note that these were taken with the ISI locked and HEPI offline.

RM1 (r12326(data), r12327(results)):

• Data: /ligo/svncommon/SusSVN/sus/trunk/HTTS/H1/RM1/SAGM0/Data/2025-05-21_0000_H1SUSRM1_M1_WhiteNoise_{L,P,Y}_0p01to50Hz.xml

• Results: /ligo/svncommon/SusSVN/sus/trunk/HTTS/H1/RM1/SAGM0/Results/2025-05-21_0000_H1SUSRM1_M1*

RM2 (r12329):

• Data: /ligo/svncommon/SusSVN/sus/trunk/HTTS/H1/RM2/SAGM0/Data/2025-05-21_0015_H1SUSRM1_M1_WhiteNoise_{L,P,Y}_0p01to50Hz.xml

• Results: /ligo/svncommon/SusSVN/sus/trunk/HTTS/H1/RM2/SAGM0/Results/2025-05-21_0015_H1SUSRM1_M1*

PM1 (r12328):

• Data: /ligo/svncommon/SusSVN/sus/trunk/HTTS/H1/PM1/SAGM0/Data/2025-05-20_2200_H1SUSRM1_M1_WhiteNoise_{L,P,Y}_0p01to50Hz.xml

• Results: /ligo/svncommon/SusSVN/sus/trunk/HTTS/H1/PM1/SAGM0/Results/2025-05-20_2200_H1SUSRM1_M1*

Comparison of all three HTTS (r12330):

• /ligo/svncommon/SusSVN/sus/trunk/HTTS/Common/Data/allhttss_May2025_BackInVac_H1SUS-RM1_RM2_PM1_ALL_TFs.pdf
• /ligo/svncommon/SusSVN/sus/trunk/HTTS/Common/Data/allhttss_May2025_BackInVac_H1SUS-RM1_RM2_PM1_ALL_ZOOMED_TFs.pdf

Yesterday I fixed the PM1 COILOUTF filters because we had accidently copied in filters from the RMs, which had a different coil driver set up due to an ECR that hadn't yet been done for PM1, so I corrected them (84455). Today, Fil changed out the PM1 coil driver so it now is correct as per ECR E2200307 and E2400048 (84490), so I've re-re-placed the PM1 coil driver filters with the now correct same filters that the RMs have. I've loaded them in.

Jordan, Travis

After yesterday's preparation works (see aLog 84475) the pumpdown procedure was started on HAM1.
2 purge chamber-air measurement was done, one was during purging; the purge air dew point was -40.3 deg C; the blowdown air dew point was -40.1 deg C. This is very dry, so a very good start for the pumpdown.
The pumpdown was started with the Supersucker cart's 500 l/min Anest Iwata scroll pump, then at <500 mTorr, the recently installed 500 l/s Leybold turbo was switched on. As the pressure dropped down to the E-5 Torr region, the purging all metal valve on the top of the chamber (valve for the new gauge) was closed.

In the meantime, the HAM1 gauge was wired (see aLog 84494). The channel was also added to DAQ for trending, and can be seen on the MEDM screen. Thanks for all of this to Fil, Dave, and Patrick!

At the moment, the pressure in HAM1 is 6.5E-5 Torr, tomorrow the chamber must be ready for leak check. In preparation for leak checking, all the double O-ring flanges were wrapped with Kapton-tape, to avoid flooding the annulus volume with He during leak checking - here note that because the feedthrus are all populated with cables, bagging of flanges will be hardly possible, so the area will be flooded with He anyways.

Tony, Jim

This afternoon we replaced the H3 L4C on HAM2 that I found broken a couple weeks ago. First we disconnected the actuator from the HEPI foot and slid it back onto a set of stairs that just happened to be the right height. Then we removed the spacers on the foot and pulled the pier chassis out of the way to allow access into the foot to release the clamp block holding the L4C in place. The access to this clamp was difficult, and took some patience with a small ratchet to get loose enough to release the L4C. Put the new L4C in and put the pier back together. The new L4C is working, but we were looking at the wrong IPS when tightening the actuator back up. I will have to go out tomorrow morning and loosen the actuator and slid it back to recenter the IPS. Not too hard, HEPI is locked , which will preserve the mechanical location. 

With the temporary HAM1 gauge PV created today and the recent HAM6 gauge upgrade VACSTAT is back to a full complement of gauges (first time since 26 Dec 2024).

Note that HAM1's slope and delta-p settings are currently non-nominal while it is pumping down, but it will alert if its vacuum glitches.

WP 12543

The new HAM1 vacuum gauge is powered up and connected to Beckhoff. Network cable to gauge is connected to the CU1128 Beckhoff hub in the biergarten (SUS-R6). One of the existing Pirani/cold cathode cables was used to provide the 24V from the LY vacuum rack.

Software configuration of the gauge is needed to read out the pressure from the Beckhoff system. Used PT100B cable to temporarily send analog signal from gauge to LY Rack to monitor pump down. See alog 84482 for details.

TITLE: 05/20 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: None
SHIFT SUMMARY: HAM1 is now pumping back down and will go overnight. Binary and coil driver chassis for RMs and PM1 were updated, see alog84490. The broken L4C at HAM2 is currently being replaced. All optics in HAMs 1 & 2 are now ALIGNED.
LOG:

Today we installed the missing dual demod chassis in ISC-R2, and Fil pulled the controls cable for the delay line. I then tested that the delay line can be controlled through EPICS and that there are readbacks from the demod chassis.

This complets the hardware installation for the LSC-REFL_B chain.

Attached is the Trello snapshot showing the last week or so of progress and this week's trajectory to opening gate valves and recommissioning the H1 detector.  Pretty on target.

WP 12554
Drawing D0902810-v11

The SUS-C3 and SUS-C4 racks in the CER were updated to match the D0902810 version 11 drawing. All binary chassis and associated cabling for the SUS IM’s were removed.

Verified all HAM-A Chassis front LEDs (Run/Acq and Enable/Disable) were powered on with no binary cables connected. The IM1-4, RM1-2, and PM1 were all checked. The PM1 HAM-A Chassis was replaced with a modified unit.

The following chassis were removed:

1. SUS Binary Out Chassis S1103230
2. SUS Binary Input Chassis S1103899
3. HAM-A Coil Driver S1106042

The following chassis was installed:

1. HAM-A Coil Driver S1106046

The SUS-C3 and SUS-C4 meta data has been updated.

TJ, Jeff, Oli

It was brought up that it would be nice if we had a guardian state for taking our regular suspension health check transfer functions, since it requires a few different things to be changed, some of which differ from sus type to sus type. So I created three new states, PREP_HEALTH_CHECK (67), which gets everything in the correct state for running our default transfer functions, HEALTH_CHECK (70), which is just an idle state, and REVERT_HEALTH_CHECK (73), which reverts the changes made in PREP_HEALTH_CHECK. I set the edge of PREP_HEALTH_CHECK to come off of FULLY_ENABLED, and then REVERT_HEALTH_CHECK comes back around and goes into FULLY_ENABLED (state graph).

The things that need to be true for every suspension type are:

• Top mass coil driver state set to 1 (unless permanently jumpered to 2) (here are the optics+stages for LHO that need their coil driver states changed)
• Test bank outputs switched ON
• Test bank gains set to 1
  • making sure to save the gain and then put it back when leaving the HEALTH_CHECK state
• Damping bank outputs OFF
• Alignment biases OFF

Additionally, there are some sus specific requirements:

• (QUAD R0) Turn off any Main Chain L2 to Reaction Chain R0 damping/tracking
  • aka turn off the outputs to the R0's L2 WIT DAMP block (SUS-{QUAD}_R0_L2DAMP_{L, P, R, Y})
• (BSFM) Turn off any optical lever damping loops
  • aka turn off the outputs to M2's OLDAMP block (SUS-BS_M2_OLDAMP_{P, Y})
• (TMTS) Turn off any QUAD to M1 damping / tracking
  • aka turn off the output to M1's FF_L block (SUS-{TMTS}_M1_FF_L)
• (OMCS / OMs / RMs) Turn off any ISC DC centering input
  • aka turn off the output to M1's LOCK block (SUS-{OMCS, HTTS}_M1_LOCK_{L, P, Y})

Ibrahim, Betsy

Today Betsy and I successfully installed the shear plates onto the BBSS. In the attached pictures, you can see:

• the shear plates with their half coating
• the Y stiffness brackets that attach the Upper structure to the PUM tablecloth to the shear plates
• The gap for the HRTS mount

Next, I intend to install the HRTS mount, the HRTS itself and start B&K (with Rahul)

FAMIS 26044, last checked in alog84268

The script reports BSC high frequency noise is elevated for the following sensors:

ITMX_ST1_CPSINF_V1
ETMY_ST2_CPSINF_V2

Dry air skid checks, water pump, kobelco, drying towers all nominal.

Dew point measurement at HAM1 purge inlet port, prior to pumpdown, -43.7 °C. 

The HAM1 purge exhaust dewpoint at top of chamber was measured to be -40.3C just before turning off the purge and starting pumpdown.

Jonathan, Patrick, Janos, Ryan, Dave:

The HAM1 pressure gauge PT100 is temporarily only available as a raw voltage signal (H0:VAC-LY_X0_PT100B_PRESS_VOLTS).

I wrote an EPICS IOC which reads this voltage signal, converts it to a pressure (Torr) and exports it as a "H1" version of the PT100B cold cathode signal. The new EPICS channel is:

H1:VAC-LY_X0_PT100B_PRESS_TORR

The new IOC is currently running in a tmux session on opslogin0. It was started at 11:31 Tue20may2025, at which time HAM1 was still at atmospheric pressure (10.0V, 750Torr)

WP12558 was opened to add this new channel to the DAQ so that this week's HAM1 pump down could be trended.

We restarted the DAQ 0-leg at 12:16, the EDC at 12:17 and the 1-leg at 12:22

I took the opportunity to complete WP12523 and add the new CDS rack power strip EPICS channels to the EDC.

EDC details:

Addition of H1EPICS_VACHAM1.ini (3 channels) and H1EPICS_PWRSTRIP.ini (36 channels) for a total additon of 39 channels, which give a new total of 59,311 chans in the EDC.

I added the new HAM1 pressure channel to the vacuum overview (see attached).

The pump down was already on its way when we added it to the DAQ, attached trend shows data is available from 12:22 onwards, at which time the pressure was at 200Torr

Gerardo, Jordan, Travis, Randy, Janos

Today between 10 am and 2 pm both the Y+ and Y- HAM1 doors have been installed. The installation went smoothly, no issues were found.
A valve on the new interlock gauge-assembly was opened, and the HAM1 volume will be purged overnight. This is to clean the in-vacuum surfaces before pumpdown, resulting in lower gas load and therefore faster pumpdown.
After the door installation, the pumpdown of the joint HAM1/HAM2 Annulus volumes have been started. 2 turbo/pumping cart systems are pumping the volume, they are standing next to HAM1 and HAM2 - see attached photo.

Please do not move the pumping carts, as it can result in immediate damage in the turbo pumps. If you still need to do any activities which need these pumping carts to be moved, please talk to vacuum team members first!