FAMIS 31128
The PMC HV supply was swapped 6 days ago (alog89349) which took the PMC down briefly. Afterwords, we did some slight PMC alignment and bumped up the pump current for PS4 to get slightly better PMC transmission. The PSL models were restarted on Friday due to timing errors (alog89413), which also briefly took things down, but everything came back without issue.
Mon Mar 09 10:16:12 2026 INFO: Fill completed in 16min 9secs
WP13077 Replace suspect first LIGO-DAC in h1sush12
Sheila, Marc, Erik, Dave:
We replaced the first LIGO-DAC in h1sush12's IO Chassis. Its ribbon cable and interface card were also replaced.
Replacement parts were removed from the DTS x7eetest1 system.
No DAQ restart was required.
JM1 and JM3 moved after this restart because of some old values in SDF for their alignment sliders, so once things were brought back up, Sheila and I realigned the suspensions and I accepted their alignment offsets in SDF. Screenshot attached.
Old Serial numbers
DAC: S2500460
Interface card: S2500449
New serial numbers:
DAC: S2500445
Interface card: 34924
Taking a look at some of the MC1 signals from over the weekend with Sheila and Jeff, we've decided to next swap the DAC card, card 0 on h1susmc1. The OSEMs are moving but not seeing a request in the MASTER_OUT. T3 noisemon seems to see it first.
TITLE: 03/09 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 4mph Gusts, 1mph 3min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.26 μm/s
QUICK SUMMARY:
The annulus ion pump for the Y-Beam manifold railed on Saturday around 9:00 PM local time. However, the signal for the controller did an odd oscillation before railing, see attached plot. Nothing noticed in the main vacuum volume signals (pressure rise), we will diagnose this system as soon as possible, we need to verify what the problem is, the ion pump body or the controller.
Sun Mar 08 10:18:18 2026 INFO: Fill completed in 18min 14secs
Sat Mar 07 10:18:00 2026 INFO: Fill completed in 17min 57secs
Erik, Dave:
I have built and installed Oli's first versions of the HAM6 SUS models: h1susom0, h1susobs, h1susam, h1susomcab, h1susom1ab, h1susom2ab and h1susom3ab.
Currently h1sush6 is running without Dolphin IPC and is not connected to the DAQ. Therefore to get this early running I edited the models and removed the remaining H1:LSC-TRIG_IFO_PCIE part from the om0, obs, am, om[123]ab models.
Only the iop model is in puppet, so I had to also do some hand editing of the rtsystab file to get the models to start.
I made a temporary edit to generate_cds_overview.py to add these models to the CDS Overview. To make space I simply stretched the window vertically. I'll reorder the lower section so the SUS column can take the full window height (also additional models will be added to h1sush5 and h1susb2h34).
Next week we will put all of this into puppet and add these models to the DAQ.
Note that the RED, WHITE DAQ blocks are due to the system not being known by the DAQ nor is it connected to the DAQ network.
S. Muusse, T. Roocke
All chassis were put on the rack for the Y table and cables attached. All electronics were tested and working however, we have realised additional power supplies are required and some additional cables are required.
Power supply/ cables still required:
A potential solution is to use pre designed 15V, 12V, 5V, regulator boards in another chassis which can be input to the table through the blank and power feedthroughs. Alternatively there are some preexisting supply boxes for 12 and 15 V
TITLE: 03/07 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY: We ran into issues trying to lock the IMC after locking JAC and GREEN_ARMs, it seems to be from MC1 (T2 and T3 mostly).
LOG:
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 15:37 | FAC | Contractor | MidY | N | Unloading bricks for CEBEX | 17:21 |
| 15:38 | FAC | Randy | EX | N | Bring ergo lift back to CS High bay, 15:57 | 17:34 |
| 16:12 | FAC | Kim, Nellie | LVEA | N | Tech clean, Nellie out 16:48 | 16:58 |
| 16:28 | SEI | Jeff | Optics lab | LOCAL | SPI work | 22:45 |
| 17:01 | PSL | Sheila | LVEA | N | Unlock rotation stage to turn up to 1W then relock, in @ 17:09 | 17:18 |
| 17:19 | ALS | Sheila | LVEA HAM1 | Y/LOCAL | SGH table alignment work | 17:51 |
| 17:34 | FAC | Randy | High bay | N | Clean lift | 17:48 |
| 17:44 | TCS | Sophie | Cheeta lab | N | Plug stuff in | 17:57 |
| 18:29 | FAC | Contractor | MidY | N | Unload bricks | 21:29 |
| 18:54 | VAC | Jordan | LVEA | N | Check HAM1 aux cart | 19:07 |
| 18:56 | TCS | Sophie | Cheeta / Prep lab | N | Cheeta work | 20:22 |
| 19:07 | EE | Marc | LVEA, HAM1 | N | Terminate cables | 22:20 |
| 20:16 | CDS | Dave | MER | N | DAQ chassis work | 21:31 |
| 21:25 | TCS | Sophie | Prep lab | N | Cheeta work | 22:04 |
| 21:48 | ISC | Masayuki | LVEA | N | Turn on whitening | 22:09 |
| 22:05 | TCS | Sophie, Thomas | Prep lab | N | Cheeta | 23:30 |
| 22:34 | EE | Marc | CER | N | See about swaping HAM2 MC1 M1 coil driver | 22:38 |
| 22:47 | EE | Marc | CER | N | Coil driver swap MC1, in at 22:55 | 23:25 |
| 22:56 | EPO | Erik + family | LVEA | N | Quick tour | 23:13 |
16:37 EX sensor correction off as the BRS was ringing up from Randys' loading of the ergo lift.
17:18 Sheila unlocked the rotation stage and I turned up the power to 1W, Sheila then relocked it.
17:35 UTC I used the ALS nodes to run an alignment for GRREN_ARMs, ALSX locked after a few taps of ETMX but ALSY took more convincing
19:30 UTC I started to run TFs for MC1, then MC3 aand RyanS did MC2
23:10 UTC SUSHB13 crashed, possible from Marc's work in a close by rack
23:17 UTC ITMX, ITMY WDs tripped, SW, ISI ST1, ISI ST2, and HEPI
23:31 I remeasured MC1 P and confirmed that the Coil Driver swap was unsuccessful, the TF still looked the same. I also ran an undamped OSEM spectra before and after the swap.
All the h1susb13 models stopped with timing errors this afternoon, followed later by h1psl0's iop model. CER activity was ongoing around this time.
We tried a model restart on h1psl0, which did not work. We rebooted the machine and all started OK.
We tried a reboot of h1susb13, but it got stuck. We then hard-reset the computer via IPMI and all came back OK.
Following the recovery of h1iopsusb13 I untripped the BSC1 and BSC3 SWWDs (the SEI had tripped).
Daniel had a new h1ascimc model to install (no DAQ restart). We stopped h1ascimc, which caused a DAC error on h1iopasc0. So as before, we had to restart all the h1asc0 models to load a new h1ascimc.
Per WP13076 we replaced the coil driver for MC1 due to suspicion it was causing MC1 to jump. The coil driver in question glitched on TOP3. Referencing D0902810-v12, the chassis in question is U41 on SUS-C4, we exchanged it with a spare.
Old SN - S1001097, New SN - S1100194
Shelia will retest the signal chain to see if this solves the issue.
It did not, at least with comparing a before/after transfer function. See alog89409.
I was adjusting the alignment of MC1 pitch earlier, when the suspension had a large alignment shift that wasn't caused by a change in the requested drive to the DACs. The attached screenshot shows my requested alignment shifts in opticalign, and the resulting change in the requested DAC counts on T2 + T3. There is a sudden 420 urad jump in the osem readbacks from pitch, and a 133 urad jump in roll at the first time cursor. The master outs only see a few cycles on oscillations, which probably is from the damping loops responding to the jump, but they settle to the same requested drive as before the jump. The jump seems to be real as the IMC flashes disappeared at this time.
I ran an undamped no alignment offsets suite of transfer functions for MC1, and MC3, the state of HAM2 at the time was HEPI locked, ISI isolated. MC1 Pitch was the worst looking TF with multiple extra peaks. Looking at the OSEM signals for MC1 T3 shows a questionable drop during the time Sheila was looking at.
MC1:
/ligo/svncommon/SusSVN/sus/trunk/HSTS/H1/MC1/SAGM1/Data/
2026-03-06_1930_H1SUSMC1_M1_WhiteNoise_P_0p02to50Hz.xml
2026-03-06_1930_H1SUSMC1_M1_WhiteNoise_R_0p02to50Hz.xml
2026-03-06_1930_H1SUSMC1_M1_WhiteNoise_V_0p02to50Hz.xml
2026-03-06_1930_H1SUSMC1_M1_WhiteNoise_Y_0p02to50Hz.xml
2026-03-06_1930_H1SUSMC1_M1_WhiteNoise_T_0p02to50Hz.xml
2026-03-06_1930_H1SUSMC1_M1_WhiteNoise_L_0p02to50Hz.xml
MC3:
2026-03-06_2130_H1SUSMC3_M1_WhiteNoise_L_0p02to50Hz.xml
2026-03-06_2130_H1SUSMC3_M1_WhiteNoise_P_0p02to50Hz.xml
2026-03-06_2130_H1SUSMC3_M1_WhiteNoise_R_0p02to50Hz.xml
2026-03-06_2130_H1SUSMC3_M1_WhiteNoise_T_0p02to50Hz.xml
2026-03-06_2130_H1SUSMC3_M1_WhiteNoise_V_0p02to50Hz.xml
2026-03-06_2130_H1SUSMC3_M1_WhiteNoise_Y_0p02to50Hz.xml
I ran the same set of transfer functions for MC2 while Ryan did the other MCs; nothing suspicious here.
Templates' location: /ligo/svncommon/SusSVN/sus/trunk/HSTS/H1/MC2/SAGM1/Data/2026-03-06_2145_H1SUSMC2_M1_WhiteNoise_{P,R,T,V,Y,L}_0p02to50Hz.xml
Remeasuring MC1_P after Marc swapped the coil driver reveals that the coil driver was not the issue.
(Travis S., Gerardo M.)
Pumpdown was started this morning, sorry no blowdown data. By the time I remembered to take the reading there was no more pressure inside the chamber. Travis connected all of the equipment while I wrestled with the viewport inspection.
Pumpdown was started without an issue. We started with the scroll pump from the super sucker 500, the scroll pump of the SS500 cart brought the pressure down to 3.7x10-01 Torr, then we switched the cart to start pumping with the turbo pump/scroll pump. We had one trip of the system, usually due to a time out, but it only took a restart of the turbo pump and the pumpdown to continue.
Unfortunately the pumpdown had to be stopped at 8:00 PM local time, with a pressure of 4.0x10-05 Torr, at that pressure the set points had not activated on the SS500 cart controller, the set points are hard set and can't be modified from the SS500 controller screen.
Pumpdown will be restarted tomorrow morning.
Note, on the attachment, regarding the screens, no set points for channel 1.
Pumpdown Update.
Last Tuesday, HAM1 pumpdown was stopped for a few minutes to move the SS500 cart, the cart was moved away from the -Y door to open up space for the table. After the relocation of the SS500 cart the pumpdown for HAM1 was restarted.
On Thursday we assisted Filiberto with the installation of a "split" cabling for the high voltage interlock at HAM1 gauge. Attached is a photo of the screen of the SS500 cart and its response to loosing gauge signal, the signal cable was removed from the vacuum gauge to install the "split". I set valve control to "open" and the override to "override" on the SS500 cart to prevent it from tripping, it worked. BTW, when the cart reported the Error-S it cleared the setpoints. Once the cable was restored and we had a signal at the SS500 cart, the setpoints were restored manually.
Pumpdown continues, pressure is currently at 2.77x10-06 Torr, attached is a trend of the pumpdwon pressure.
The HAM1 +Y door was installed today. Installation went smoothly and the O-ring cooperated by staying in place today. I will note that torquing the door bolts is becoming more of an issue due to wrench access being increasingly limited near the 5-way cross/feedthru areas now that all the in-air cables have been attached.
The HAM1/HAM2 annulus system is being pumped by 2 turbos/aux carts on the -Y side of the chambers. The mysterious ~order-of-magnitude pressure readout difference at the two carts on the shared annulus system is again present, as was noted the last time this annulus system was pumped down. Not an issue per se, just some vacuum phenomena that leaves us scratching our heads.
Vent Recovery Update
Annulus ion pumps for HAM1 and HAM2 continue to pumpdown, we closed the isolation valves for both of them, but the ion pumps were not able to maintain pressure. On 3/4/26 valves were opened again to continue using the aux carts, and yesterday (3/5/2026) we isolated the aux cart for HAM2 only, all of that can be seen on the attached plot.
Aux carts will be left pumping and annulus systems are going to be evaluated next Monday.
I isolated the HAM1 aux cart (HAM2 was isolated on Friday) from the annulus this morning. The IP current only rose to ~4.7mA before turning around showing it could hold the annulus pressure.
HAM1/2 annulus aux carts were shut down and decoupled this afternoon. The SS500 cart remains at HAM1 and will continue to pump until the pressure is low enough to valve in the HAM1 ion pump.
Now that HAM7 is at sufficient pressure (6.4E-7 Torr as of this post) we helium leak checked the two re-installed 12" CF blanks removed for table locking and the entire relay tube assembly which had been removed for the viewport adapter on HAM5.
The Helium background was unstable, so we turned on the HAM7 cleanroom (on 2/18 afternoon) to help flush the area of helium overnight.
Today the background stabilized around 1.5E-10 Torr-l/s, so we were able to continue leak checking. No significant helium signal observed above the leak detector background which hovered between 1.5&2E-10 Torr-l/s during leak checking.
HAM7 continues to pump down with the turbopump, next steps are chamber RGA scans.
HAM7 RGA scans and scan information posted to T2600064. RGA is dirty, but the chamber comparison to last pumpdown (Mar 2024) is posted.
Late entry.
Last Tuesday, 2/24/2026, HAM7 ion pump was introduced to the chamber's main volume, and currently the chamber is pumped with both; the ion pump and the turbo pump. Attached is a trend snap of the chamber's pressure since the introduction of the ion pump.
Vent Recovery Update
On Wednesday (3/4/26) the turbo pump was isolated from HAM7 to leave the ion pump take care of the pumpdown.
Since the ion pump was able to take of the pumpdown on its own, on Thursday the SS500 cart then was turned off, and eventually detached from the turbo pump at HAM7. HAM7 is back to normal operations.