WP13054 h1guardian1 reboot
Ryan S, Dave:
At 07:59 PST we rebooted h1guardian1. This was a maintenance reboot, it had been running for 74 days (since 19dec2025). No changes to the OS were made.
18 nodes had been pending loading the modified ISC_library.py common file. Ryan had checked that none of these nodes were using the modified section of this file.
There were no problems with the reboot and restart of the 176 guardian nodes.
ALIGN_IFO NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:12 Fri 19dec2025
ALS_COMM NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
ALS_DIFF NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
ALS_XARM NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:12 Fri 19dec2025
ALS_YARM NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
CAMERA_SERVO NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
H1_MANAGER NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:34 Fri 19dec2025
INIT_ALIGN NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
ISC_DRMI NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:12 Fri 19dec2025
ISC_LOCK NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:12 Fri 19dec2025
LASER_PWR NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
OMC_LOCK NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:12 Fri 19dec2025
SEI_CONF NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
SEI_ENV NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:12 Fri 19dec2025
SQZ_FC NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
SQZ_MANAGER NOT LOADED ISC_library.py 13:08 Tue 24feb2026 09:21 Tue 23dec2025
TMS_SERVO NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
VIOLIN_DAMPING NOT LOADED ISC_library.py 13:08 Tue 24feb2026 13:11 Fri 19dec2025
After the reboot, there were a handfull of nodes I requested to different states:
All other nodes I left either to their initial requests or INIT, which is how they come up by default.
TITLE: 03/03 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
SEI_ENV state: MAINTENANCE
Wind: 7mph Gusts, 4mph 3min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.15 μm/s
QUICK SUMMARY: HAM1 pumpdown had to be stopped last night and will be restarted this morning. Cable pulling, cleanroom moves, and a reboot of the Guardian machine are also planned for today.
Workstations were updated and rebooted. This was an OS packages update. Conda packages were not updated.
(Travis S., Gerardo M.)
Pumpdown was started this morning, sorry no blowdown data. By the time I remembered to take the reading there was no more pressure inside the chamber. Travis connected all of the equipment while I wrestled with the viewport inspection.
Pumpdown was started without an issue. We started with the scroll pump from the super sucker 500, the scroll pump of the SS500 cart brought the pressure down to 3.7x10-01 Torr, then we switched the cart to start pumping with the turbo pump/scroll pump. We had one trip of the system, usually due to a time out, but it only took a restart of the turbo pump and the pumpdown to continue.
Unfortunately the pumpdown had to be stopped at 8:00 PM local time, with a pressure of 4.0x10-05 Torr, at that pressure the set points had not activated on the SS500 cart controller, the set points are hard set and can't be modified from the SS500 controller screen.
Pumpdown will be restarted tomorrow morning.
Note, on the attachment, regarding the screens, no set points for channel 1.
Pumpdown Update.
Last Tuesday, HAM1 pumpdown was stopped for a few minutes to move the SS500 cart, the cart was moved away from the -Y door to open up space for the table. After the relocation of the SS500 cart the pumpdown for HAM1 was restarted.
On Thursday we assisted Filiberto with the installation of a "split" cabling for the high voltage interlock at HAM1 gauge. Attached is a photo of the screen of the SS500 cart and its response to loosing gauge signal, the signal cable was removed from the vacuum gauge to install the "split". I set valve control to "open" and the override to "override" on the SS500 cart to prevent it from tripping, it worked. BTW, when the cart reported the Error-S it cleared the setpoints. Once the cable was restored and we had a signal at the SS500 cart, the setpoints were restored manually.
Pumpdown continues, pressure is currently at 2.77x10-06 Torr, attached is a trend of the pumpdwon pressure.
We migrated the last system to the new VM cluster and powered down cdsproxmox0, cdsproxmox1, cdsproxmox2, cdsfs0 cdsfs1.
Cdsfs0, cdsfs1 have been moved into the DTS to become part of a cluster there along with the old commvault machine.
cdsproxmox2 was a spare w2275 and Dave will be using that to support some of the upcoming work.
I'll close the work permit tomorrow after we pull cdsproxmox0 & 1 from the racks.
TITLE: 03/03 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: None
SHIFT SUMMARY: HAM1's main volume is pumping down nicely as of this morning. Cable pulling and parts cleanup also took place out in the LVEA.
LOG:
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 15:38 | FAC | Randy | LVEA | N | Changing HAM1 cleanroom bolts | 16:02 |
| 15:39 | VAC | Richard | LVEA | N | Checking HAM1 annulus | 15:41 |
| 15:53 | FAC | Kim, Nellie | LVEA | N | Technical cleaning | 17:30 |
| 16:55 | EE | Jackie | LVEA | N | Pulling chassis from SUS-R7 | 17:02 |
| 16:55 | VAC | Travis | LVEA | N | HAM1 pumpdown | 19:30 |
| 17:22 | FAC | Nellie | EY | N | Technical cleaning | 18:07 |
| 17:37 | FAC | Kim | EX | N | Technical cleaning | 18:38 |
| 17:47 | ISC | Jennie, Masayuki | LVEA | N | Parts cleanup at HAM6 | 18:42 |
| 17:57 | VAC | Gerardo | LVEA | N | HAM1 pumpdown | 19:31 |
| 18:11 | ISC | Elenna, Oli | Optics Lab | N | BHD optics build (Elenna out @ 18:54) | 19:59 |
| 18:51 | SAF | Richard | LVEA | N | Checking on HAM1 | 19:10 |
| 20:35 | SPI | Jeff | Opt Lab | N | Taking pictures | 23:01 |
| 20:37 | FAC | Richard | LVEA | N | Rebooting PSL environment controller | 21:10 |
| 20:42 | FAC | Randy | LVEA | N | Plugging in scissor lifts | 21:15 |
| 20:46 | ISC | Elenna, Oli | Opt Lab | N | BHD optics build | 23:42 |
| 20:53 | AOS | Betsy | LVEA | N | Putting away parts | 21:18 |
| 21:24 | ISC | Jennie | LVEA | N | Parts cleanup at HAM6 | 21:43 |
| 21:29 | EE | Fil, Marc | LVEA | N | Pulling cables | 21:39 |
| 21:30 | ISC | Camilla | Opt Lab | N | Looking at parts | 21:57 |
| 21:51 | AOS | Mitchell | EX, EY | N | FAMIS checks | 22:32 |
| 21:57 | AOS | Betsy | Opt Lab | N | Checking parts | 22:57 |
| 21:57 | VAC | Gerardo, Travis | LVEA | N | HAM1 pumpdown | Ongoing |
| 22:39 | EE | Fil | EY | N | Grabbing power supplies | 23:38 |
| 22:44 | SAF | Richard | LVEA | N | Looking for Travis | 22:52 |
| 23:39 | EE | Marc, Fil | LVEA | N | Pulling cables | Ongoing |
| 23:51 | ISC | Betsy, Jennie | LVEA | N | Looking at parts at HAM6 | 00:05 |
| 00:06 | ISC | Elenna, Oli | Opt Lab | N | BHD optics build | Ongoing |
FAMIS 31127
Jason made a brief trip into the enclosure 4 days ago, which is seen in the usual places. Also, the differential pressure between the laser room and anteroom changed around that time. Nothing much else to report this week.
TITLE: 03/02 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
SEI_ENV state: MAINTENANCE
Wind: 3mph Gusts, 0mph 3min avg
Primary useism: 0.01 μm/s
Secondary useism: 0.14 μm/s
QUICK SUMMARY: The HAM1 annulus has been pumping over the weekend, and if that's reached a low enough pressure, then pumping can start on the HAM1 main volume today. A few decent-sized earthquakes rolled through overnight as well.
Sun Mar 01 10:09:25 2026 INFO: Fill completed in 9min 21secs
Sat Feb 28 10:10:32 2026 INFO: Fill completed in 10min 28secs
S. Muusse, G. Valente
Electronics integration
We are running into some issue with the linear translation stage and rotation stages which we can't operate through the medm. Both stages are operational when using a newport motor driver and when driven through directly through beckhoff into the CHETA chassis and motor driver chassis but havent been shown to work with the code taken from the CO2 rotation stages. Patrick has been looking through the software but we are yet to find the solution. Yesterday (02/26) we determined the motor power cables from the CHETA chassis to the motor driver chassis were crossed which was a possible source of this issue, however, this was consistent with the wiring in the motor driver chassis albeit unorthodox. We elected to modify the cables so they are wired conventionally and in turn changed the wiring in the motor driver chassis by switching the + and - wires at the back of the chassis where the power is outputted. Now internally the black = positive and red = Negative which is unfortunate but the best solution. We have seen both stages work after this effort using the newport motor driver so we dont believe this is the issue.
Setting up Table 2
Table 2 is in the process of being populated. Today we installed the PD system using the electronics to align the PDs and also have inserted the first telescope lens. Initially, when installing and aligning the PDs the PD power supply chassis was supplied with pos =+24 and gnd =0V instead of ±18V leading to both PDs outputing -11V. This fried the first transimpedence amp in the circuit board which we have now replaced with alot of help from Marc. Both PDs are now working and aligned on the table.
Calibration and power budget on table 1
We measured the power after all optics, where the powermeter fit ,as well as the I[A]/P[W] relationship of the laser to calibrate laser power on the MEDM.
The min/max power that can be achieved using the waveplate in rotation stage and the 2 polariser was measured:
Min ≈20mW ,
Max = 1.44W
Both powers were measured after the 2nd polariser. This corresponds to a maximum power which can be output off the table of 1.308W, which was measured after the last pickoff.
I'll attach plots/ figures in the comments showing these measurements.
As part of WP 13002 we migrated more services from the old VM cluster to the new one. The most visible one was migration of the dust monitor and weather station IOCs.
We waited to do this migration until the doors were back on to make sure the dust monitors did not disappear while they were needed.
We have one more VM to migrate, we will do that on Monday and then de-commission the old VM cluster and supporting file servers.
TITLE: 02/28 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: N/A
SHIFT SUMMARY: Both doors and all viewports are now in place on HAM1 and the annulus is pumping down. Next week, after the main volume is at low enough pressure, we'll start commissioning the new JAC in with the rest of the IFO.
LOG:
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 15:36 | FAC | Kim, Nellie | LVEA | N | Technical cleaning | 16:19 |
| 15:46 | FAC | Randy | LVEA | N | Prep for HAM1 door | 16:31 |
| 16:09 | VAC | Jordan | LVEA | N | Isolate turbopump on BSC8 | 16:17 |
| 16:49 | VAC | Travis | LVEA | N | HAM1 viewport and door prep | 17:34 |
| 16:50 | VAC | Jordan | LVEA | N | HAM1 viewport and door prep | 17:34 |
| 16:50 | FAC | Kim, Nellie | LVEA | N | Technical cleaning | 18:03 |
| 17:10 | SAF | Richard | LVEA | N | Checking in w/ VAC team | 17:32 |
| 17:48 | FAC | Randy | LVEA | N | Measuring cleanrooms in W-bay | 19:48 |
| 17:54 | SPI | Jeff | Opt Lab | Local | SPI fiber characterization | 20:11 |
| 17:55 | VAC | Jordan, Mitchell | LVEA | N | Prep for HAM1 cleanroom move (Mitchell out @ 18:11) | 19:27 |
| 17:57 | VAC | Betsy, Travis | LVEA | N | HAM1 door and cleanroom move (Betsy out @ 18:12) | 19:27 |
| 18:25 | TCS | Sophie, Gabriele | Prep Lab | Local | CHETA table work | 18:45 |
| 18:49 | FAC | Kim | LVEA | N | Technical cleaning | 19:35 |
| 18:51 | SAF | Richard | LVEA, CER | N | Safety checks in walls | 18:58 |
| 18:54 | TCS | Sophie | Prep Lab | Local | CHETA table work | 20:03 |
| 20:22 | SPI | Jeff, Jim | Opt Lab | Local | SPI fiber characterization (Jeff out @ 23:30) | 23:38 |
| 20:29 | FAC | Randy | LVEA | N | Craning scissor lift | 21:10 |
| 20:37 | VAC | Jordan | LVEA | N | Setting up HAM1 pump carts | 21:38 |
| 20:42 | VAC | Travis | LVEA | N | Setting up HAM1 pump carts | 21:38 |
| 22:15 | SQZ | Sheila | LVEA | N | Turning on SQZ laser | 22:45 |
| 22:45 | VAC | Travis | LVEA | N | Checking HAM1 pumps | 23:03 |
| 22:54 | TCS | Gabriele, Sophie | Prep Lab | Local | CHETA table work (Gabriele out @ 23:15) | Ongoing |
| 23:08 | JAC | Jennie, Keita | LVEA | N | Checking parts, turning off JAC heater (Keita out @ 23:40) | 00:08 |
| 00:23 | ISC | Elenna, Oli | Opt Lab | N | Assembling optics | Ongoing |
Richard, Marc, Jonathan, Erik, Dave:
On Monday 23rd the CHETA slow controls chassis was installed in the Vacuum Prep Lab (VPL). We connected its ECAT ethernet port to the MSR Beckhoff chassis AUX-1 port partially by copper (VPL to Rm163 into CUR) and partially by fibre (CUR CDS edge switch to MSR edge switch).
This appeared to work, but later we found out that the CDS core switch was complaining constantly of rapid movement of a MAC address on the ports used by CHETA. Further reading suggests ECAT does not strictly support unique MAC addresses per Beckhoff terminal, and it appears they may all have the same non-standard MAC address.
In addition, since Monday we have been seeing regular dropouts of the CUR network switch stacks, every 6 hours at 00:41, 06:41, 12:41 and 18:41 for about a minute apiece.
This afternoon we ran a long 148' cat5e ethernet cable from room 163 to the far end of the MSR. At 16:01 we routed the CHETA chassis connection to use this cable, we now have contiguous copper run between the two chassis.
This will fix the MAC movement issues on the CDS core, and time will tell if if fixes the CUR switch dropout.
Initial reports suggest the 6-hourly drop outs of sw-cur-aux-stk has been fixed by this move.
EtherCAT terminals do not have MAC addresses. Typically, the network adapter sends an Ethernet frame with its own Ether type that will flow through all terminals without being absorbed. Terminals will just read or write individual bits and bytes of this frame as it passes by, and eventually this frame will be reflected back to the adapter. This is called EtherCAT direct mode and has very good latency characteristics, but it is not comaptible with an IP based infrastructure.
Jordan and Camilla pulled the FirstContact from the JAC viewport in prep to install it on the -Y door. Unfortunately the inner glass surface was still showing many dots of particulate so we decided to re-FirstContact clean it. I poured on a new sheet and we are now letting it dry overnight. Install will need to happen tomorrow after we pull the sheet and hopefully have better results. This seems to be typical of FirstContact these days - needing to reapply a second time on numerous optics...
The second application of FirstContact was peeled from the vacuum-side of the viewport this morning. During the peel, the TopGun N2 ionizer output was at ~10 psi, with a follow-up TopGun pass at ~30 psi. I observed no macroscopic particulate within the ~2.5 inch diameter of the FC coverage after removal. At the edges of the FC sheet (the ~2.5 inch diameter centered on the viewport optic), I noted two "smudges" of FC film. I removed these smudges with a small Contec swab dipped in reagent grade acetone. Another ~1 mm droplet of FC was noted at the optic/mounting ring interface. I also attempted to remove this with a swab/acetone wipe, but since it was in an interior corner, some FC remained where it had wicked into the interface.
The viewport was then installed on the HAM1 -Y door A2F3 flange. Torquing of the CF flange and installation of the guillotine assembly went without a hitch. We didn't do any cleaning to the in-air side of the viewport post-installation.
Randy, Ibrahim
Randy and I put the BBSS in the can today, as shown by pictures.
Following LLO's advice from their experiences in alog 79950, we were able to use the aluma lift to can the BBSS. What we did in order:
1. Strip locked-mass BBSS of all cables, check for looseness, bring alum-a-lift into position to lift BBSS from lifitng bars
2. Undo dog clamps to let lift take suspension load, ensure suspension is secured on the lift. Move BBSS out of test stand - as LLO said, very small clearance.
3. Prepare temporary platform for lift fork adjustment
4. Set BBSS on temporary platform - this part was successful but needed extra blocks to avoid interference with Y-bracks on the Lower Structure.
5. Adjust forks to be allow for canning without bottoming out, relift.
6. Put into wiped-down can and check for looseness. Rewipe, check for particulate and ensure secure fit
7. Close doors.
See pictures below.
EPO taggin'.
Ibrahim, Randy
Randy and I opened the can to:
See pictures below for each and another overall image.
We then buttoned the BBSS and tightened all screws. It is now ready for transport.