J. Oberling, F. Clara
Summary: We think the 9.5Hz peak is coming from the new flow meters installed as part of the 2021 PSL laser upgrade.
Today we continued hunting for the 9.5Hz comb triplet that Ansel had tracked to the PSL back in May. Robert subsequently used a temporary magnetometer and found a strong 9.5Hz feature in the neoLASE LVEA Control Box. To try to isolate the source of this 9.5Hz peak we shut down the PSL in a controlled manner and watched the peak on the magnetometer. The order of the shutdown:
There was no change in the peak observed during the PSL shutdown, so we next turned off the neoLASE Control Box; the peak clearly disappeared with the Control Box off. Our next step was to check if the peak was being fed to the Control Box via something plugged into it. We turned the Control Box back on and the peak returned (as expected), then we unplugged the cable that feeds the flow/temperature sensor signals to the Control Box (via the PSL Sensor Interface Chassis (SIC), which feeds the Control Box with several monitor signals (PDs, power meters, flow meters) and the system shutters). Lo and behold, the peak was much smaller, sometimes disappearing entirely (see attached pictures; first is with the PSL off but the flow meters plugged in, second is with the flow meters unplugged). We next took the top off of the SIC and placed the magnetometer near where the flow meter signals both come into and exit the SIC, and did not see the peak. Fil then placed the magnetometer on the flow sensor connectors themselves (the flow sensor cables route to a pig tail on the back of the SIC, which goes from an individual M12 connector for each flow meter into a DB25 connector to plug into the SIC) and the 9.5Hz peak was clearly visible again. If we unplugged the main flow meter cable from the back of the Control Box the peak again disappeard; this makes sense as the Control Box supplies the 24 VDC the flow meters need, and unplugging the cable removes power from the flow meters. We then put the top back on the SIC and placed the magnetometer back in its spot directly underneath the Control Box and plugged the flow meters back in; the peak again returned. Fil disconnected the individual M12 connectors for the flow meters and the peak again disappeared. We then plugged each individual flow meter in to observe how the peak behaved (I did not take pictures of this, unfortunately). With 1 flow meter plugged in the peak was visible but much smaller; with 2 plugged in the peak was even smaller (???); with three plugged in the peak was back to its usual magnitude. With this odd behavior we suspect a grounding issue somewhere. To finish things off we unplugged the other items feeding signals into the Control Box (shutters, PDs/power meters, NPRO, Diode Box A, Diode Box B), one at a time, to observe how/if the peak changed; we observed no change in the peak during this process, the only thing that changed the peak with the Control Box turned on was unplugging the flow meter cable. We think this fairly definitively tags the flow meters as the source for the 9.5Hz peak seen in Robert's magnetometer, and likely the source of the comb triplet seen around this frequency.
We are now looking through documentation to see how we can mitigate this. One option is to tie the grounds of all 3 flow meters together and see if this changes the peak; this requires running long cables from each flow meter to tie them all at the same spot. We also plan on using the spare Control Box to check into how the internal grounding of the Beckhoff terminal is done and see if there's any improvements we can make there. Another option is to isolate power to the flow meters; currently they are powered by the Control Box, but we could power them separtely and see if that changes the peak. Investigation continues, more to come.
The PSL is now back ON, and the PMC and FSS RefCav are locked; I'm letting the system warm up for a bit before locking the ISS. This closes LHO WP 12031.
Edit to add: When unplugging the flow meters individually via their M12 connectors, we did try each of the 3 flow meters by themselves; the peak was the same regardless of which individual flow meter was plugged in. Things got weird when we had 2 plugged in at the same time (the peak was smaller than with any of the individual flow meters plugged in, hence why we suspect a grounding issue somewhere), and the peak returned to its starting magnitude when all 3 were plugged in at the same time.
The bake for the RGA on the end door for HAM6 has started, turned on at 16:00 utc.
RGA system at 150 oC, cal leaks are open, time start 00:01:00 utc 08/14/2024.
This morning around 15:45 utc, after checking the health of the ion pump on HAM7 (IP19), its isolation valve was opened. No issues doing so, pressure in HAM7 is going down, see attached trend plot.
During the Tuesday maintenance reboots, zotvac0 did not shutdown properly, meaning the regular 08:00 did not run. Erik got the machine back and I rescheduled today's fill for 08:20
Tue Aug 13 08:29:17 2024 INFO: Fill completed in 9min 13secs
Gerardo confirmed a good fill curbside.
7min 20sec into the fill (1min 48sec before fill completion) TC-B failed. Its temperature reading went from -52C to +1372C
The fill completed using TC-A. In the attachment, the TC-B line disappears due to it being an inverted log plot.
TC-B was replaced following this failure. One of the leads was broken and the other had very little insulation remaining, so it was swapped with a new one. I wrapped TC-A together with TC-B with electrical tape in the most damage-prone area at the bend entering the exhaust line.
I also closed FRS 30032 which requested that this TC be swapped.
TITLE: 08/13 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Preventative Maintenance
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
SEI_ENV state: MAINTENANCE
Wind: 14mph Gusts, 9mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.05 μm/s
QUICK SUMMARY: We continue to pump down. Tuesday maintenance day activities planned for today.
This is a sister alog to LLO:72613. After discussion with the calibration group, Joe and I have proposed and pushed a small but important fix to the CAL_LINE_MONITOR_MASTER/COHERENCE block. When the IFOs are down, the coherence for various calibration and monitoring line transfer functions is 0 (this is good). However, the TF uncertainty, which is also produced by the coherence block, was found to be railing at 0 too. How can the uncertainty be 0 if there is no coherence?! It turns out that a coherence of 0 was causing a divide by 0 that, on the front end, presents as ... you guessed it, 0. The new h1calcs models (updated at LLO then synchronized by me to LHO) fix that. Now the coherence signal entering the uncertainty-calculation logic will be kept within 1e-6 to 2. We allowed for a coh value above 2 in order to avoid inadvertently 'hiding' egregious errors in the coherence calculation by forcing it to stay below 1. Thinking about this further now that I'm writing this...that wasn't necessary as the coherence channel itself isn't affected by our new changes. oh well. I've attached an annotated screenshot of Joe's screenshot in LLO:72613 to show the new component we've installed. The h1calcs updates should be put in place during tomorrow's maintenance period.
Today's activities: - After reaching ~1.6E-6 Torr in the Corner (according to BSC3's gauge), GV1 was opened. This immediately brought down the corner pressure to ~9E-7 Torr - A few hours after this, GV2 was also opened. This quickly brought down the pressure further to ~6.5E-7 Torr - Opening up of GV1 and GV2 was 2 steps of a strategy, which used the Y and X manifolds as "vacuum accumulators" (the turbo pumps were switched on these volumes for the entire vent, so the pressure in these volumes was very low - ~2-3E-8 Torr). This came with immediate pressure drops, and now there is 6500 l/s pumping speed in the corner instead of the 2500 l/s. This means now a ~15 m^3/h real throughput instead of ~1.5 m^3/h before these actions - Because of these openings, the pressure in CP2 started to rise slowly, because GV7 was only soft-closed. So, after reaching 3.4E-9 Torr, more pressure was applied to GV7 - it was noticed, that most likely because of the inst. air actions, the pressure there was very low, <5 psi. Now it was set to ~12 psi, and so CP2 pressure stabilized The pressures currently: - Corner (at BSC3): 6.1E-7 Torr - HAM6: 7.4E-7 Torr - HAM7 + Relay tube: 2.64E-7 Torr
TITLE: 08/12 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
SHIFT SUMMARY:
H1 is in a vacuum pump down state since last week. We are still pumping down.
Current corner station vacuum pressure occording to VAC channel H0:VAC-LY_Y1_PT120B_PRESS_TORR : 6.25e-07
VAC team decided to open the 2 of the 4 Gate valves today. This allows them to pump down with more vacuum pumps thus increaseing the rate at which the vacuum chamber is evacuated.
19:16 UTC Vacuum pressure high at BSC8, BSC2 and HAM6 This is because Gate Valve 1 was opened by Janos and Gerardo.
21:55 UTC Vacuum team opened Gate valve 2, this opened BSC7 to the rest of the CS.
GV7 was soft closed, I was monitoring H0:VAC-LX_X3_PT134B_PRESS_TORR and saw an increase in pressure.
23:23 UTC Janos put more pressure from the intrument air onto GV7 which stopped the increase in pressure at H0:VAC-LX_X3_PT134B_PRESS_TORR.
LOG:
Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
---|---|---|---|---|---|---|
14:35 | FAC | Karen | Optics & Vac Lab | N | Technical cleaning | 14:53 |
15:40 | FAC | Karen & Kim | LVEA | N | Technical Cleaning, Kim out, Karen still in. | 18:08 |
19:05 | VAC | Gerado & Janos | LVEA | N | Opening GV1 Between BCS8 and BSC1 | 19:42 |
21:30 | VAC | Gerardo & Janos | LVEA | N | Opening Gate valve 2 | 22:30 |
21:52 | FAC | Tyler | LVEA | N | Talking to Gerardo | 22:22 |
22:42 | VAC | Gerardo Fil | HAM Shaq | No | Running cables | 23:36 |
Closes FAMIS#26469, last checked 79342
HEPI pump trends looking as expected (attachment)
Mon Aug 12 08:08:15 2024 INFO: Fill completed in 8min 12secs
TITLE: 08/12 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
CURRENT ENVIRONMENT:
SEI_ENV state: MAINTENANCE
Wind: 12mph Gusts, 9mph 5min avg
Primary useism: 0.01 μm/s
Secondary useism: 0.06 μm/s
QUICK SUMMARY:
Pump down has continued throuout the weekend undisturbed.
Pump down is expected to continue for the rest of the week.
Current pressure as read by the channel H0:VAC-LY_Y1_PT120B_PRESS_TORR = 1.67e-06
Sun Aug 11 08:08:59 2024 INFO: Fill completed in 8min 55secs
Sat Aug 10 08:08:48 2024 INFO: Fill completed in 8min 44secs
TITLE: 08/09 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Oli
SHIFT SUMMARY:
2 tours came through the control, but other than that we have just been watching the Vacuum system slowly evacuate the vacuum chamber today.
LOG:
Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
---|---|---|---|---|---|---|
18:03 | VAC | Janos | LVEA | No | Checking pumps | 20:03 |
18:20 | Tour | Mike + 12 | Overpass | N | Giving a tour | 19:20 |
22:44 | VAc | Janos | LVEA | N | looking for rounds | 23:09 |
TITLE: 08/09 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
SEI_ENV state: MAINTENANCE
Wind: 13mph Gusts, 10mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.06 μm/s
QUICK SUMMARY:
H1 still pumping down.
There aren't any alerts about Vacuum Pressure High at BSC2 or HAM6
Vac channel : H0:VAC-LY_Y1_PT120B_PRESS_TORR currently reads 3.83e-6 Torr and falling.
Fri Aug 09 08:06:56 2024 INFO: Fill completed in 6min 53secs
Today's activities: - In order to advance further safely, the instrument air system needed to be fixed, at least temporarily, but reliably. Currently the Kobelco compressor is being used as an instrument air compressor, for details, see aLog: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=79512 - The HAM6 turbo was started up with its designated backing cart. This cart needs instrument air, as one if its safety valves is pneumatic. This turbo was running for 2-3 hours before it was valved in, to eliminate any potential risks with the inst. air - The HAM6 turbo was valved in at ~12:50. The pressure immediately went down in HAM6 by 2E-6 Torr, and in the corner in general by 5E-7 Torr - The HAM5/HAM6 annulus system pumpdown continued. The Annulus Ion Pumps have been also switched on, and after some struggle they came alive. The pumping carts are still there though, they are at <1E-5 Torr - As HAM7 caught up with the relay tube, the pumping of the relay tube was stopped, and it was valved together with HAM7 (RV-2 open) - The rebuilt HAM6 RGA is being pumped, before it will be baked out, starting with next Monday. A pumping cart has been brought from the FCES for this purpose The pressures currently: - Corner (at BSC3): 4.3E-6 Torr - HAM6: 5.8E-6 Torr - HAM7: 5.9E-7 Torr - Relay tube: 6.5E-7
TITLE: 08/08 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: n/a
SHIFT SUMMARY:
Corner station volume continues with pump down. LVEA cleaning parting in the afternoon.
LOG:
J. Kissel Took the opportunity to grab post-OFI-repair vent, in-vac transfer functions of the following HAM5 and HAM6 SUS. Of course, we're not at full observing-level vacuum, but most of the molecules are out of the chamber that would contribute to buoyancy, so the SUS are at the position they will be in observing-level vacuum. Executive summary: I'm happy with all SUS that I measured -- including the two that were "interesting" and or touched during this vent -- SRM and OFI. For the story about SRM, see LHO:79433. And of course, the OFI had had optics replaced. Several other SUS in HAM5 / 6 were not touched during the vent (aside from a few alignment sliders), so we didn't expect any degradation in performance and don't see any. These SUS are ready to resume observing gravitational waves! Processed results will be posted as comments below. Some relevant details of the status of the suspensions and their surroundings during the measurements. Environment Status (Same for HAM5 and HAM6) ISI DAMPED HPI OFFLINE and mechanically locked VAC Pressure at 1e-5 Torr Status of alignment offsets and coil driver switchable filter requested (CD) State. OPTIC ON/OFF Value (if on, [urad]) CD State SRM ON P/Y : +1663 / -3095 1, LP OFF ZM6 OFF (n/a) Fixed in 2, LP ON OFI OFF (n/a) Fixed in 2, LP ON OMC ON P/Y : +340.5 / 0.0 1, LP OFF OM2 ON P/Y : +716 / -1701 Fixed in 2, LP ON OM3 ON P/Y : -1147 / +20 Fixed in 2, LP ON File Names: SRM HSTS/H1/SRM/SAGM1/Data/2024-08-08_1823_H1SUSSRM_M1_WhiteNoise*.xml ZM6 HXDS/H1/ZM6/SAGM1/Data/2024-08-08_1828_H1SUSZM6_M1_WhiteNoise*.xml OFI OFIS/H1/OFI/SAGM1/Data/2024-08-08_1850_H1SUSOFI_M1_WhiteNoise*.xml OMC OMCS/H1/OMC/SAGM1/Data/2024-08-08_1900_H1SUSOMC_M1_WhiteNoise*.xml OM2 HXDS/H1/OM2/SAGM1/Data/2024-08-08_1920_H1SUSOM2_M1_WhiteNoise*.xml OM3 HTTS/H1/OM3/SAGM1/Data/2024-08-08_1935_H1SUSOM3_M1_WhiteNoise*.xml
Found at $(sustrunk)/HSTS/Common/Data/allhstss_2024_08-08_H1SUSSRM_M1_PostOFIVentAirvVac_ALL*_TFs.pdf
Found at $(sustrunk)/HXDS/Common/Data/allhxdss_2024-08-08_H1SUSZM6_InAirvVac_ALL*_TFs.pdf
Found at $(sustrunk)/OFIS/Common/Data/allofiss_2024-08-08_H1SUSOFI_PreVentvPostInAirvInVac_ALL*_TFs.pdf
Found at $(sustrunk)/OMCS/Common/Data/allomcss_2024-08-08_H1SUSOMC_PreVentvPost_ALL*_TFs.pdf
Found at $(sustrunk)/HXDS/Common/Data/allhxdss_2024-08-08_H1SUSOM2_Vac_ALL*_TFs.pdf
Found at $(sustrunk)/HTTS/Common/Data/allhttss_2024-08-08_OM3_H1SUSHTTS_PostVentInAirvVac_ALL*_TFs.pdf