Attached is a snapshot of the Trello from this last week and into next showing what has been checked off. Major milestones met of ISI in correct location (IAS survey complete), 90% cables installed and mostly tested, nearly all IO/ISC components landed on tables with the exception of diodes which are WIP. A fair amount of troubleshooting, hunting and fixing of ground loop and cable issues in and out of vac. Next week we get serious about letting the PSL beam in and aligning the paths.
Jonathan, Dave:
Jonathan has his new h1daqfw2 frame writer running and writing byte identical frame files compared to production. I have extended the DAQ DETAIL MEDM to perform a check between all three frame writers, warning if their frames differ in size or content.
The attached MEDM shows all three frame writers with identical full and second trend frame files, but with FW2 having written a different minute trend file. This is because FW2 was started during the past hour and therefore wrote an incomplete file.
FW2 is currently not in production, which is shown by its error/warnings being indicated in YELLOW instead of RED.
After another hour and a full minute trend frame being written there are differences between the h1daqfw[01] and h1daqfw2 frames minute trend frames. Looking at H-H1_M-1429650000-3600.gwf there are 767 channels with differences. All of the differences are .mean channels. I randomly sampled a small number of these channels. The difference that I saw in all the channels I looked at was that h1daqfw[01] had set the mean value to -0.0, and h1daqfw2 had computed the mean to be 0.0 (no minus sign). Since having the sign bit set means the underlying bytes are not all zeros the compression isn't as nice and the h1daqfw[01] frame is slightly larger. Seeing differences like this is not unexpected. This is a different implementation and different implementation technology so floating point rounding errors are an issue. My goal is still to be able to get this to be byte for byte identical output. Here is a dump of a minute trend channel that shows the difference (using framecpp_dump_channel). from h1daqfw0: DEBUG: Processing channel: H1:SUS-MC2_M3_MASTER_OUT_LL_DQ.mean DEBUG: Trying as ADC DEBUG: Is a ADC Name: Compress: 257 (Endianness: Little Scheme: 1) Type: 2 NData: 60 NBytes: 22 Data: -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0 from h1daqfw2: DEBUG: Processing channel: H1:SUS-MC2_M3_MASTER_OUT_LL_DQ.mean DEBUG: Trying as ADC DEBUG: Is a ADC Name: Compress: 266 (Endianness: Little Scheme: 10) Type: 2 NData: 60 NBytes: 8 Data: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
I keep forgetting to reset the dust monitor alarm levels after IOC restarts so I wrote a little script to check all of the dust monitor alarms levels and reset them if needed based on E1600132. It does not reset the LVEA ones, it just notifies, as these ones alarm levels are sometimes modified.
The scripts is called check_dm_alarms_lvls.py and lives at /ligo/cds/userscripts/. I've also updated the wiki with this info.
I have added a call to this script to the aliased "check_dust_monitors_are_working" bash script that operators usually run at the beginning of the shift.
Oli, Daniel, Fil, Betsy, Rahul
We have re-installed SUS RM1 and RM2 Tip Tilt suspensions into HAM1 chamber which has the new ISI table - see several pictures attached below. RM1 is on the left (towards PSL side) hand side of this picture and RM2 is on the right (zoomed in). Both the suspensions are positioned roughly as per D1000313_V19, and will be fine tuned (pitch and yaw of the mirror as well) during beam alignment work. The blade springs of both the suspensions have been un-muted - see LHO alog 83984 for details.
Both RM1 and RM2 have been plugged-in to their electronics chain. We had to perform some troubleshooting with RM1 after observing grounding issues. At first Fil and Daniel ruled out any issues with the satellite box and in-air cables. After some investigation, the grounding issue was suspected to be coming from the in-vacuum quadrupus cable (connecting to the Bosem), which is an old cable and looks worn out too.
This morning I replaced the old Bosem cable (D1000228_v3 s/n S1105240) with a 25in long (D2100049 S/N 2200358) cable. Before connecting it to the Bosem, Daniel and I verified that the cable is fine (i.e not grounding) and later on we connected each pin (Bosem) one by one. We had to re-adjust the Kapton tape (installed for additional shielding) since the cable bosem connector was grounding to the chamber. After all these modifications, we can confirm that RM1 and RM2 are free from any grounding issues for now. We will check again before closing the chamber.
Next, I will start taking measurements on these suspensions and will post their health report soon.
Things to do on RM1 (hardware side),
1. Find and replace the new cable (25in long D2100049 S/N 2200358) ) which has just been installed with another one since this one is missing pin no. 1.
2. replace the old style cable table bracket which is currently in HAM1 for RM1 with a new one for the new style of cable. The old style cable brackets don't fit.
Also, adding a picture of the chamber with labels showing RM1 and RM2 suspension.
Electronics testing of both the sus is still ongoing.
Adding a reminder here that the sign of both RM1 and RM2 OSEM readbacks have changed, after the discovery that "more light" = "more negative signal" in LHO:84027.
Tagging EPO with the cool new digs inside HAM1 and on the new ISI.
We have been running the dust monitor vacuum pumps with the assumption that they had to be set to 19 inHg according to T2100415, I went out to ENDY to test this out. When I first checked out the vacuum pump, it was at 20.5 inHg, the flow meter on the dust monitor was reading at 2.8 L/min which is what we want, I then lowered the pumps pressure down to 13 and rechecked the dust monitor and its flow was now at ~2.7. I then adjust the pump down to 10 which brought the dust monitors flow to 2.65. These 1/10s of a L flow are correctable by adjusting the internal flow rate on the dust monitor. Before I left I reset the pumps pressure to 19 inHg.
It seems like we do not need to have the vacuum dust monitor pumps set to 19 inHG, the dust monitors flow rate isn't as strongly dependent on it as we thought?
Closes FAMIS 26397, last checked in alog 84018
Laser Status:
NPRO output power is 1.841W
AMP1 output power is 70.43W
AMP2 output power is 140.4W
NPRO watchdog is GREEN
AMP1 watchdog is GREEN
AMP2 watchdog is GREEN
PDWD watchdog is GREEN
PMC:
It has been locked 2 days, 23 hr 11 minutes
Reflected power = 23.28W
Transmitted power = 105.5W
PowerSum = 128.8W
FSS:
It has been locked for 1 days 20 hr and 35 min
TPD[V] = 0.8003V
ISS:
The diffracted power is around 3.7%
Last saturation event was 1 days 20 hours and 35 minutes ago
Possible Issues:
PMC reflected power is high
Closes FAMIS 26040. Last checked in alog 83968.
HAM4, HAM7, HAM8, EX ST1 all have elevated peaks, mostly around 1Hz line. This is expected due to high maintenance and vent activity.
Closes FAMIS 26375. Last checked in alog 84000,
Everything is under threshold. There are different fans on than last week so difficult to compare. MR FAN3 in CS has a slightly higher noise character in last 3 days
While prepping the HAM1 ISI for testing last night, I noticed that one of the HAM2 HEPI L4Cs seemed non-responsive. It seems this has been the case for quite some time, 206 days according to the attached ndscope. Something happened on Oct 1st last year that cause this L4C to suddenly have only ~1/50th the signal the H2 L4C on the HEPI to sees. I have poked at it a bit, checked cabling on the pier and gently hit the foot to see if I could revive the sensor. I have a couple other tests I want to do before deciding to remove the actuator to replace the sensor, not exactly eager to have to do that.
Dry air skid checks, water pump, kobelco, drying towers all nominal. Dew point measured at HAM1, reading -43.8°C
. Main turbo pumps, XBM, YBM and OMT, all nominal, and temperature is good and stable. HAM6 turbo pump and cart are nominal.
WP12488 HAM8 ISI binary readback broken following site outage FRS33798
Fil, Erik, Jim, Dave:
Yesterday we power cycled the binary input/output and interface chassis, which did not fix the issue. Today we stopped all the models on h1cdsh8, powered down the computer, then power cycled the IO Chassis.
This also did not fix the problem. Investigation continues.
Fri25Apr2025
LOC TIME HOSTNAME MODEL/REBOOT
11:15:32 h1cdsh8 ***REBOOT***
11:17:25 h1cdsh8 h1iopcdsh8
11:17:38 h1cdsh8 h1isiham8
11:17:51 h1cdsh8 h1susfc2
11:18:04 h1cdsh8 h1sqzfces
11:18:17 h1cdsh8 h1susauxh8
11:18:30 h1cdsh8 h1pemh8
Some changes to the RM1/RM2/PM1:
All other things equal the local damping loops should have the correct sign now. Thic change will also require a sign change in the ASC centering servos.
"Yes, and..." - It makes sense to zero the ALIGNMENT offsets for RM1, RM2, and PM1. They're physically aligning the optics in chamber this week, so we don't want to get confused by having digital offsets driven out to the coils. - Daniel says "Changed the signs of the coil outputs for RM1 and RM2. This should account for the sign change of the OSEM voltages." Here, "the change in sign of the OSEM voltages" means the fixing of the "has been like that forever as far as we can tell" issue that the RM1 and RM2 OSEM PD sensor readback's ADC voltage had been negative when under light. The fix was in the in-air DB25 feedthru to satamp cable; see LHO:84027 and subsequent comment about the differences in satellite amplifiers. He acknowledges that this fix will disrupt the overall sign of the damping loops, so he just *picked* a place to flip the sign. He chose the COILOUTF banks, but . we use these banks to explicitly compensate for the magnet polarity, and . RM1 and RM2 have already confirmed to be different in this regard in Mar 2018; see LHO:40853 -- and in that aLOG we concluded it was RM2 that was abnormal because RM2 required a different damping loop gain than RM1, OM1, OM2, and OM3 when all *settings* were otherwise the same. . HOWEVER given that the OSEM sensor readback itself had a negative in it, I'm no longer convinced it's RM2 that's the problem. But at least we know they're different. . I tried to have Betsy/Rahul go in chamber to confirm magnet polarity but these HTTS magnets are particularly hard to get at / measure, given that they're buried within the flag and the PEEK flag can no longer be unscrewed from the aluminum optic holder; see LHO:84178 So, since we're left stuck not understanding the coil actuator magnet polarity, I *don't* want to fix the *sensor* sign here. We've reverted the above change of COILOUTF signs. They're now restored to $ caget H1:SUS-RM1_M1_COILOUTF_UL_GAIN H1:SUS-RM1_M1_COILOUTF_LL_GAIN H1:SUS-RM1_M1_COILOUTF_UR_GAIN H1:SUS-RM1_M1_COILOUTF_LR_GAIN H1:SUS-RM1_M1_COILOUTF_UL_GAIN 1 H1:SUS-RM1_M1_COILOUTF_LL_GAIN -1 H1:SUS-RM1_M1_COILOUTF_UR_GAIN -1 H1:SUS-RM1_M1_COILOUTF_LR_GAIN 1 $ caget H1:SUS-RM2_M1_COILOUTF_UL_GAIN H1:SUS-RM2_M1_COILOUTF_LL_GAIN H1:SUS-RM2_M1_COILOUTF_UR_GAIN H1:SUS-RM2_M1_COILOUTF_LR_GAIN H1:SUS-RM2_M1_COILOUTF_UL_GAIN -1 H1:SUS-RM2_M1_COILOUTF_LL_GAIN 1 H1:SUS-RM2_M1_COILOUTF_UR_GAIN 1 H1:SUS-RM2_M1_COILOUTF_LR_GAIN -1 - He's confirming our work on PM1 from LHO:83293 - 2025-04-29, Daniel has made the model changes in h1sushtts, but they've not yet been installed as this is a trivial top-level model change that has been there forever, and is blocked from reaching the DAC by just turning of the SUS-RM*_ISCINF_L bank inputs. - We'll re-address the damping loop signs once the dust settles with chamber work.
FAMIS27814
Our cadence is a bit off from biweekly, but no matter since no water was added and the chillers look good (updated the T2200289 sheet). There is a small amount of blueish gunk starting to form in the wire mesh diffuser in the TCSX chiller. I didn't see any of these in our parts bin so I'll put a few on order.
Fri Apr 25 10:08:57 2025 INFO: Fill completed in 8min 53secs
Gerardo confirmed a good fill curbside.
FranciscoL, SheilaD, JoeB
In summary:
I regenerated the mcmc fit on report 20250327T160138Z at 5, 10 and 20 Hz, to compare how the mcmc model would change given different frequency ranges for the fit. We see that the residual of the measurement to the model is better as we increase the minimum frequency threshold. We think this is due to external effects, aside SRC, confusing the model.
We want to give more validity to our observations on alog 83592 -- that the sensing model at low frequencies is a good descriptor of the sensing function. On 83592, I claimed that the observed changes in the sensing function come from changes in SRCL detuning. This is backed up by a low residual on the measured/modeled sensing function. In today's alog, we evaluate how well the model describes the sensing function by (1) having a *low* residual and (2) decreasing the optical spring frequency.
The first three figures attached ('sensing_mcmc_compare_*Hz.png') are the sensing function plots generated by pydarm for measurement 20250327T160138Z. This measurement is not a representation of a calibrated interferometer, since we tuned the SRCL detuning (83585).
From the transfer function on the left, the green dots represent the measurement, the orange trace is the mcmc model from the measurement, and the blue trace is the mcmc model from the last validated calibration measurement (20250222T193656Z). The residual from the measurement to both models are plotted on the right, matching the colors of the respective models. We are interested in the change of the orange trace when we modify the frequency range of the fit.
Each figure has a pair of vertical lines denoting the limits on the frequency range used for the fit. I regenerated the measurement report for 05, 10, and 20 Hz for the minimum frequency. The maximum frequency at 1200 Hz was not changed.
For the residual plots, we see that the magnitude is largest when fitting to 5 Hz, and that the change between the 10 and the 20 Hz fits is negligible. For illustration purposes, I plotted the different sensing models, along with the measurement, together in the last figure (20250327T160205Z_compare_models). The phase plot is not the same as the plots given by pydarm due to missing factors. We see that the three models match above 30 Hz in magnitude.
For the optical spring frequency, we see a decrease from 2.19 Hz to 0.43 Hz when fitting to 10 Hz, and 20 Hz, respectively. This decrease on the SRC spring suggests that the model does not fit the sensing function well, below 20 Hz.
So, regenerating the sensing model shows that the model is good when we see anything above 20 Hz. But we see from the measurement that the SRCL detuning does change the sensing function at low frequencies. The model, however, cannot fit the data from the measurements at these frequencies. We have seen before (80267) that angle to length cross coupling influences the sensing function in the low frequencies. A scenario of cross coupling affecting the sensing function could explain why the model cannot fit the measurement.
Repeating the analysis on cross-coupling, as done in 80267, could improve the sensing function.
TIL; pydarm edition:
The approach that enabled me to regenerate pydarm reports, albeit messy, was to
Modify the pydarm_cmd_H1.yaml
that is in the report to be regenerated, in this case 20250327T160138Z. In particular, the mcmc_fmin
value.
Once the YAML file is configured, run
This command will re-run the calibration report, and, by using the configuration that lives in that directory, make a new model to the sensing function measurement.
Jordan, Janos, 04-24-2025, ~15:00 The wire gaskets for VBO-C and VBO-D have been leaking continuously for a very long time. Lots of vendors have been contacted, and after a lot of unsuccessful tests, it seems that we finally found the good one manufactured by Torr Scientific (UK). This gasket is extremely annealed, butter-soft, needs super careful handling, but even after some suboptimal installation steps (mea culpa), it finally sealed a new chamber successfully. We tested it right after receiving (had a thriller with customs) with a new C+B chamber, pumped it down and tested on the same day. After the test, we let the pump running during the weekend just to see the ultimate pressure it can reach. They are still very pricey though, 193 GBP (= 257 USD)/pc., even in the case of purchasing 100 pcs., it's 147 GBP (= 196 USD)/pc. So, certainly not an optimal, long-term solution, but a good bridge between having nothing and manufacturing our own.
Good news! Let's take some detailed measurements (as possible), also how close they fit over the flange lip, etc. Will help us when we start fabrication in house.
Oli, Matt, Camilla
Late entry, Jim and I worked the last few half days to lace up all of the cabling for SUS, SEI, and IO/ISC in prep for the main reinstall of table components. Fil, some Richard, and I did preliminary ground loop checks on all SUS and IO/ISC cables so we have a handle on the situation before complicating with components. I routed cables on the table top in an attempt to be near where things plug in but also not in the way of landing objects. Further dressing and coiling will be needed after troubleshooting plugins with components as we go. Pics of the pre round of cabling.
Oli, Camilla
This morning we swapped the direction of HW1 so that it is facing the same as before. We also placed L2, M15, LSC POP A and ASC POP X although the diodes are not cabled up yet. They can be next week as Daniel confirmed chassis are powered off.
ASC POP X is new, the box is marked D1102004-v6 S/N 016 S1300637, ICS link. Have added this to the D1000313 BOM googledoc and HAM1 ICS assembly.
Tagging with EPO to document the process.
S. Koehlenbeck, J. Freed, J. Kissel, J. Oberling, R. Short
The SPI pick-off path installation on the H1 PSL table is now complete. The beam in the new SPI path has been reduced to 20mW and is currently being dumped with a razor dump between SPI-L1 and SPI-L2. Pictures attached reflect the final installation and layout, which will be be reflected in the updated as-built layout at a later date.
Associated entries: 83925, 83933, 83956, 83961, 83978, 83983 (and more to come)
ECR E2400083 IIET 30642 WP 12453 Here's Ryan's birdseye view labeled with all the components. For details of the components, see the SPI BOM, T2300363, exported from its google sheets to -v4 as of this entry.
Tagging EPO for photos.
83996 Power In ALS / SQZ / SPI Paths Post SPI Pick-off Install
Richard powered up h1seih23 from the front panel, it is now running.
We power cycled h1psl0, it is back.
Erik reconfigured EX Dolphin front ends to communicate with port 8 of the EY Dolphin switch to get them to get the models going. CDSRFM at EX points to port 7 of EY's switch.
All models are running, only IPC errors are long-range-dolphin.
Should someone from operations start activating the SEI and SUS systems?
After the power outage, I went out to the site ~10pm Pacific to check status of purge air skid. Kobelco, water pump and drying towers were all running upon arrival. I verified the dew point was still ok (-41.8C measured at HAM1) and verified the drying towers switched normally. Note: Purge air was only connected to HAM1 since doors are off, corner volume is vented but not actively purging with no doors off.
While there I also restarted the controller for 3 of the FCT ion pumps and reset solenoid valve on aux cart connected to GV5 annulus. All looks ok. Left site ~10:40pm.
h1guardian1 did not reboot, here is the guardian MEDM
Systems currently down as of 20:49:
FMCS/BACNET
Corner Station HEPI Pump Controller
PSL Diode Room slow controls
CP1 overfill
NCAL
BRS
HWS
DTS test stand
To continue running vac alarms overnight I have bypassed the currently active alarms for the next 24 hours.
Bypass will expire:
Mon Apr 7 09:28:54 PM PDT 2025
For channel(s):
H0:FMC-CS_CY_H2O_PUMPSTAT
H0:FMC-CS_CY_H2O_SUP_DEGF
H0:FMC-CS_FIRE_PUMP_1
H0:FMC-CS_FIRE_PUMP_2
H0:FMC-EX_CY_H2O_PUMPSTAT
H0:FMC-EX_CY_H2O_SUP_DEGF
H0:FMC-EY_CY_H2O_PUMPSTAT
H0:FMC-EY_CY_H2O_SUP_DEGF
H1:CDS-DTS_ENV_TEMPERATURE_DEGF
H1:CDS-STAT_NUMBER_OF_ERRORS
Regarding the 4UHV ion pump controller, the autostart setting is enabled now, thus the unit should power ON after a power glitch. See some of the instructions to access the general menu.