E. Capote, J. Kissel, S.Dwyer The ~1 Hz ring-up that had been marginally stable and transient is now fully unstable during the lock acquisition sequence as we recover from maintenance. The suspicion is that the change in ITM M0/R0 satamps might have changed the top mass damping loop OLGTF enough to augment the damped plant that's the plant for the never-really-well-designed L2DAMP that takes the L2 (or PUM) OSEMs and feeds back their sensor signal to the reaction chain top mass OSEMs actuators. But also, we've never measured these loops in any substantial form, so we wanted to just see what they were doing. Attached are the results. Here're the templates: /ligo/svncommon/SusSVN/sus/trunk/QUAD/H1/ITMY/SAGR0/Data/ 2025-07-08_2240UTC_H1SUSITMY_R0_WhiteNoise_0p02to50Hz_L2DAMP_OLGTF_P.xml 2025-07-08_2240UTC_H1SUSITMY_R0_WhiteNoise_0p02to50Hz_L2DAMP_OLGTF_R.xml Very little loop gain and only at 3.3 Hz. The loop stability is questionable at that frequency -- for a few averages the suppression (i.e. the gain peaking) looks really sharp around 3.3 Hz. T'was really tough to get good coherence; the excitation is pretty well tailored, but it's tough to fight the 1/f^6 suppression of the physical suspension and dirt coupling. I had to turn OFF the R0 alignment offsets to get this data for pitch. (They were ON for the Roll measurement). So -- perhaps not the source of the ~1 Hz IFO instability, but boy could this loop use some TLC in order to more effectively achieve its goals...
TITLE: 07/08 Eve Shift: 2330-0500 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: Oli
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 21mph Gusts, 15mph 3min avg
Primary useism: 0.07 μm/s
Secondary useism: 0.06 μm/s
QUICK SUMMARY:
IFO is in DOWN and MAINTENANCE
We have so far been unable to recover lock after maintenance. While we've had roughly 2 hours cumulative of Earthquake-caused lock interference.
We can't lock because there seems to be a strange 1Hz oscillation, thought to be related to the L2 to R0 damping that we have on the ITMs. The sat-amps for these were swithced out today in an attempt to ameliorate the issue, but we don't know if this worked.
Right now, Oli, Jeff and Elenna are taking measurements of the relevant channels in an effort to better understand what's going on with these.
Violins are huge because of morning ITM and ETM measurements from this morning and evening. They are off the screen so we foresee a long damping session tonight (if lucky).
Jordan V, Anna I
The controller for the HAM3 AIP had maxed out on 7/02/25, so during maintenance today we swapped controllers with a refurbished Minivac controller. The ion pump initially was able to hold pressure and even decrease for a couple hours until it railed again, so this is more likely a failure of the ion pump. See attached plot of pressure after the controller swap.
We will swap the ion pump during next Tuesday maintenance.
Gerardo, Janos Today at ~9 am the actuation range of CP7's LLCV valve was shifted towards "more open". This was necessary, as the actuator could not open up enough to maintain the 92% LN2 content in the cryopump's LN2 reservoir, if the LN2 amount in the dewar was less than ~40% (more data about this issue can be found in aLog 84846). The lowest impact solution was to add a shim washer between the bonnet and the cast aluminum frame of the actuator. On the valve's drawing, the bonnet is no. 2 (on the drawing, the actuator frame is not highlighted). The procedure was simple: loosen the locknut (no. 12), lift the actuator assembly (along with the stem), and put a shim washer between it an the bonnet. In order to reach the nut, the only suitable tool is a so-called crowfoot-wrench (see in the picture, along with the U-shaped shim washer). The thickness of the shim washer is 0.1"; the change on the valve "openness" is ~15% (from 95% to 80%). This is enough to handle the issue, however, the stem and/or the whole valve assembly needs replacement or cleaning in the long run, as either there is contamination at the orifice, or the stem is not properly assembled (too long). This will be handled after O4.
The well pump was forced on for 4 hours today beginning at 8:14a local to partially refill the fire water tank.
Closes FAMIS#26050, last checked 85408
HAM2 V1/V3 elevated between 50 - 60 Hz
HAM3 V2/V3 elevated between 50 - 60 Hz
BS St2 V2/V3 becomes elevated above 20 Hz and is sustained, but settles back down around 80 Hz
Last week Edgard and Ivey got new fits for the H1:SUS-SR3_M1_YAW_EST_MODL_SUSP_Y_2GAP and H1:SUS-SR3_M1_YAW_EST_MODL_DRV_Y_2GAP filter banks (85446). Now that we had those, it was time for us to try out the estimator damping again, putting into use all the measurements we did.
To take these regular damping vs estimator damping measurements for SR3 Y:
- I updated those filters in the filter banks listed above
- I exchanged the nominal OSEMINF gains with the gains from 84367
- I turned on the gain in FM7 of the DAMP filter bank that I had found a few weeks ago to counter the new OSEMINF gains (85288)
- Changed the Y damping gain to -0.1
- Additionally, last week we swapped out the SR3 satamp 85463, giving us better noise performance 85485
Measurement times
SR3 Y damp -0.1
2025-07-08 14:54:00 - 15:03:00 UTC
SR3 Y damp -0.1, OSEM damp -0.4
2025-07-08 15:03:15 - 15:48:00 UTC
SR3 Y damp -0.1, Estimator damp -0.4
2025-07-08 15:48:40 - 16:33:00 UTC
The transition between the OSEM damping and estimator damping went well and the estimator did a great job at lowering the noise (ndscope)!
The OSEMINF gain, DAMP Y gain, and DAMP filters have been set back to their nominal settings
The damaged filter housing described in alog 85121 was replaced with a new Donaldson "direct replacement" housing, which of course needed to have the piping modified since the new housing has a wider, thankfully not narrower, connection width than the old housing. The new housing and the 2 pre-existing housings were then fitted with new filter elements. Next week we plan to run the system, flush out the lines into the LVEA, and take FTIR samples to qualify the new Kobelco for acceptance.
WP 12655
ECR E2400330
Drawing D0901284-v5
Modified List T2500232
The following SUS SAT Amps were upgraded per ECR E2400330. Modification improves the whitening stage to reduce ADC noise from 0.05 to 10 Hz.
Suspension | Old | New |
ITMY MO | S1100071 | S1100155 |
ITMY MO/RO | S1100150 | S1100175 |
ITMY RO | S1100075 | S1100178 |
ITMX MO | S1100163 | S1100136 |
ITMX MO/RO | S1100147 | S1000275 |
ITMX RO | S1100132 | S1100090 |
SR2 TOP | S1100112 | S1000296 |
SR2 TOP RT/SD | S1100121 | S1100067 |
F. Clara, J. Kissel, O. Patane, M.Pirello
ITMX ITMY Hardware Watchdog Countdowns, Test Mass Sat Amp swap-outs done in less than 20 minutes.
Here's the characterization data and fit results for S1100155 , assigned to ITMY M0's F1F2F3SD OSEMs (Fil refers to this as just "ITMY MO" above). The data was taken per methods described in T080062-v3. The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ plotresponse_S1100155_ITMY_M0_F1F2F3SD_20250703.m Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are Optic Stage Serial_Number Channel_Number OSEM_Name Zero_Pole_Hz R_TIA_kOhm Foton_Design ITMY M0 S1100155 CH1 F1 0.0966:5.28 120.0 zpk([5.28],[0.0966],1,"n") CH2 F2 0.0961:5.25 120.0 zpk([5.25],[0.0961],1,"n") CH3 F3 0.0960:5.23 120.0 zpk([5.23],[0.0960],1,"n") CH4 SD 0.0951:5.18 120.5 zpk([5.18],[0.0951],1,"n") The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ As with LHO:85504 :: Even though I've fit for the transimpedance gain, I've elected *not* to include a gain in the foton design string relative to "ideal," as there are more scale factors in play that determine the overall [(meters)/(ADC cts)] scale in the calibration of the OSEMs (LED light power, PD response, any cable loss, ADC channel gain, etc.). Determining this overall scale is better left to different methods, a la LHO:84548, which we (eventually) anticipate doing for all SUS with ECR E2400330 upgraded satamps.
Here's the characterization data and fit results for S1100175 , assigned to ITMY M0/R0's LFRT/LFRT OSEMs, respectively (Fil refers to this as just "ITMY MO/RO" above). The data was taken per methods described in T080062-v3. The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ plotresponse_S1100175_ITMY_M0R0_LFRTLFRT_20250703.m Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are Optic Stage Serial_Number Channel_Number OSEM_Name Zero_Pole_Hz R_TIA_kOhm Foton_Design ITMY M0 S1100175 CH1 LF 0.0947:5.18 120 zpk([5.18],[0.0947],1,"n") M0 CH2 RT 0.0969:5.30 120 zpk([5.30],[0.0969],1,"n") R0 CH3 LF 0.0969:5.29 120 zpk([5.29],[0.0969],1,"n") R0 CH4 RT 0.0965:5.28 120 zpk([5.28],[0.0965],1,"n") The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ See above comment LHO:85626 regarding the intentional exclusion of transimpedance gain in the foton design string.
Here's the characterization data and fit results for S1100178 , assigned to ITMY R0's F1F2F3SD OSEMs (Fil refers to this as just "ITMY RO" above). The data was taken per methods described in T080062-v3. The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ plotresponse_S1100178_ITMY_R0_F1F2F3SD_20250703.m Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are Optic Stage Serial_Number Channel_Number OSEM_Name Zero_Pole_Hz R_TIA_kOhm Foton_Design ITMY R0 S1100178 CH1 F1 0.0966:5.29 120 zpk([5.29],[0.0966],1,"n") CH2 F2 0.0957:5.23 120 zpk([5.23],[0.0957],1,"n") CH3 F3 0.0978:5.36 120 zpk([5.36],[0.0978],1,"n") CH4 SD 0.0955:5.22 120 zpk([5.22],[0.0955],1,"n") The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ See above comment LHO:85626 regarding the intentional exclusion of transimpedance gain in the foton design string.
Here's the characterization data and fit results for S1100136 , assigned to ITMX M0's F1F2F3SD OSEMs (Fil refers to this as just "ITMY MO" above). The data was taken per methods described in T080062-v3. The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ plotresponse_S1100136_ITMX_M0_F1F2F3SD_20250703.m Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are Optic Stage Serial_Number Channel_Number OSEM_Name Zero_Pole_Hz R_TIA_kOhm Foton_Design ITMX M0 S1100136 CH1 F1 0.0955:5.22 121.10 zpk([5.22],[0.0955],1,"n") CH2 F2 0.0980:5.35 121.25 zpk([5.35],[0.0980],1,"n") CH3 F3 0.0950:5.18 121.50 zpk([5.18],[0.0950],1,"n") CH4 SD 0.0977:5.34 121.50 zpk([5.34],[0.0977],1,"n") The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ See above comment LHO:85626 regarding the intentional exclusion of transimpedance gain in the foton design string.
Here's the characterization data and fit results for S1000275 , assigned to ITMX M0/R0's LFRT/LFRT OSEMs, respectively (Fil refers to this as just "ITMX MO/RO" above). The data was taken per methods described in T080062-v3. The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ plotresponse_S1000275_ITMX_M0R0_LFRTLFRT_20250703.m Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are Optic Stage Serial_Number Channel_Number OSEM_Name Zero_Pole_Hz R_TIA_kOhm Foton_Design ITMX M0 S1000275 CH1 LF 0.0963:5.27 120 zpk([5.27],[0.0963],1,"n") M0 CH2 RT 0.0950:5.19 120 zpk([5.19],[0.0950],1,"n") R0 CH3 LF 0.0960:5.26 120 zpk([5.26],[0.0960],1,"n") R0 CH4 RT 0.0960:5.26 120 zpk([5.26],[0.0960],1,"n") The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ See above comment LHO:85626 regarding the intentional exclusion of transimpedance gain in the foton design string.
Here's the characterization data and fit results for S1100090 , assigned to ITMX R0's F1F2F3SD OSEMs (Fil refers to this as just "ITMX RO" above). The data was taken per methods described in T080062-v3. The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ plotresponse_S1100090_ITMX_R0_F1F2F3SD_20250703.m Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are Optic Stage Serial_Number Channel_Number OSEM_Name Zero_Pole_Hz R_TIA_kOhm Foton_Design ITMX R0 S1100090 CH1 F1 0.0945:5.16 120.25 zpk([5.16],[0.0945],1,"n") CH2 F2 0.0963:5.26 120.00 zpk([5.26],[0.0963],1,"n") CH3 F3 0.0977:5.34 120.00 zpk([5.34],[0.0977],1,"n") CH4 SD 0.0950:5.19 120.00 zpk([5.19],[0.0950],1,"n") The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ See above comment LHO:85626 regarding the intentional exclusion of transimpedance gain in the foton design string.
Here's the characterization data and fit results for S1000296 , assigned to SR2 M1's T1T2T3LF OSEMs (Fil refers to this as just "SR2 TOP" above). The data was taken per methods described in T080062-v3. The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ plotresponse_S1000296_SR2_M1_T1T2T3LF_20250703.m Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are Optic Stage Serial_Number Channel_Number OSEM_Name Zero_Pole_Hz R_TIA_kOhm Foton_Design SR2 M1 S1000296 CH1 T1 0.0955:5.22 122.00 zpk([5.22],[0.0955],1,"n") CH2 T2 0.0960:5.25 121.75 zpk([5.25],[0.0960],1,"n") CH3 T3 0.0969:5.28 122.25 zpk([5.28],[0.0969],1,"n") CH4 LF 0.0929:5.06 121.50 zpk([5.06],[0.0929],1,"n") The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ See above comment LHO:85626 regarding the intentional exclusion of transimpedance gain in the foton design string.
Here's the characterization data and fit results for S1100067 , assigned to SR2 M1's RTSDxxxx OSEMs (Fil refers to this as just "SR2 RT/SD" above). The data was taken per methods described in T080062-v3. The data was processed and fit using ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ plotresponse_S1100067_SR2_M1_RTSDxxxx_20250703.m Explicitly, the fit to the whitening stage zero and pole, the transimpedance feedback resistor, and foton design string are Optic Stage Serial_Number Channel_Number OSEM_Name Zero_Pole_Hz R_TIA_kOhm Foton_Design SR2 M1 S1100067 CH1 RT 0.0977:5.35 120 zpk([5.35],[0.0977],1,"n") CH2 SD 0.0975:5.34 120 zpk([5.34],[0.0975],1,"n") CH3 xx 0.0959:5.25 120 zpk([5.25],[0.0959],1,"n") CH4 xx 0.0969:5.31 120 zpk([5.31],[0.0969],1,"n") The attached plot and machine readable .txt file version of the above table are also found in ${SusSVN}/trunk/electronicstesting/lho_electronics_testing/satamp/ECR_E2400330/Scripts/ See above comment LHO:85626 regarding the intentional exclusion of transimpedance gain in the foton design string.
Got a notification for H1 assistance. H1 dropped from observing just under 3.5hrs ago. After a couple of PRMI cycles, DRMI did lock within 40min, but immediately lost lock at DRMI LOCKED CHECK ASC. Then an Initial Alignment was immediately run in just under an hour @1024UTC. At this point, H1 attempted lock but had another lockloss at DRMI LOCKED CHECK ASC again, but after this lockloss H1 made it up to CARM 5PicoMeters for the next lockloss. Next attempt made it to a CARM OFFSET Reduction Lockloss. Then after no luck locking PRMI and it being over 3hrs, I got the wake up call.
It's sounding like H1 has been having this Locking ailment as of late.
Going to let it finish a 2nd IA I started at @1253UTC after seeing that PRMI looked fine (flashes looked good on camera and are over 100 counts for POP18/90). Initial Alignment just completed and locking clock is restarting.
(Which is good, because my NoMachine is having the symptom that the mouse pointer does not match it's location in my No Machine session intermittantly (basically I'll have the mouse pointer over a spot, i.e. "X" to close an MEDM window, but when I try to click the "X" with the pointer, the pointer is actually, in the No Machine session, about 10" to the left (or right---basically somewhere else!), so it's hard to operate in my NoMachine session. The mouse would be fine on my laptop's monitor. I've had this issue off and on for a few months. With that said, ....after about 15min, my mouse now works OK in NoMachine!)
H1 is now back to DRMI (after the recent Alignment). Camera flashes look centered and POP18/90 flashes are just under 200. Leaving H1 for automatic operations while I see if I can get return to sleep. At 1325UTC/625amPDT, DRMI locked, btw.
It looks like before Corey was called, there were some locking attempts that got up past DRMI. Before he was called, there were two locklosses from DRMI_LOCKED_CHECK_ASC (which we have figured out was due to some SRC1 offsets being turned on by accident), then one lockloss from CARM_5_PICOMETERS, then two lockloss from CARM_OFFSET_REDUCTION. So that makes a lot more sense as to why it didn't call for three hours!
Today while in DRMI ASC, and while trying to debug other problems with DRMI acquisition, Ryan, Tony, and I saw that the DRMI ASC starting pulling the buildups in a bad direction, which made no sense. We were trying to figure out which loops were the culprit, when I saw that the SRC1 offsets were engaged. These offsets had been put in place during the problems with the OFI, and we don't run with these offsets in full lock anymore. I turned the offsets off and the buildups starting moving in the good direction again. This is very confusing, because we've been running like this for ages probably without a problem. Today it was suddenly a problem. I commented out the lines in the ISC_DRMI guardian state PREP_DRMI_ASC where these offsets are turned on and loaded.
I made a minor mistake here- I only commented out the lines in ISC_DRMI where the offset is SET, but I didn't comment out lines where the offset is turned ON. However, ISC_LOCK sets a random offset in SRC1 as well, and doesn't turn it on, but then we were in a situation where ISC_LOCK sets a weird offset, and then ISC_DRMI turns it ON. This meant today the DRMI ASC came on in a very strange way and pulled the alignment far off. I have now commented out all lines in both ISC_DRMI and ISC_LOCK that set these offsets, and turn the offsets on. Hopefully, this won't be an issue again.
TITLE: 06/27 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 9mph Gusts, 4mph 3min avg
Primary useism: 0.01 μm/s
Secondary useism: 0.05 μm/s
QUICK SUMMARY:
14:46 UTC DRMI is struggling when ASC starts, I'm going to run a manual_IA
At the time of the request for help 10:22 UTC, we were in in CHECK_AS_SHUTTER where it was presumably stuck at SHUTTER_FAIL which I've encountered again at 15:22 UTC. It's been in that state since 09:05 UTC when we lost lock from 25Ws and the SHUTTER GRD reported "No kick... peak GS13 signal = 51.226"
The shutter did not trigger in this last lockloss, it looks like the light heading to the AS port was not high enough to trigger the shutter.
The lockloss_shutter_check guardian check for a kick in the HAM6 GS 13s anytime that we loose lock with more than 25kW circulating power in the arms. In this case we had just reach 100kW circulating power, so the guardian expected the shutter to trigger. This lockloss looks unusual in that there isn't a increase in the power going to the AS port right before the lockloss.
Ryan ran the shutter test and the shutter is working correctly now.
I think that this is probably the result of an usual lockloss happening at somewhat lower circulating power than usual. We should probably edit the logic in IFO notify to call for operator assistance whenever the shutter is in the failed state.
This is OK, the AS port went dark and stayed there for about 70ms or so after the lockloss and there was no excessive power surge that would have caused the fast shutter to be triggered.
The maximum power of ~1.4W was observed ~160ms after the lockloss, which is well below the threshold for the analog FS trigger (3 to 4W, I don't remember the exact number).
If something similar happened with 60W, though, FS might have been triggered.
Attached is the estimate of the power coming into HAM6 using two different sensors (PEM-CS_ADC_5_19 = HAM6 power sensor in the AS camera can which monitors power before the fast shutter, and ASC-AS_A_DC_NSUM after). Neither of these have hardware whitening, neither saturated (ASA was close to saturation but the HAM6 power sensor saturation threshold is about 570W when beam diverter is open, 5.7k if closed).
Note that the calibration of the PEM channel is a factor of 10 smaller than that in the observation mode (0.177W/ct, see alog 81112 and git repo for lock loss tool) because the beam diverter (90:10) was open.
I made a timeline of what ISC_LOCK was doing during this event.