Sheila, Naoki
As shown in the attached figure, SQZ ASC ran away and was turned off since the SQZ ASC trigger was below the threshold. We removed squeezing and reset the AS42 offset and push the graceful clear history. After we brought back the squeezing, everything seems working now.
Even with new input matrix, SQZ ASC ran away today. I reset the AS42 offset and pushed the graceful clear history. The SQZ ASC came back, but I am not sure why this happened.
Naoki, Vicky - We added checkers to both SQZ_MANAGER and SQZ_FC guardians, to give notifications if ASC is not on.
SQZ_MANAGER now checks for ASC on if SQZ-ASC_ANG_P/Y_INMON = 0; same for filter cavity asc, which checks if SQZ-FC_ASC_ANG_P/Y_INMON = 0. Earlier today, looks like the ASC_WFS switch was ON but the alignment ran away, so the signal fell below trigger threshold, and AS42 ASC did not engage. Hope these notifications make it easier to catch when ASC isn't on.
These guardian edits for SQZ_MANAGER and SQZ_FC were the only SVN diffs; the changes are now committed to SVN version 25945.
T. Guidry During today's testing of the OSB fire suppression system all air handlers that serve the OSB were taken down. This is a normal and necessary function of our fire alarm system during an alarm. All AHU's are back online and functioning normally. The duration of the interruption was 37 minutes (10:08-10:45 local time)
Here are a couple plots of oplevs and OSEM signals during our last lock, focusing on the change related to turning on the OM2 TSAMS. I plotted test mass top mass OSEMs and oplevs (in case they told different stories), and the OSEMs for SRM, SR2, OM3 and OMC.
Conclusion: there doesn't appear to be an alignment change in the arms. We hypothesized that a change in the signal to WFS 45 could change the DHARD and MICH alignment loops. That doesn't appear to be the case. However, we do see a pretty clear change in the OM3 and OMC alignment. There might be some OMC ASC alignment offsets we could try in order to mimic the changes we are seeing from the OM2 change.
From squeezer ASC trends as OM2 heats up, we also see an alignment shift, most strongly related to AS_A_RF45_Pitch (corresponds to SQZ-ASC_ANG_P_OUTPUT). Not implausible that the output alignment is what's changing; this would also be consistent with having to reset SQZ ASC offsets 70890 with a hot OM2.
At 76W, Camilla and I tried walking OMC ASC QPD offsets and didn't find anything better (69307). Before at 60W, Koji had success increasing the optical gain by walking the OMC ASC QPD offsets, but that was in mid-March before great TCS tuning, 67994. I thought this helped to improve and stabilize the range then. He also light heated OM2 after the new OMC ASC offsets 68048 and saw a similar ~2% reduction in optical gain, consistent with now 70859.
So, I also think it makes sense to try walking the OMC alignment, and generally the IFO's output alignment.
Fast Channels Removed:
> H1:LSC-POPAIR_B_RF18_I_ERR_16k_DQ 4 16384
> H1:LSC-POPAIR_B_RF90_I_ERR_16k_DQ 4 16384
Slow Channels Removed:
> H1:VID-CAM15_AUTO 4 16
> H1:VID-CAM15_XY 4 16
Implemented my code to speed up lockloss down time in the ACQUIRE_DRMI_1F and ACQUIRE_PRMI checks with a successful run!
Changes made to my code since previous post (70765):
In ACQUIRE_DRMI_1F, switched to check POPAIR_B_RF18_I_ERR_DQ data instead of POPAIR_B_RF90_I_ERR_DQ and lowered max threshold to 130 from 200. Changes made due to better correlation between RF18 and DRMI.
Code is easy to update if better thresholds are found.
WP11274 Replace HAM6 camera, h1cam15
Fernando, Fil, Betsy, Patrick, Erik, Dave:
h1cam15 (HAM6) camera was replaced (see Fil's alog for details). Patrick install a new server for this camera on h1digivideo3 (was running on h1digivideo1). Erik added the new camera to the DHCP server, I added the new camera to H1EPICS_DIGVIDEO.ini. DAQ restart was required.
WP11276 Add FCES channels to FMCS EPICS
Patrick, Dave:
Patrick added the FCES FMCS channels to his EPICS IOC (see Patrick's alog for details). I added the new channels to H0EPICS_FMCS.ini. DAQ restart was required.
WP11285 CW hardware injection code, use gpstime, discontinue using tconvert
Joe B, Keith R, Mike Thomas, Erik, Dave:
Following LLO, we upgraded the psinject (CW hardware injection) code on h1hwinj1 to no longer use tconvert to get GPS time, and use gpstime instead.
For gpstime to run, the version of python3 needed to be updated from 3.4 to 3.6. Mike did this and reinstalled gpstime.
I applied Joe's changes by "svn up" to get the latest code and verifying the changes.
When we tried to run the new code it would not start. We tracked this down to an install issue, lalapps were removed by the upgrade. Mike re-installed lalapps and the new code is now running.
The first run only ran for about a minute, exiting with an awg-stream error. On the second restart awg-stream connected and the exitation is stable.
WP11278 h1lsc add filtermodules, remove two 16kHz DQ channels
Dave:
I installed the latest h1lsc model (two additional filter modules) and changed the DAQ_Channels block to no longer send the H1:LSC-POPAIR_B_RF[18,90]_I_ERR_16k_DQ channels to the frame. DAQ restart was required.
DAQ Restart
Dave:
The DAQ was restarted twice, first around 10:45 in conjuction with h1lsc model and EDC restarts to apply LSC changes and add the FCES FMCS channels.
A second restart, along with a EDC restart, around 12:46 to add the new h1cam15 channels.
Tue27Jun2023
LOC TIME HOSTNAME MODEL/REBOOT
10:42:56 h1lsc0 h1lsc <<< add 2 FM, remove 2 DQ channels
10:43:54 h1daqdc0 [DAQ] <<< 0-leg
10:44:07 h1daqfw0 [DAQ]
10:44:07 h1daqnds0 [DAQ]
10:44:07 h1daqtw0 [DAQ]
10:44:15 h1daqgds0 [DAQ]
10:44:50 h1susauxb123 h1edc[DAQ] <<< EDC add FCES FMCS channels
10:45:36 h1daqgds0 [DAQ] <<< 2nd gds0 restart
10:48:03 h1daqdc1 [DAQ] <<< 1-leg
10:48:12 h1daqfw1 [DAQ]
10:48:12 h1daqtw1 [DAQ]
10:48:13 h1daqnds1 [DAQ]
10:48:21 h1daqgds1 [DAQ]
10:49:06 h1daqgds1 [DAQ] <<< 2nd gds1 restart
12:46:33 h1daqdc0 [DAQ] <<< 0-leg
12:46:45 h1daqfw0 [DAQ]
12:46:45 h1daqtw0 [DAQ]
12:46:46 h1daqnds0 [DAQ]
12:46:54 h1daqgds0 [DAQ]
12:47:20 h1susauxb123 h1edc[DAQ] <<< Add h1cam15 new chans
12:48:00 h1daqgds0 [DAQ] <<< 2nd gds0 restart
12:50:47 h1daqdc1 [DAQ] <<< 1-leg restart
12:50:57 h1daqfw1 [DAQ]
12:50:57 h1daqtw1 [DAQ]
12:50:59 h1daqnds1 [DAQ]
12:51:07 h1daqgds1 [DAQ]
12:51:41 h1daqgds1 [DAQ] <<< 2nd gds1 restart
The camera has been removed from the IOC and the server has been stopped and removed from monit on h1digivideo1. The camera has been added to puppet control on h1digivideo3. The H1VID_DIGITAL_OVERVIEW.adl medm screen has been updated. There is a video image, but the IFO is not locked, so it is black.
There is now a beam image.
Maintenance and measurements finished up just before 1pm. We are relocking now and skipped initial alignment, so far it's looking good.
WP 11283
Pulled in a network and power cable from BSC9 to the mechanical room. Currently the VEA Beckhoff electronics are powered from the SUS field rack. Power will be moved to a dedicated power supply in the mechanical room. An open frame rack was installed in the mechanical room. This will house the power supply, Beckhoff computer, and Beckhoff terminals.
Tue Jun 27 10:08:06 2023 INFO: Fill completed in 8min 5secs
Closes WP 11276. List of added channels: H0:FMC-FCES_AHU1_COLD_COIL_TEMP_DEGF H0:FMC-FCES_AHU1_FAN_SPEED H0:FMC-FCES_AHU1_HEPA_PRESS H0:FMC-FCES_AHU1_MIXED_TEMP_DEGF H0:FMC-FCES_AHU1_PRESS H0:FMC-FCES_AHU1_RELIEF_FAN_AIRFLOW_CFM H0:FMC-FCES_AHU1_RETURN_TEMP_DEGF H0:FMC-FCES_AHU1_SUP_FAN_AIRFLOW_CFM H0:FMC-FCES_AHU1_SUP_TEMP_DEGF H0:FMC-FCES_AUX_OSA_TEMP_DEGF H0:FMC-FCES_AUX_OUTSIDE_HUMIDITY H0:FMC-FCES_AUX_SPACE_HUMIDITY H0:FMC-FCES_AUX_SPACE_PRESS H0:FMC-FCES_AUX_SPACE_TEMP_DEGF H0:FMC-FCES_AUX_SUP_TEMP_DEGF H0:FMC-FCES_H2O_TANK_LEVEL H0:FMC-FCES_VEA_OSA_TEMP_DEGF H0:FMC-FCES_VEA_OUTSIDE_HUMIDITY H0:FMC-FCES_VEA_SPACE_HUMIDITY H0:FMC-FCES_VEA_SPACE_PRESS H0:FMC-FCES_VEA_SPACE_TEMP_DEGF H0:FMC-FCES_VEA_SUP_TEMP_DEGF H0:FMC-FCES_AHU1_RELIEF_FAN_STATUS H0:FMC-FCES_AHU1_SUP_FAN_STATUS H0:FMC-FCES_AUX_MAIN_FAN_STATUS H0:FMC-FCES_H2O_TANK_LOW_LEVEL_ALARM H0:FMC-FCES_VEA_MAIN_FAN_STATUS H0:FMC-FCES_AHU1_COLD_COIL_TEMP_DEGC H0:FMC-FCES_AHU1_MIXED_TEMP_DEGC H0:FMC-FCES_AHU1_RETURN_TEMP_DEGC H0:FMC-FCES_AHU1_SUP_TEMP_DEGC H0:FMC-FCES_AUX_OSA_TEMP_DEGC H0:FMC-FCES_AUX_SPACE_TEMP_DEGC H0:FMC-FCES_AUX_SUP_TEMP_DEGC H0:FMC-FCES_VEA_OSA_TEMP_DEGC H0:FMC-FCES_VEA_SPACE_TEMP_DEGC H0:FMC-FCES_VEA_SUP_TEMP_DEGC
Dave restored the alarm fields using the snapshot file from /ligo/cds/lho/h0/burt/2022/01/31/00:00/h0fmcs.snap. He wrote a script to parse out the values and caput them.
WP 11274
Betsy transitioned to laser hazard. The OMC HAM6 camera was replaced. Betsy/Sheila realigned new camera using local laptop. Betsy transitioned back to laser safe.
Afterwards Patrick/Dave/Erik updated the server/software. Camera housing is a smaller version of our standard camera enclosure. The network cable on the rear of the camera hits the back of the housing.
New Camera Basler ACA720-290gm
Old Camera Basler ACA640-120gm
D. Barker, F. Clara, S. Dywer, P. Thomas, B Weaver
Per WP11286 in response to ALOG70818
We installed the spare S1302233 Baffle PD Amplifier directly below the installed Baffle PD Amplifiers into SUS R5 in the Biergarten. ITMX is S1400066 and ITMY was S1400065 but we swapped functionality to the spare we just mounted below it (this rack is mostly empty).
If there are issues with the spare we can immediately swap functality back to the old unit. If the spare fixes the issues, we will recover the old unit and perform a post mortem. As there was no laser light during install we could not verify functionality while on the floor.
M. Pirello, F. Mera
We seem to see some signal on this pd now, unlike before. Interestingly, the pd seems a fair amount of light during the ISC_LOCK state of Move_Spots. Clearly we are moving through a place with high scatter, best to avoid.
I pulled the ITMY Baffle PD Amplifer S1400065 and moved the replacement S1302232 up to occupy positions 11 & 12 to the right of the ITMX Baffle PD Amplifier. Post mortem on S1400065 showed that one of the internal cables was loose, this has been fixed and all cables pull tested. S1400065 has been included in the spares inventory.
This completes WP11286
Since we are back to a lower and more stable operating power, I reduced the power step up time in the MAX_POWER guardian state. Each 5W step up from 25W took 60 seconds, now that time is down to 45 seconds. Tomorrow, I'd like to try a 30 second step time, which I think should be fine since the ASC is much more stable up to 60W. This has already reduced the relocking time by almost 2 minutes, and will reduce it more.
Today I reduced this timer to 30 seconds. We just had a successful power up! This is now guardianized. Should shave 3.5 minutes off the locking process.
Note: the ASC loops are notoriously unreliable. I encourage operators to pay attention to the power up process and if there are problems with ASC instabilities during the final power up, increasing this timer will likely be a good solution. Tagging ops.
Reduced the baffle PD gain in ETMX PD 1 to 0dB from 20dB (H1:AOS-ETMX_BAFFLEPD_1_GAIN), since it was pegged at 10V.
ETMX baffle PDs 2, 3 and 4 are ok and gain at 20dB.
ETMY baffled PD 1 is close to saturation and gain is 20dB.
ETMY baffle PDs 2 and 3 have a gain of 0dB and show very small signals. Could be increased to 20dB.
ETMY baffle PD 4 saturates at the begining of a lock. Gain could be decreased to 0dB.
ITMX baffle PD 1 gain at 0dB, 20dB for all others, signal ok.
The ITMY baffle PD 3 readout seems broken since at least January 2022; maybe related to this upgrade.
All gains at 20dB, signals for all but PD 3 seem ok.
The attached trend plot shows all baffled PDs at 75W and 60W input. The transition happens near -5d in the plot. The power reduction is as high as a factor of 2 but typically much less.
Daniel has also taken it as an action item to work with an operator to ensure that the gain changes are handled nicely by the baffle PD script, since we use PDs 1 and 4 on each test mass to help us align the green beam if we cannot see flashes. Daniel's suggestion is to have the script increase the gain values to what they had been, and then after the script completes, return them to the lower values that they have now. This should allow the PDs to be useful for both the pre-initial alignment, as well as be useful when in full lock.
Changed and restored the baffled PD gain in the baffleAlign.py script (function alignToBafflePD). Needs testing.
Script tested
Vibration coupling has increased since early in the O3-O4 break, both input jitter coupling and scattering noise. Jitter coupling at the PSL increased by a factor of about 5 between 2021/05/12 (just after TM replacement), when IMC-PWR-IN indicates we were running at 46 W, and 2023/03/17 when we were running at 60 W, just before the increase to 75W (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=68036 ).
Scattering noise from the cryobaffle at EX increased dramatically between 2020/09/14 when we were running at 38W, and during PEM injections this May when we were at 75W (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=69578 ). Scattering noise also increased at the LLO EX cryobaffle (https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=65641 ).
We have speculated that this increase in coupling is associated with the power increases and might be a general tendency. It might be caused by increased thermal distortion of test mass surfaces around coating defects, resulting in a greater fraction of light scattered and higher vibration coupling through retro-reflected scattered light. Greater thermal distortions associated with defects might also make the arms less symmetrical, reducing common mode rejection of input noise and increasing vibration coupling at the PSL. An alternative hypothesis is that we increased clipping or vignietting when power increased.
The drop in power this week provides an opportunity to test the correlation between power and vibration coupling, and to test it with evaluations right before and after a power change, rather than the years intervening between the measurements referenced above.
Jitter coupling dropped by a factor of roughly two with the reduction in power from 75W to 60W
Figure 1 (3 pages) shows a comparison of the predicted contribution of background vibration to DARM made during PEM injections in May, when we were running at 75 W, versus 60W from this week. While the difference varies across the spectra, I would say the average reduction in jitter coupling is about a factor of two.
The coupling at 60W is simiar to the coupling at 60W measured in March, before we went to 75W,
The amplitude of scattering peaks from EX cryo-baffle decreased by a factor of 3 with the reduction from 75W to 60W
Figure 2 shows that, for similar velocities of the beam tube supporting the cryo-baffle, the amplitude of scattering peaks at harmonics of 4Hz dropped by a facor of about 3 with the decrease in power.
Coupling of the EX cryo-baffle did not strongly limit us at 75 W, though it was close; at 60 W an increase in motion by about a factor of 2.5 causes peaks to show in DARM
An important question for the eventual return to 75W is whether we were limited by cryobaffle scattering in the 20-40 Hz region at 75W or whether some other noise dominated. Figure 3 (two pages) shows, on page one, that, at 75W, an increase in motion of about 1.5 was needed for the peaks to show clearly in DARM. On the second page, a 1000-average spectrum at 75 W does not clearly show peaks at harmonics of 4Hz, supporting the argument that some other noise source dominated in this band.
Figure 4 shows that, at the new 60 W setting, an increase in motion by a factor of 2.5 is now needed for peaks to show in DARM. Thus the 4Hz peaks may show during high winds, off-road traffic within a couple of km, and other activities that increase ground motion at 4 Hz by at least 2.5.
Is there a visible difference in the pattern of scattered light at different powers?
Figure 5 shows photographs taken in 2019 (38W) and 2023 (75W) of ETMX during high-sensitivity operation. I don’t see any obvious changes in the light pattern but there is perhaps relatively more light lighting up the cage at 75W, consistent with an increased fraction of light scattered out of the beam.
Peter, Wen
We took a look at the arm cavity baffle PD before/after powering down. These PD are registering low-angle scatter, which includes higher-order spatial modes of the beam scattered from possible thermal deformation of the test mass. The power measured on the baffle PD (after IFO thermalized) depends on the arm power, beam spatial position relative to each baffle, and distance between beam spot center and point absorber if there are any. The measured PD power (dark offset subtracted) are summarized in the table below.
| Variable | Measured value @ 75W [a.u.] | Measured value @ 60W [a.u.] | Normalized PD power @ 75 W / Normalized power @ 60 W |
| Input power on PRM | 71.2 | 57.3 | |
| PRG | 47.1 | 50.4 | |
| ITMX PD1 | 2.47 | 1.77 | 1.20 |
| ITMX PD2 | 0.033 | 0.027 | 1.05 |
| ITMX PD3 | 0.061 | 0.052 | 1.00 |
| ITMX PD4 | Saturated | 0.080 | N/A |
| ETMX PD1 | Saturated | Saturated | N/A |
| ETMX PD2 | 0.068 | 0.055 | 1.06 |
| ETMX PD3 | 0.034 | 0.026 | 1.12 |
| ETMX PD4 | 0.645 | 0.346 | 1.60 |
| ITMY PD1 | 0.96 | 0.66 | 1.24 |
| ITMY PD2 | 0.070 | 0.055 | 1.09 |
| ITMY PD3 | Dead | Dead | N/A |
| ITMY PD4 | 0.26 | 0.21 | 1.06 |
| ETMY PD1 | Saturated | 0.85 | N/A |
| ETMY PD2 | 0.016 | 0.014 | 0.98 |
| ETMY PD3 | 0.044 | 0.041 | 0.92 |
| ETMY PD4 | 0.95 | 0.80 | 1.02 |
The normalized PD power is computed by (Measured PD power) / (input*PRG*270/2). There's more round-trip loss in ppm when the IFO was operating at higher power (last column number > 1) for the X arm. Y arm doesn't see as much of a difference as X arm. This agrees with our knowledge that ITMX has point absorbers. However, the uncertainties of the results are large, since we are assuming that the beam position doesn't change before/after powering down. Note that PD1 and PD4 are much closer to the beam spot center (see llo54050) and have much higher SNR than PD2 and PD3. However, 3 of total 8 PD are saturated.
The time series of the baffle PD are shown here: ITMX, ETMX, ITMY, ETMY.
WP 11173
Rack 1 – Electronics Bay
Rack 2 – FE Rack (Receiving area)
Continued with troubleshooting of the EY rack temperatures. On May 4th, (alog 69322), we swapped the field rack 1 and rack 2 cabling on the End Link Chassis. Issue followed field cabling not Beckhoff channel. This morning I reverted the cable swapp and moved cables from ports 1&2 to 3&4. Internal cabling from rear panel to EL3202 terminal was re-wired.
All connections on both temperature sensors (RTP PT100) were re-landed to make sure all connections were making solid contact. A new field cable and temperature sensor was staged in the electronics bay incase it is needed for more troubleshooting.
The M12 connectors (ports 1&2) were replaced on the End Link Chassis. The field and internal chassis cabling was moved from ports 3&4 back to ports 1&2.