After Filiberto made the hardware changes to make the end X Beckhoff HEPI pump controller match the end Y Beckhoff HEPI pump controller, I went in and updated the software. I installed Git and Visual Studio 2019, uninstalled Visual Studio 2017 (I needed the disk space) and updated the version of TwinCAT 3 to match that at end Y. I checked the code out of Git, and was able to cut, copy and paste the terminals in the IO tree to match the new configuration without losing any variable links. It is now running the code at commit c5b7d9505806fc91ef30caae79ca4e95c4f0c216 from https://git.ligo.org/cds/ifo/beckhoff/hepi-pump-ctrl/h1-hepi-pump-ctrl-ex.
Tue Jul 25 10:23:51 2023 INFO: Fill completed in 8min 47secs
The code did not start at the scheduled time of 10:00:00, it was rescheduled and started at 10:15:00
EX HEPI has been recovered, so we are starting recovery of H1. Starting with initial alignment.
DIAG_MAIN reports that the IMC WFS need centering.
This morning I was able to epoxy 2 L4Cs under HAM1, as I outlined last night. I was also able to get the transfer functions, get a quick design for a filter and test it. So it works pretty well, but there is still a fair amount of work to do. First attached asds show the improvement I was able to get on HAM1 with this initial ff on vs off, red and green are the new ground L4Cs, pink and blue are the in loop HEPI L4Cs, where pink is the FF off and blue is with the FF on. Between a couple hz and 25 hz I was able to get up to a factor of 10 reduction in the Z motion of the HEPI. Out at 50hz the motion is a little bit worse, but I have some more tuning to do. Should be able to fix that.
Second attached asds compares the HAM1 improvement to some of the other HAM ISIs. Brown is the HAM1 HEPI with FF off, greens is with the FF on. Pink is the HAM2 GS13s and light blue are the HAM7 GS13s. Basically, HAM2 is doing as good HAM1 at 15hz, without additional FF and doing better by factors of 10 or more everywhere else. HAM7 is better than either by ~20 at 15hz, because of in-vac FF L4Cs and its stiffer support structure, which shows up as the peak in light blue at 38hz.
More work to be done, so we won't trying running this full time yet. I will post pics of L4C locations and some filter design data in a bit. I will also try to get some quick on/off data with the interferometer this afternoon. This could affect the HAM1 asc feedforward, so I'll get in touch with Gabriele or Elenna about that.
Adding photos of the L4Cs in place. First one is by the ISC table on the west side of the chamber, by the -X/+Y pier, serial number L41419. This has a styrofoam insulating cover, and is kind of hard to get to, I had to crawl under the ISCT1. Second photo is by the -X/+Y pier on the east side of the chamber. I don't have the cover for this yet, serial number is L41444.
LVEA swept as in T1500386.
Nothing of note. Unplugged forklift in High-bay, Mega cleanroom light off, unplugged a couple of extension cords and took the batteries out of PSL and SQZ bay (already dead) phones.
Noisiest things in LVEA: MSR fans, EX VAC computer stand, HWS power supply (already on foam pad), pipes above SQZ tables.
Wrapping up maintenance, only EX HEPI pump station upgrade left. The LVEA has been swept, and we are ready to lock as soon as we get HEPI back.
Jenne, Camilla
In 71559 Ryan C and Daniel found that there was a DHARD_Y Transient during DHARD_WFS state that was larger since June 29th, coinciding with the time the violins starting to ring up 71404.
Yesterday TJ and Oli increased the DHARD_P and _Y TRAMPs from 5s to 15s in this state 71672 and in the two relocks since then we've only had to stay in OMC_ WHITNIENTING damping violins <15 minutes 71676. The transient is much smaller after this TRAMP increase but still there and a factor 2 larger than pre-June 29th.
The step response of the only filter on at the time (FM6 "newcntrl") is very big when it's turned on, but over in 1sec, attached. Currently FM6 is turned on and then 1 second later the input and gain is turned on, plot attached. Jenne suggests as the step response starts when the input is turned on, we should turn the input on with FM6 and then after 1 second, ramp the gain up. I've added this to ISC_LOCK and loaded. Hope this will remove the transient and then we can reduce the tramp back to 5s.
Daniel pointed out that FM4,7,9 are turned off when FM6 is turned on. These filters all have 5 second ramp times so we are probably getting the transient from these filters being still ramping down as the gain is ramped up. I've accepted these as OFF in the safe.snap sdf see attached. So they should be reverted on sdf revert.
We saw a similar transient in DHARD_P but haven't looked at it yet so we should repeat this process with other filters.
Most of the individual filters in this module have a 2-5s ramp time. For whatever reason, FM4/5/7/9 are all on just before DHARY gets engaged, They then get turned off while FM6 is turned on, a short time before the input to the moduleis turned on. However, there is not enough time for these filters to ramp off, so they are still on during the inital gain ramping.
This transient is not visible in the optics anymore, see attached. I'll put the tramp back to 5s but ISC_LOCK will need to be reloaded when we are out of observe. Opened and closed FRS 28647.
Brina, Genevieve and Lance are going to look for transients from other filters and at other times, Elenna suggested checking DHARD_P.
J. Kissel Completing the template building that I started last week (LHO:71465), I've finished the M3 optical lever to M2 OSEM coils open loop gain transfer function characterization. The templates work for both with M1 to M1 OSEM damping loops ON and OFF, and I characterized the M3-to-M3 oplev loops under both conditions of M1-to-M1 OSEM loops. The M2 coil driver is in state 2 (ACQ ON, LP OFF) and the OLDAMP filters' output limiters are OFF. /ligo/svncommon/SusSVN/sus/trunk/BSFM/H1/BS/Common/Data/ 2023-07-25_1529UTC_H1SUSBS_M2M3_OplevDamping_P_WhiteNoise_0p01to50Hz_OpenLoopGainTF.xml (M1-to-M1 OSEM damping ON) 2023-07-25_1529UTC_H1SUSBS_M2M3_OplevDamping_Y_WhiteNoise_0p01to50Hz_OpenLoopGainTF.xml 2023-07-25_1600UTC_H1SUSBS_M2M3_OplevDamping_P_WhiteNoise_0p01to50Hz_OpenLoopGainTF.xml (M1-to-M1 OSEM damping OFF) 2023-07-25_1600UTC_H1SUSBS_M2M3_OplevDamping_Y_WhiteNoise_0p01to50Hz_OpenLoopGainTF.xml I'll now take this data, as well as that from LHO:71269 make some plots and a few statements about the loops (in a separate log, in due time).
We've noticed the ISS diffracted power trending high in the past week, so I've adjusted the RefSignal to -1.98V to bring the diffracted power back down to around 2.5% from 3.0%. I also accepted this value in the PSLISS safe.snap table.
Workstation OS packages and Conda environment packages were both updated around 1330 UTC. These updates triggered a latent bug in the conda startup code that prevented graphical login to the workstations. This was cleared up by patching the conda startup code at around 1545 UTC.
The following conda packages were updated:
cds-crtools -> 4.0.2
* Higher precision cursor readouts for diaggui and foton.
* arbitrary awgstream memory leak fixed
* improved stability when reading live data from NDS2 servers
Complete change history is here: https://git.ligo.org/cds/software/dtt/-/wikis/ChangeHistory
dttxml -> 1.1.7
* Some transfer functions which could not previously be exported can now be exported.
TITLE: 07/25 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 7mph Gusts, 4mph 5min avg
Primary useism: 0.09 μm/s
Secondary useism: 0.06 μm/s
QUICK SUMMARY: Started off the day with issues logging into the control room work stations, but maintenance is moving on. Both PEM injections and SUS charge measurements were taken before the lock was intentionally killed for other on site measurements.
I've recalibrated the PSL rotation stage this morning using the following steps:
| Power in (W) | D | B (Minimum power angle) | C (Minimum power) | |
| Old Values | 104.199 | 1.990 | -24.816 | 0.000 |
| New Values | 100.917 | 1.990 | -24.827 | 0.000 |
After our first lock loss this morning I loaded I loaded in changes I made last week.
We have evidence that DARM noise in the 10-50 Hz region seems to be modulated by residual motion of DHARD_Y at 2.6 Hz.
At the same time, now DHARD_Y is not limiting DARM by linear coupling above 10 Hz, due to the improved A2L (-30 db)
So there's room to test a new DHARD_Y controller that could give us 2x more suppression at 2.6 Hz, at the price of about 2x more noise above 10 Hz.
Attached a proposed new controller that could be tested to see if DARM stationarity improves.
I uploaded the new controller in FM1 as a multiplicative change to the previous controller. So one can turn on this 2.6 Hz boost by engaging FM1 (5 second ramp) together with the other FMs while locked.
Tested on and off a couple of times while in NLN, the transition is smooth. The effect is shown in the attached spectrum:
All in all the DHARD_Y RMS is reduced to 0.64 times the value with the old controller.
I left this controller active in Observing and Cory accepted the SDF. We should take a look at DARM in the next few hours to see if 1) there is less non-stationary noise in the 20-50 Hz region 2) the higher DHARD_Y control noise doesn't affect us.
I have updated ISC_LOCK to engage this controller when in LOWNOISE_ASC. I did not reload ISC_LOCK code
Attached is the FM1 diff from Gabriele's change this afternoon.
Gabriele has also made a change for this FM1 change in ISC_LOCK, line 4490. Next time H1 is OUT of OBSERVING, please hit LOAD on ISC_LOCK.
ISC_LOCK has been re-loaded.
The new controller improved a lot the DHARD_Y motion at 1.3 Hz and 2.6 Hz, reducing the RMS.
It looks like CHARD_Y coudl use a similar improvement, since it still has a large 1 Hz and 2.6 Hz peaks. The 1 Hz peak is due to gain peaking in the current design, and the loop has little gain at 2.6 Hz
DARM still shows bicoherence with CHARD_Y
CO2X unlocking has kicked us out of observing 3 times in the last month 71213 each time it relocked within 1m30.
Each time this is seen, the average CO2X laser temperature has risen above 24.265degC. 1 month plot attached, look at yellow H1:TCS-ITMX_CO2_LASERTEMPERATURE channel. I seen no slow trend with the PZT but it does quickly fall down towards zero before unlock.
Solutions: We could take a average of temperature over 60 seconds (as this is +/-0.1deg) and if we are in ISC_LOCK DOWN state and if it's over this, have TCS_ITMX_CO2 relock. A better solution would be to tweak chiller setpoint slightly lower, I would do this at the start of Tuesday maintenance so we have time to see any immediate issues.
Other times we've gone out of observing in the past month is for the SQZ SHG to relock for 1m50s on 07/09 71595 and 1m45s on 07/11 for FC IR unlock 71597. Tagging GRD for Automation timers.
This morning I changed H1:TCS-ITMX_CO2_CHILLER_SET_POINT_OFFSET form 20.78 to 20.65degC, see attached. Hoping that his will keep the laser in a happier place.
The guardian does move this set point around when it cannot lock with the PZT alone, see attached, so I'm not sure how much this will help.
J. Kissel This is a continuation of building up a template suite for characterization of H1 SUS BS damping loops (most recently rekindled in LHO:71269). Got some more data on the beam splitter damping today. Got open loop gain TFs of the top mass M1 damping with M3 optical lever to M2 actuator damping loops on to see the impact (to make sure they weren't causing the top mass to go unstable), then got one DOF's worth of drive tuning (pitch) to measure the optical lever damping's open loop gain TFs. The data lives here: /ligo/svncommon/SusSVN/sus/trunk/BSFM/H1/BS/SAGM1/Data 2023-07-18_1740_H1SUSBS_M1_CDBIOState_1_OLDampingON_WhiteNoise_L_0p01to50Hz_OpenLoopGainTF.xml 2023-07-18_1740_H1SUSBS_M1_CDBIOState_1_OLDampingON_WhiteNoise_P_0p01to50Hz_OpenLoopGainTF.xml 2023-07-18_1740_H1SUSBS_M1_CDBIOState_1_OLDampingON_WhiteNoise_Y_0p01to50Hz_OpenLoopGainTF.xml /ligo/svncommon/SusSVN/sus/trunk/BSFM/H1/BS/Common/Data/ 2023-07-18_1805UTC_H1SUSBS_M2M3_OplevDamping_OpenLoopGainTF.xml The results, comments, etc. will be shared in the fullness of time; just have time for this "I measured the thing!" aLOG.
FYI, the optical lever damping loop open loop gain transfer functions were taken with the M2 (triple acquisition, or TACQ) coil driver set to state 2 (ACQ ON, LP OFF), and -- importantly -- the optical lever damping loop filter bank limiters, H1:SUS-BS_M2_OLDAMP_P_LIMIT and H1:SUS-BS_M2_OLDAMP_Y_LIMIT is OFF (it's nominally set at 99e3 [ct]).
Marissa, Camilla
We edited the filters in H1:OAF-RANGE_RBP_9 to be a bandpass for the 52Hz noise (70950, 70250) we've been seeing. Hopefully it will be easier to monitor this peak. Can be monitored as H1:OAF-RANGE_BAND_9 from 22:50UTC. Filters attached.
Accepted in SDF
See attached plot of this channel over last few days. Some of this 52Hz noise seems to be coupled with the wind speed.
The TRANSITION_BACK_TO_ETMX state of the SUS_CHARGE guardian has been causing the majority of our locklosses prior to Tuesday Maintenance, e.g. 69437, 70861.
I looked into this again and found that the ITMX LOCK_L gain is being turned off before the ITMX to ETMX transition has completed. The ramp time of the swap is 20 seconds but the code only waits 15 seconds before moving on, see attached plot and code.
We are using a ezca.get_LIGOFilter and the wait=True parameter isn't waiting the full ramp_time specified, created Issue 15. I've changed the wait to be False and added a 20s sleep timer. The SUS_CHARGE guardian will need to be reloaded when we are out of observing.
Once I remembered that SUS_CHARGE is not one of the guardian nodes monitored for the observation intent bit (although its subordinate nodes are), I reloaded it this morning at 14:02 UTC so Camilla's changes are pushed in.
This fixed the SUS_CHARGE code and we successfully stayed locked throughout both ESD transitions on Tuesday.
Total time taken is 18 minutes, this is longer than the 15 minute target. I've reduced the slowy_ramp_on_bias() ramptime back from 60 to 20 seconds, as I've reverted some of 68987 changes I was blaming for the looklosses. ESD_EXC_{QUAD} will need to be reloaded for each quad for this to take effect.
Currently the SUS_CHARGE code is taking 16m45s. We could look at further decreasing some tramps.
TJ noticed this morning that the L2L tramps for ITMX and ETMX weren't reverted by the code. I've added lines to SUS_CHARGE to save and revert these after the ESD transitions and also save adn reset the ITMX_L2L gain so it's not hard-coded.
WP 11331
Summary of harware configuration at EX:
Software was updated by P. Thomas and required a trip to EX to reconfigure the networking settings. Jim was able to bring back HEPI but reported oscillations on the pressure gauges.
Headed back to EX to troubleshoot. The Delta power supply was replaced with a Kepco power supply. The LVEA chassis was moved back to the pressure sensors to eliminate the DB9 extension cables. Both changes were done to save time as we were well beyond the maintenance period. Field chassis can be moved back to the half-rack at a later date. Jim reports oscillations are gone.
F. Clara, M. Pirello, P. Thomas, J. Warner