J. Kissel
ECRs E1700387 and E2400116
IIETs 9392 and 30863
WPs 11743 and 11797

With Dave's help we installed all of the front-end model changes described in LHO:76850, and I've followed up behind the model changes with upgrades to the MEDM screens to show the new infrastructure -- then filled in the infrastructure with EPICs values, settings, and filters.

So, as of now,
    - As with the rest of the BSC Quads, Triples, Doubles, and HAM Triples, the the HAM doubles and single suspensions now are ll using a fully functional, sensible BLRMS system, calibrated into microns_RMS.
    - The thresholds have been generically set to 150 [um_RMS] for starters, since we have little history of understanding how the SUS behave in this 0.1 to 10 Hz band in a physically meaningful way. Please tune as you see fit if it's troublesome at this starting value.
    - The OMC ASC that's fed to the OMC SUS is now routed through the SUS-OMC_M1_LOCK_[LPY] bank, now has proper 2k to 16k AI filtering in all of those DOFs, and the LOCK_[LPY] output goes through the DRIVEALIGN matrix.
    - However, in an effort to make the least changes to the existing functionality, I've turned OFF the P2L and Y2L gains that were in place but not used for the time being. Will commission the drivealign gains again later, but for now the OSC ASC path should behave exactly as it had before, just with a proper 2k to 16kHz AI filter that should not impact the control stability.
    - The channels that are stored in the frames that record the history of the control signal have changed 
        from H1:SUS-OMC_M1_ASC_[LTVRPY]_IN1_DQ
        to H1:SUS-OMC_M1_LOCK_[LPY]_IN1_DQ
        which includes H1:SUS-OMC_M1_ASC_[PY]_IN1_DQ which were (for some reason) 
        in the GDS Broadcaster frame list, so they should now be mapped one-to-one to 
        H1:SUS-OMC_M1_LOCK_[PY]_IN1_DQ. 
      @DetChar -- Maybe these were used in the summary pages or something? 
    - All model, MEDM, and filter changes have been committed to the userapps repo.
    - All new EPICs settings have been save in every impacted SUS's SDF system.

More details to come tomorrow.

TITLE: 04/02 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: SEISMON_ALERT
    Wind: 29mph Gusts, 24mph 5min avg
    Primary useism: 0.06 μm/s
    Secondary useism: 0.20 μm/s
QUICK SUMMARY:

Currently relocking and in ENGAGE_ASC_FOR_FULL_IFO. Wind picking up and maxing out at 35mph

TITLE: 04/02 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
INCOMING OPERATOR: Oli
SHIFT SUMMARY:

• Maintenance Day was fairly busy, but straightforward and mostly ended around noon.
• Unfortunately, Input_Align issues continue:  While locking red-X-arm, IMC gets a kick and loses lock.  PLUS if the Xarm does lock & is held at WFS Centering, the x-arm alignment immediately gets misaligned by the INPUT_ALIGN state.  (This has been an issue over the last few weeks.)
• H1 at the hand-off is now past CARM Offset Reduction (woo woo!)----this was by SKIPPING Input_Align part of Initial Alignment.
• In hindsight, probably should not have spent ~3hrs trying to figure out Input Align.  At any rate, Oli got it to NLN at the beginning of their shift.
• Ryan S also helped me heaps during the shift!

LOG:

• 1420 note of magnetic injections
• 1458 Lockloss
• 1512-1533 pmc align (ryanS)
  • 1533 then did rotation stage calib (ryanS)
• 1506 FARO set up and work (tyler)
• 1509 cable tray work in FC tube enclosure for non-invasive work (ken)
• 1517 Transitioing LVEA to SAFE (camilla)
• 1521-1625 Acme pest control escort around site (chris)
• 1525-1854 Purge air at end stations (gerardo)
• 1529-1554 Technical Cleaning at FCES (kim, karen)
  • 1555 EX (kim)
  • 1555-1646 EY (karen)
  • 1841 completed LVEA cleaning before maintenance ended (karen.kim)
• 1538 H1LSC guardian node RELOAD (camilla)
• 1555-1847 3ifo inventory checks + FARO (jason)
• 1601-1703 Beckhoff work (fernando)
• 1601-1703 Turbo pump test in lvea (janos)
  • 1743-1854 Back out to start spin down turbo & power off (janos)
• 1603-1858 Flipping sign for ETMx for 3hrs (rahul)
• 1618-1746 Maggie, new LHO Science educator, getting lvea  tour (Mike)
• 1624-1801 PSL down for beckhoff work (patrick.ryanS)
• 1626 FAMIS checks (chris)
• 1626-1839 DAQ restart related to model changes (dave.jeff.erik) 
  • Erik will try out an ndscope start script for after DAQ restarts ("restart_all_ndscopes" from terminal prompt runs it & it looked OK and restores within about 4-5min)
• 1637-1740 End Stations FAMIS work (mitch)
• 1653-1930 OMC DCPD Transimpednance Amp response meas at HAM6 racks (francisco.louis)
  • Misalign PRM, BS, ITMs, SR2 ?
  • 1910 Jeff joining
• 1710-1804 EX accelerometer checks + EY rack temperature investigation (marc)
• 1728 HAM7 ISI watchdog trip
• 1748-1835 Connect BRSX picomotor to EX picomotor chassis (fil, jim)
• 1748 Dropping sqz stuff off in lvea (camilla, eric)
• 1756-1943 SHG work in optics lab (julian)
• ~1800 Input Pointing work (jennie, camilla)
• 1831 Investigating missing channel in CER (marc, fernando)
• 1913-1930 LVEA sweep by Camilla (with OMC dcpd group still out on floor)
• 1925 Initial Alignment
  • Attempt #1 (right after Maintenance):  input align was having trouble.  IMC losing lock with lots of MC2 saturations.  Then after about 10min red Xarm locked, but Input Align drives it into misalignment and then the IMC breaks lock.  This was with a few different types of IM4 pit/yaw.
  • Attempt #2 (Tony's reversion):  After getting no where, decided to revert to Tony's gps time from Friday (see his alog #76483).  And then restarted the alignment (x & y green arms needed to be touched up by hand); then guardian took over for the green arms.  Unfortunately, the first attempt had the IMC break lock (as the red X-arm was acquiring).
  • Attempt #3 (Reversion to before last lock from yesterday):  Looking at another time to revert to (at 2123utc on 4/1/24,  which was about 24hrs ago right before the last lock when ISCLOCK was in the READY state).
    • GPS Time:  1396041841  (restore all except sqz)
    • Similarly to Attempt #2, X& Y green arms had to be touched up by hand (even more off on this attempt comparted to Attempt-2).
    • Once again, INPUT ALIGN knocked the IMC out of lock when trying to lock the red-X-arm.
  • Attempt #4 (Using #3, but now skipping Input Align!):  So, after ALIGN IFO went to DOWN-----> SKIPPED Input_Align step and continued with alignment.
    • PRC Align, Mich Bright each completed fine.
    • SRC needed to be helped by hand via SRM (mostly pitch adjustment).
    • Attempted locking after this.
• Lock#1:  Y-arm had decent flashes, but would not stay locked, and would be taken to several rounds of INCREASE FLASHES.
  • Since Increase Flashes was not making progress (would barely get Y-arm above 0.8); went to Green Arms Manual and touched up alignment by hand to around 1.0, and FINALLY made it to Locking ALS, but then there was an immediate lockloss.
  • ...and...Winds are picking up into the 30mph range. 
• Lock#2:  No Increase flashes needed for Locking Arms Green.  Went immediately to PRMI & locked.  DRMI locked.  Made it passed Prep ASC
• 2102-2121 Resetting EX HVAC chiller (eric)
• 2220 PCal Lab work (francisco)
• 2242 FC Enclosure to drop off parts (jordan)

Jennie W, Camilla

We set up this measurement by mis-aligning ITMX, ITMY, BS, PRM and SRM during the maintenance period (after the PSL and IM suspensions came back up).

NB. PSL was at 2W but ISS was off.

Then we adjusted IM3 till we were centred in pitch and yaw on IM4 TRANS.

Then we searched each direction of IM1 and 2 that gave us clipping on IM4 TRAns - IE. THE nsum DROPPEED.

Then we went back to a point near clipping on IM 2 and mpved IM1 in that DOF to try and bring back the NSUM since we managed to do this for all DOFs (up and down in pitch and yaw)we assume no translation oft he beam improves NSUM so we are probably not clipping through OFI.

First image is starting sliders.

Second is sliders after we  aligned IM3.

This is ndscope trends throughout our tests. By far the largest improvement to IM4 Trans was moving IM3 and nothing we did with IM1 or 2 really improved this.

This morning I tweaked up the PSL PMC alignment remotely using the picomotor-controlled mirrors with the ISS off. I was only able to get about 60mW of power increase out of the PMC, but I decreased reflected power by about 200mW, so perhaps there's some mode-matching that could be optimized in the future.

After that, I recalibrated the rotation stage using the following process:

• Took IMC offline and paused LASER_PWR guardian
• Ran the script /opt/rtcds/userapps/release/psl/h1/scripts/RotationStage/MoveRotationStage.py to create PSL_rotStageCalib_2Apr2024.txt (attached)
• Changed PSLvals in CalibRotStage.m for correct file name then ran to create calibration curve (attached) and get new fit values (in table)
• Input new fit values into calibration and accepted new values in SDF (screenshot attached)
• Searched for HOME on rotation stage
• Resumed LASER_PWR guardian to go back to 2W input power

J. Oberling, R. Crouch, T. Guidry

Since the last update we have tried a couple of things.

First, we attempted to align the FARO to the BSC2 chamber directly.  We placed the FARO in the biergarten and shot the chamber-side door flanges.  We used the SMR to sweep along the outside of the +X side of the +Y door flange, and the +Y side of the +X door flange.  The shots were done as circles, which we could then place a point at circle center to represent the center of the door flange.  The hope was we could then project lines to represent the X and Y axes, and use that to align to our origin using a Plane/Axis/Center Point alignment routine.  I won't go into much detail, as it didn't work.  We used this alignment to look at PSI-6 in the biergarten, and it was off by over 1cm; we then looked at LV-35 and that one was off by even more.  I don't remember exact values as we didn't save screenshots since the devaiations were so large.  Let's say that if these deviations were true, I'm not sure how we would have a functioning, aligned, and alignable IFO.  So, that experiment didn't work, moving on.

Today we decided to try using height marks 901, 902, and 903 to set our Z axis alignment.  We looked at marks 901 and 902 with an autolevel and compared them to our BSC2 door flange scribe average we had used the water tube level to measure (alog 75974) and the listed local Z coordinate from T1100187.  Results:

• 901:
  • T1100187: -54.7 mm
  • Water level: -55.2 mm
  • Difference: -0.5 mm
• 902:
  • T1100187: -62.7 mm
  • Water level: -62.9 mm
  • Difference: -0.2 mm

Not bad, and definitely not the almost +4mm the FARO was measuring for these marks w.r.t. BTVE-1.  If we use our differential height survey from alog 75771 we can also calculate the local Z axis coordinate for mark 903, which comes out to -79.8 mm (versus the listed coordinate of -79.9 mm from T1100187), and again not the almost +4mm reported by the FARO when measured w.r.t. BTVE-1.  It's looking more and more like there was some error in the Z axis position of BTVE-1; whether that error was in setting the monument itself or in setting BSC2 w.r.t. to BTVE-1 we cannot say.  The fact of the matter is that when compared to BSC2 as it sits right now based on our water level survey, the height marks we've looked at are pretty close to their listed coordinates and BTVE-1 is definitely not.  We also happened to do the same differential height survey with BTVE-1 (also in alog 75771), so we can calculate a local Z coordinate for BTVE-1 based on our height mark 903 (which we have now tied to BSC2 Z=0).  Doing so gives a local Z axis coordinate for BTVE-1 of -1060.3 mm (Z₉₀₃ - deltaZ_BTVE-1/903 = -79.8 - 980.5); this becomes -1060.9 mm once we convert to our global coordinate frame (which is decidedly not the -1057.2 mm all of the old documentation lists it as).

Now that we have local Z coordinates for 901, 902, 903, and BTVE-1 that have all been tied to BSC2 Z=0, we used these to set a new alignment.  First, we need global Z coordiantes for our 3 height marks.  We got these by using a autolevel to set a magnetic nest inline with the height mark to be measured.  The FARO was put into an intial alignment using BTVE-1, PSI-1, PSI-2, and PSI-6 (X and Y are generally pretty good, Z is off but we don't care yet), and we then placed the SMR on the nest we placed inline with 903.  From this we got an X/Y coordinate for the nest that we could use to calculate our global Z.  Once this was done we added 903 into the alignment routine, and removed PSI-1, PSI-2, and PSI-6 for the Z axis fit.  we then moved the FARO to a position where we could see 901 and 902 and shot them in the same way we did 903.  Ultimately, we ended up with an alignment using BTVE-1, PSI-1, PSI-2, and PSI-6 for X and Y and 901, 902. and 903 for Z.  The coordinates used for the height marks were (format is [X,Y,Z]):

• 901: [1075.8, 12366.1, -55.7] mm
• 902: [1078.9, 24938.8, -63.3] mm
• 903: [1082.6, 37455.6, -80.0] mm

Tyler has a screenshot of the results of the alignment that he'll post as a comment to this alog.  One thing to note here: We were not using the new BTVE-1 Z as part of the Z axis alignment, only marks 901, 902, and 903.  It's somewhat comforting to see the Z axis coordinate of BTVE-1 as calculated by the alignment routine as close as it is to what we think the nominal shoud be based on our water level survey of BSC2.  The PSI monuments are all out, especially PSI-2.  I'm not sure how concerning this is, as we don't trust the Z coordinates of the PSI monuments anyway.

One caveat here, all of the height marks we used are almost in a line down the Y axis, with very little deviation in the X axis coordinate (a deltaX of less than 7mm across ~25m of the Y axis).  I'm not entirely confident we're picking up the global tilt of our X axis with this configuration.  We want to take this alignment and test and refine it; first thing we can do is get some more height marks spread along the X axis to hopefully catch the global X axis tilt.  More to come!

The BRSX remote mass adjust pico motor was finally connected to the EX pico controller this morning. This was done with the pigtail D2400020 that Fil made, to allow the BRS to use channel 5 on the controller. Channel 5 is then sent to a DB9 that goes into the box, and connects to a DB9 soldered to the pins on a feedthru on BRSX vacuum vessel. 

I tested that we are able to run the motor from the MEDM, and made an adjustment to the mass on the BRSX to try to get us better centered. I can now recenter the BRS from my office, nice.

The pico controller screen is accessed through LSC->Picmotors->End X (X PICO A) screen. To recenter push "Enabled", push the  wait for the err codes to clear, set the step size and speed. "Sprint" is fine for the speed, for the BRS at least. The motor is, I think, 10k steps per revolution, when UW was here, we used 1250 steps per move. Today, I went +2250 steps, saw that was the wrong direction, then -5000 steps from and then a final +1000 steps to disengage the mass adjuster drive yoke. The net move was -1750 steps. This is recorded in epics as H1:SYS-MOTION_X_PICO_A_CURRENT_X_POSITION, but that seems to depend on the channels selected on the chassis.

 The buttons on the pico motor screen are kind of confusing " < " makes positive steps, and " > " is negative. This moved the BRSX driftmon channel about -20k counts, and it just barely on the bottom edge of it's readout range. I've turned the voltage on the heater down to bring it back to center, but that will take a day or two to settle out, so I've taken the BRS out of loop for now.

Summary (highlights) of findings during the DQ Shift

-Engineering run with ongoing commissioning
-Numerous lock-losses due to earthquakes/seismic activity
-BNS range averages between 145 and 150 Mpc. Sometimes there are fluctuations to 130 Mpc.
-Glitchgrams show a reoccuring glitch around 40 Hz that occurs a few times a day.
-The h(t) spectrogram showed sporatatic increases of amplitude relative to the median in low-frequency range. Anomalies can be associated with drops in BNS range.
-Fscans are unavailable at this time.
-A few pyCBC triggers with new SNR > 10 (short) occured 2 out of 7 days.
-PEM, ASC and LSC type channels showed often as round winners in Hveto associated with low-frequency noise around 40 Hz and 50 Hz respectively.
-New channels of interest include IMC and HPI. Channel activity can be linked to sustained winds. Also could be round winners because of shorter observation runs.



DQ Shift Report: 

https://wiki.ligo.org/DetChar/DataQuality/DQShiftLHO20240318

Attachment 1 - Unsqueezed (left) vs. squeezed (right) DARMs in ER16, with respect to O4a. Like Gabriele's plots, the ratios on the bottom show a given trace / O4a reference, so ratio > 1 means more noise now than O4a.

• (left) Last night's no-sqz time 76872 looks quite similar to O4a around 100 Hz, so maybe any excess non-quantum noises around 100 Hz are improving?
   
• (right) Squeezing does not look super optimal right now - definitely would be interesting to do more optimization.. It both looks highly frequency-dependent, and SQZ possibly even lossier now than earlier in ER16, though this is very difficult to say. Recently we've been optimizing SQZ around 350 Hz for range, so it's hard to evaluate high-freq SQZ, since it is misrotated due to the strong frequency-dependence. But even recent ASQZ efficiency could use another look. Maybe we could check OPO temperature, try re-running alignment scripts, etc. General SQZ efficiency is worth some investigations, alongside optimizing PSAMS or SRCL.

Colors: Rainbow colors are ordered past to present: e.g. Red = O4a, ... Purple = now.

Attachment 2 - SQZ dBs after subtracting non-quantum noise, with various recent PSAMS settings. Upper subplot: Anti-SQZ, Lower subplot: SQZ. Lines are a moving average of the data. Some caveats for the subtraction: here I am using the same H1 unsqueezed quantum noise model for all subtractions; this may not be the most accurate, but I think it should be fairly close.

Reference alogs wth times that I used to grab GDS_STRAIN_CLEAN:

• O4a references - 74935 and Gabriele's times from 76825
• 76434 - PSAMS 120/120, March 17
• 76540 - PSAMS 150/90, March 20
        --> corresponding SQZ data set saw ~10-15% higher frequency-dependent losses at 100 Hz vs. kHz unlike earlier Mar16, possibly due to PSAMS settings following 76507
• 76777 - PSAMS 100/100, March 29
• 76798 - PSAMS 125/125, March 29  low-freq glitch in this short SQZ time, difficult to compare. Expect it is similar to 120/120 data?
• 76825 - PSAMS 100/100, March 30  - Gabriele showing total DARM at 100 Hz similar to O4a
• 76872 - PSAMS 100/100, April 2 - opportunistic NO SQZ from last night.

This could be a bad readback, or real, we are not sure.  The signal should be 23dB, but the readback is showing 16.8dB on NDSCOPE and has been trending down for a while, it is throwing an error flag in beckhoff.  This signal is located in ISC-R1 slot U16 on the left side.  The LO signal is 9MHz and is fed through a delay line chassis, there are also baluns, cabling, etc...  We were not able to determine the source of the readback error before noon, so this will have to wait until a maintenance window.

M. Pirello, F. Mera, D. Sigg

This morning I flipped the sign (after talking to the Operator) of the ETMX ESD lock bias voltage from -450V to +450V for 3hrs to reduce the charge build up in the optic.

After 3hrs I reversed the sign and set it to nominal.

ndscope trends are attached below.

Per WP11795 I looked at the EX Accelerometer Power Conditioner based on what Robert said about one of the channels not functioning.  Visually, there is no power on channel 8, the power good light is off.  We moved the signal to channel 9 and will go back to fix channel 8 when time permits.  This channel was working when the chassis was installed.

Lights off, paging system unplugged, WAP stayed off. Some unused extension cords unplugged, genie lift charging block unplugged. Turned off SQZ laptop and monitor. 

All else looked good, followed T1500386

WP 11799.

Patrick T., Ryan S.

The PSL Beckhoff PLC has been updated to match the changes made at LLO to add a watchdog channel. This is the code from 'TwinCAT Projekt_LIGO_240124.zip' in https://dcc.ligo.org/E2100235. The now running version is in git at https://git.ligo.org/cds/ifo/beckhoff/lho-psl and commit 9b6ad86cb26660b136242b334bd4831a683f1c5b. It appears there was also a change to fix a bug where toggling the noise eater tripped the NPRO. Ryan verified that this now appears to be fixed. I also removed the ${IFO}.PSL.LASER.MAIN.SHUTTER alias of the bEnable_Shutter2 variable, which according to LLO is not being used. The channel was not being exported to EPICS.

There were two minor issues. The first was that the version of a couple of the terminals found from a scan of the EtherCAT bus did not match what was in the IO configuration, and this appeared to be the cause of an INIT error for a couple of terminals (presumably those) when the system was put into run mode. This was fixed by changing the IO configured versions to match the scanned versions. The other was that I forgot to comment out the inversion of the signal from laser PD1, which was not realized until we started everything and were wondering why the power was reading as negative for that PD. I fixed that in a new commit and restarted the code again.

Tue Apr 02 10:06:37 2024 INFO: Fill completed in 6min 34secs

Jordan confirmed a good fill curbside

Picket Fence was updated.  The only major change is that one channel "HLID" was dropped for another, "HWUT".

Camilla, Vicky

Screenshot of how we found the SQZ_FC and SQZ_OPO this AM. Two separate issues.

Only to-do might be fixing the SQZ_FC issue in FIND_IR. I hope the SQZ_OPO_LR issue is solved, but we will watch out for it.

More description of the issues from this morning/last night and how to fix it:

• SQZ_FC -  same stuck in FIND_IR issue as yesterday 76862. See the execution time over-run circled in today's guardian screenshot. 
   
  • To fix: brought SQZ_FC to "DOWN". That was it, SQZ_MANAGER took over the rest.
     
  • To-do: Unsure why we have gotten multiple timeouts in the past 2 days, try to resolve this. 
    • Maybe in SQZ_FC, on Line 112, we need to wrap the cdu.getdata() function call in TJ's timeout checker function. 
    • In the meantime, I have added a log statement to Line 111 just before cdu.getata() is called, and reloaded SQZ_FC. So if FIND_IR gets stuck again, we can confirm whether it is really this function call that times out the guardian.
       
• SQZ_OPO_LR - did not scan to high enough PZT voltage to find co-resonance. Hopefully we have now increased the scan range enough to fix this for O4b. Steps taken today:

  1. From it being STALLED and MANAGED, I took SQZ_OPO_LR to AUTO. This worked to un-stall it, and let it continue locking. If the scan is fixed, this should be enough to solve the problem.
      --> Not sure why OPO guardian was "STALLED" when it jumped to "IDLE" state, but I think this is related to the guardian "jump" to IDLE. I'm not sure if we even want to fix this -- leaving the SQZ_OPO_LR guardian in STALLED was probably the correct move, otherwise it would have spent hours scanning the PZT without finding co-resonance. Better to fix issues with not finding co-resonance, maybe not much to do here.

  2. OPO couldn't lock on co-resonance b/c that was at 104V, but PZT1 was only scanning up until 100V -- I changed the scan offset.

     • OPO did not find co-resonance within its scan range, then it timed out, jumped into IDLE, and the guardian STALLED. See sqz locking screenshot -- increasing the scan range let it find co-resonance at 104V, and then SQZ_MANAGER easily went back up to FDS.

     • I increased H1:SQZ-OPO_PZT_1_OFFSET from 0V to 15V, and saved this 15V offset in SDF. We cannot increase H1:SQZ-OPO_PZT_1_SCAN_STOP above 100V in slowcontrols, so we have to increase the scan offset to get the PZT there.

       • Note, back when OPO_PZT2 was at 0V (before 76688), I think the OPO would often find co-resonance between 100-110 V during O4a. 

     • Based on a previous opo co-resonance issue, 76642 seemed to want to the scan starting around ~50V. Today's issue shows we want to scan beyond 105V, probably up to 110V or even 115V (the max). Based on OPO cavity scans (e.g. screenshot from LHO:75285), I think OPO PZT1 is not super effective (in terms of um/V scanning) below 50V or so.

     • Let's try scanning from 55V - 115V to guarentee we find co-resonance. This is how it's currently set. This corresponds to the first screenshot, where H1:SQZ-OPO_PZT_1_OFFSET = 15V (saved in SDF), H1:SQZ-OPO_PZT_1_SCAN_START = 40, H1:SQZ-OPO_PZT_1_SCAN_START = 100V.

FAMIS 20022

The main chiller alarm was active last night, so this morning I added 100mL of water to stope the low-level alarm.

We plan on doing a remote PMC alignment tomorrow during maintenance to address the recent rise in PMC reflected power.