Alerted around 645 PT about initial alignment not completing. I found it with MICH not locking, but flashes seemed a bit off. I tried realigning via the camera and trying to maximize the flashes in AS_A. Even after getting a good spot on camera and having good flashes, it would diverge when tring to lock. MICH dark was the same. I tried to skip this state but SR2 alignment wouldn't even lock, and trying it by hand seemed that there was something very off.

Richard had told me that he set a node to INIT around 630PT, trending back it looked like in Input_Align. We've been having issues with this state recently, I guess that makes sense. I then tried this state and it locked without issue. I then tried PRC, no issues. Back to MICH, same issues as before. Tried skipping it with a well aligned spot on camera, and ran into the same issues with SR2 alignment.

I'm handing off to Corey with the suggestion to restore alignment to the last alignment Oli did and then trying an alignment since I was able to lock with this earlier.

TITLE: 04/03 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Aligning
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 11mph Gusts, 9mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.23 μm/s
QUICK SUMMARY:

H1 lost lock about 3.5hrs ago (1238utc), and TJ (owl shift-er) has been trying to help H1 with an alignment, BUT the Initial Alignment woes continue.  Strangely, Input Align appeared to work for him, but every other (atleast MichBright and SRC) were not completing for him. 

Will try an overall alignment revert to the alignment Oli/TJ had last night.  And from that will run an alignment to see if Initial Alignment steps still have issues.  (if they do, I'll skip the alignment and try locking with the reverted alignment.

Enviornmentally all is quiet after the windy and shaky night.

Oli handed off the IFO with the wind keeping ALS from locking, but they made it through an initial alignment earlier. Once the wind calmed down a bit ~30min later, it was able to keep ALS locked and then went right up without intervention. 

TITLE: 04/03 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Earthquake
INCOMING OPERATOR: TJ
SHIFT SUMMARY: Relocking from when I showed up went quickly, but after the earthquake knocked us out it's been impossible to lock due to the ground motion and the rising high winds :(
LOG:

23:30 Reached NOMINAL_LOW_NOISE    
23:41 Observing

00:11 Lockloss due to large earthquake from Taiwan (76897), holding in IDLE

Relocking
04:45 Untripped all watchdogs, attempting to lock green arms before starting an inital alignment
    - Green arms are super wiggly still from the earthquake/wind/both but I finally got them both
05:18 Started MANUAL_INITIAL_ALIGNMENT
    05:24 Skipped INPUT_ALIGN_OFFLOADED due to same issues everyone else has had recently (76870, 76843)
        - I couldn't even get it to ACQUIRE_XARM_IR
05:41 Initial alignment complete, relocking
    - Having trouble locking ALSY due to high wind at EY - you can see the spot swinging on camera :(. Flashes look great though!
05:56 Going to sit in DOWN for 10 mins and rest the quads - wind just broke past 40mph
06:02 Tried to take advantage of a slight drop in wind speeds, but didn't work
06:16 Went back to sitting in DOWN
06:33 Trying locking again
06:35 Lockloss during LOCKING_ALS
06:48 Back to DOWN after multiple locklosses during LOCKING_ARMS_GREEN and LOCKING_ALS                                                                                                                                                                                                                                                                                                                                                                                                         

Ran the data for the In Lock SUS Charge Measurements that ran this morning. Like Camilla said (76895), the excitations only worked for the ETMs, so although I am attaching all four plots, note that the last plot point for the ITMs is March 13th (ETMX, ETMY, ITMX, ITMY).

We've been sitting in IDLE since the earthquake rolled through three hours ago. Ground motion is still too high to lock.

Functionality tests were performed on the LVEA turbo stations. The details:

X-manifold:
Bearings: 100%
Turbo hours: 1950
Scroll hours: 1946
The recently installed separated water lines were also tested and found water-tight.

Y-manifold:
Bearings: 100%
Turbo hours: 951
Scroll hours: 2279

OMC:
Bearings: 100%
Turbo hours: 5964
Scroll hours: 5903

Lockloss at 04/03 00:11UTC from Taiwanese earthquake that Jamie experienced in person

Last week Artem, Louis and I copied the new DARM filter into ITMX 76685. This week we got through the ETMX to ITMX ESD DARM transition but lost lock 12 seconds into the 20 second ITMX to ETMX transition.  Plot attached. The EX signal looks much noisier than IX, I need to check all filters are correct.

Again, as seen last week 76637, although both ETM excitations ran fine, neither ITM excitation ran. It's the same guardian code and logs that it's sending the excitations but they are not seen on the H1:SUS-ITMY_L3_LOCK_BIAS_EXCMON and H1:SUS-ITMX_L3_DRIVEALIGN_L2L_EXCMON channels. Will get help from cds team.

J. Kissel
ECRs E1700387 and E2400116
IIETs 9392 and 30863
WPs 11743 and 11797

With Dave's help we installed all of the front-end model changes described in LHO:76850, and I've followed up behind the model changes with upgrades to the MEDM screens to show the new infrastructure -- then filled in the infrastructure with EPICs values, settings, and filters.

So, as of now,
    - As with the rest of the BSC Quads, Triples, Doubles, and HAM Triples, the the HAM doubles and single suspensions now are ll using a fully functional, sensible BLRMS system, calibrated into microns_RMS.
    - The thresholds have been generically set to 150 [um_RMS] for starters, since we have little history of understanding how the SUS behave in this 0.1 to 10 Hz band in a physically meaningful way. Please tune as you see fit if it's troublesome at this starting value.
    - The OMC ASC that's fed to the OMC SUS is now routed through the SUS-OMC_M1_LOCK_[LPY] bank, now has proper 2k to 16k AI filtering in all of those DOFs, and the LOCK_[LPY] output goes through the DRIVEALIGN matrix.
    - However, in an effort to make the least changes to the existing functionality, I've turned OFF the P2L and Y2L gains that were in place but not used for the time being. Will commission the drivealign gains again later, but for now the OSC ASC path should behave exactly as it had before, just with a proper 2k to 16kHz AI filter that should not impact the control stability.
    - The channels that are stored in the frames that record the history of the control signal have changed 
        from H1:SUS-OMC_M1_ASC_[LTVRPY]_IN1_DQ
        to H1:SUS-OMC_M1_LOCK_[LPY]_IN1_DQ
        which includes H1:SUS-OMC_M1_ASC_[PY]_IN1_DQ which were (for some reason) 
        in the GDS Broadcaster frame list, so they should now be mapped one-to-one to 
        H1:SUS-OMC_M1_LOCK_[PY]_IN1_DQ. 
      @DetChar -- Maybe these were used in the summary pages or something? 
    - All model, MEDM, and filter changes have been committed to the userapps repo.
    - All new EPICs settings have been save in every impacted SUS's SDF system.

More details to come tomorrow.

TITLE: 04/02 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: SEISMON_ALERT
    Wind: 29mph Gusts, 24mph 5min avg
    Primary useism: 0.06 μm/s
    Secondary useism: 0.20 μm/s
QUICK SUMMARY:

Currently relocking and in ENGAGE_ASC_FOR_FULL_IFO. Wind picking up and maxing out at 35mph

TITLE: 04/02 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
INCOMING OPERATOR: Oli
SHIFT SUMMARY:

• Maintenance Day was fairly busy, but straightforward and mostly ended around noon.
• Unfortunately, Input_Align issues continue:  While locking red-X-arm, IMC gets a kick and loses lock.  PLUS if the Xarm does lock & is held at WFS Centering, the x-arm alignment immediately gets misaligned by the INPUT_ALIGN state.  (This has been an issue over the last few weeks.)
• H1 at the hand-off is now past CARM Offset Reduction (woo woo!)----this was by SKIPPING Input_Align part of Initial Alignment.
• In hindsight, probably should not have spent ~3hrs trying to figure out Input Align.  At any rate, Oli got it to NLN at the beginning of their shift.
• Ryan S also helped me heaps during the shift!

LOG:

• 1420 note of magnetic injections
• 1458 Lockloss
• 1512-1533 pmc align (ryanS)
  • 1533 then did rotation stage calib (ryanS)
• 1506 FARO set up and work (tyler)
• 1509 cable tray work in FC tube enclosure for non-invasive work (ken)
• 1517 Transitioing LVEA to SAFE (camilla)
• 1521-1625 Acme pest control escort around site (chris)
• 1525-1854 Purge air at end stations (gerardo)
• 1529-1554 Technical Cleaning at FCES (kim, karen)
  • 1555 EX (kim)
  • 1555-1646 EY (karen)
  • 1841 completed LVEA cleaning before maintenance ended (karen.kim)
• 1538 H1LSC guardian node RELOAD (camilla)
• 1555-1847 3ifo inventory checks + FARO (jason)
• 1601-1703 Beckhoff work (fernando)
• 1601-1703 Turbo pump test in lvea (janos)
  • 1743-1854 Back out to start spin down turbo & power off (janos)
• 1603-1858 Flipping sign for ETMx for 3hrs (rahul)
• 1618-1746 Maggie, new LHO Science educator, getting lvea  tour (Mike)
• 1624-1801 PSL down for beckhoff work (patrick.ryanS)
• 1626 FAMIS checks (chris)
• 1626-1839 DAQ restart related to model changes (dave.jeff.erik) 
  • Erik will try out an ndscope start script for after DAQ restarts ("restart_all_ndscopes" from terminal prompt runs it & it looked OK and restores within about 4-5min)
• 1637-1740 End Stations FAMIS work (mitch)
• 1653-1930 OMC DCPD Transimpednance Amp response meas at HAM6 racks (francisco.louis)
  • Misalign PRM, BS, ITMs, SR2 ?
  • 1910 Jeff joining
• 1710-1804 EX accelerometer checks + EY rack temperature investigation (marc)
• 1728 HAM7 ISI watchdog trip
• 1748-1835 Connect BRSX picomotor to EX picomotor chassis (fil, jim)
• 1748 Dropping sqz stuff off in lvea (camilla, eric)
• 1756-1943 SHG work in optics lab (julian)
• ~1800 Input Pointing work (jennie, camilla)
• 1831 Investigating missing channel in CER (marc, fernando)
• 1913-1930 LVEA sweep by Camilla (with OMC dcpd group still out on floor)
• 1925 Initial Alignment
  • Attempt #1 (right after Maintenance):  input align was having trouble.  IMC losing lock with lots of MC2 saturations.  Then after about 10min red Xarm locked, but Input Align drives it into misalignment and then the IMC breaks lock.  This was with a few different types of IM4 pit/yaw.
  • Attempt #2 (Tony's reversion):  After getting no where, decided to revert to Tony's gps time from Friday (see his alog #76483).  And then restarted the alignment (x & y green arms needed to be touched up by hand); then guardian took over for the green arms.  Unfortunately, the first attempt had the IMC break lock (as the red X-arm was acquiring).
  • Attempt #3 (Reversion to before last lock from yesterday):  Looking at another time to revert to (at 2123utc on 4/1/24,  which was about 24hrs ago right before the last lock when ISCLOCK was in the READY state).
    • GPS Time:  1396041841  (restore all except sqz)
    • Similarly to Attempt #2, X& Y green arms had to be touched up by hand (even more off on this attempt comparted to Attempt-2).
    • Once again, INPUT ALIGN knocked the IMC out of lock when trying to lock the red-X-arm.
  • Attempt #4 (Using #3, but now skipping Input Align!):  So, after ALIGN IFO went to DOWN-----> SKIPPED Input_Align step and continued with alignment.
    • PRC Align, Mich Bright each completed fine.
    • SRC needed to be helped by hand via SRM (mostly pitch adjustment).
    • Attempted locking after this.
• Lock#1:  Y-arm had decent flashes, but would not stay locked, and would be taken to several rounds of INCREASE FLASHES.
  • Since Increase Flashes was not making progress (would barely get Y-arm above 0.8); went to Green Arms Manual and touched up alignment by hand to around 1.0, and FINALLY made it to Locking ALS, but then there was an immediate lockloss.
  • ...and...Winds are picking up into the 30mph range. 
• Lock#2:  No Increase flashes needed for Locking Arms Green.  Went immediately to PRMI & locked.  DRMI locked.  Made it passed Prep ASC
• 2102-2121 Resetting EX HVAC chiller (eric)
• 2220 PCal Lab work (francisco)
• 2242 FC Enclosure to drop off parts (jordan)

Jennie W, Camilla

We set up this measurement by mis-aligning ITMX, ITMY, BS, PRM and SRM during the maintenance period (after the PSL and IM suspensions came back up).

NB. PSL was at 2W but ISS was off.

Then we adjusted IM3 till we were centred in pitch and yaw on IM4 TRANS.

Then we searched each direction of IM1 and 2 that gave us clipping on IM4 TRAns - IE. THE nsum DROPPEED.

Then we went back to a point near clipping on IM 2 and mpved IM1 in that DOF to try and bring back the NSUM since we managed to do this for all DOFs (up and down in pitch and yaw)we assume no translation oft he beam improves NSUM so we are probably not clipping through OFI.

First image is starting sliders.

Second is sliders after we  aligned IM3.

This is ndscope trends throughout our tests. By far the largest improvement to IM4 Trans was moving IM3 and nothing we did with IM1 or 2 really improved this.

This morning I tweaked up the PSL PMC alignment remotely using the picomotor-controlled mirrors with the ISS off. I was only able to get about 60mW of power increase out of the PMC, but I decreased reflected power by about 200mW, so perhaps there's some mode-matching that could be optimized in the future.

After that, I recalibrated the rotation stage using the following process:

• Took IMC offline and paused LASER_PWR guardian
• Ran the script /opt/rtcds/userapps/release/psl/h1/scripts/RotationStage/MoveRotationStage.py to create PSL_rotStageCalib_2Apr2024.txt (attached)
• Changed PSLvals in CalibRotStage.m for correct file name then ran to create calibration curve (attached) and get new fit values (in table)
• Input new fit values into calibration and accepted new values in SDF (screenshot attached)
• Searched for HOME on rotation stage
• Resumed LASER_PWR guardian to go back to 2W input power

J. Oberling, R. Crouch, T. Guidry

Since the last update we have tried a couple of things.

First, we attempted to align the FARO to the BSC2 chamber directly.  We placed the FARO in the biergarten and shot the chamber-side door flanges.  We used the SMR to sweep along the outside of the +X side of the +Y door flange, and the +Y side of the +X door flange.  The shots were done as circles, which we could then place a point at circle center to represent the center of the door flange.  The hope was we could then project lines to represent the X and Y axes, and use that to align to our origin using a Plane/Axis/Center Point alignment routine.  I won't go into much detail, as it didn't work.  We used this alignment to look at PSI-6 in the biergarten, and it was off by over 1cm; we then looked at LV-35 and that one was off by even more.  I don't remember exact values as we didn't save screenshots since the devaiations were so large.  Let's say that if these deviations were true, I'm not sure how we would have a functioning, aligned, and alignable IFO.  So, that experiment didn't work, moving on.

Today we decided to try using height marks 901, 902, and 903 to set our Z axis alignment.  We looked at marks 901 and 902 with an autolevel and compared them to our BSC2 door flange scribe average we had used the water tube level to measure (alog 75974) and the listed local Z coordinate from T1100187.  Results:

• 901:
  • T1100187: -54.7 mm
  • Water level: -55.2 mm
  • Difference: -0.5 mm
• 902:
  • T1100187: -62.7 mm
  • Water level: -62.9 mm
  • Difference: -0.2 mm

Not bad, and definitely not the almost +4mm the FARO was measuring for these marks w.r.t. BTVE-1.  If we use our differential height survey from alog 75771 we can also calculate the local Z axis coordinate for mark 903, which comes out to -79.8 mm (versus the listed coordinate of -79.9 mm from T1100187), and again not the almost +4mm reported by the FARO when measured w.r.t. BTVE-1.  It's looking more and more like there was some error in the Z axis position of BTVE-1; whether that error was in setting the monument itself or in setting BSC2 w.r.t. to BTVE-1 we cannot say.  The fact of the matter is that when compared to BSC2 as it sits right now based on our water level survey, the height marks we've looked at are pretty close to their listed coordinates and BTVE-1 is definitely not.  We also happened to do the same differential height survey with BTVE-1 (also in alog 75771), so we can calculate a local Z coordinate for BTVE-1 based on our height mark 903 (which we have now tied to BSC2 Z=0).  Doing so gives a local Z axis coordinate for BTVE-1 of -1060.3 mm (Z₉₀₃ - deltaZ_BTVE-1/903 = -79.8 - 980.5); this becomes -1060.9 mm once we convert to our global coordinate frame (which is decidedly not the -1057.2 mm all of the old documentation lists it as).

Now that we have local Z coordinates for 901, 902, 903, and BTVE-1 that have all been tied to BSC2 Z=0, we used these to set a new alignment.  First, we need global Z coordiantes for our 3 height marks.  We got these by using a autolevel to set a magnetic nest inline with the height mark to be measured.  The FARO was put into an intial alignment using BTVE-1, PSI-1, PSI-2, and PSI-6 (X and Y are generally pretty good, Z is off but we don't care yet), and we then placed the SMR on the nest we placed inline with 903.  From this we got an X/Y coordinate for the nest that we could use to calculate our global Z.  Once this was done we added 903 into the alignment routine, and removed PSI-1, PSI-2, and PSI-6 for the Z axis fit.  we then moved the FARO to a position where we could see 901 and 902 and shot them in the same way we did 903.  Ultimately, we ended up with an alignment using BTVE-1, PSI-1, PSI-2, and PSI-6 for X and Y and 901, 902. and 903 for Z.  The coordinates used for the height marks were (format is [X,Y,Z]):

• 901: [1075.8, 12366.1, -55.7] mm
• 902: [1078.9, 24938.8, -63.3] mm
• 903: [1082.6, 37455.6, -80.0] mm

Tyler has a screenshot of the results of the alignment that he'll post as a comment to this alog.  One thing to note here: We were not using the new BTVE-1 Z as part of the Z axis alignment, only marks 901, 902, and 903.  It's somewhat comforting to see the Z axis coordinate of BTVE-1 as calculated by the alignment routine as close as it is to what we think the nominal shoud be based on our water level survey of BSC2.  The PSI monuments are all out, especially PSI-2.  I'm not sure how concerning this is, as we don't trust the Z coordinates of the PSI monuments anyway.

One caveat here, all of the height marks we used are almost in a line down the Y axis, with very little deviation in the X axis coordinate (a deltaX of less than 7mm across ~25m of the Y axis).  I'm not entirely confident we're picking up the global tilt of our X axis with this configuration.  We want to take this alignment and test and refine it; first thing we can do is get some more height marks spread along the X axis to hopefully catch the global X axis tilt.  More to come!

The BRSX remote mass adjust pico motor was finally connected to the EX pico controller this morning. This was done with the pigtail D2400020 that Fil made, to allow the BRS to use channel 5 on the controller. Channel 5 is then sent to a DB9 that goes into the box, and connects to a DB9 soldered to the pins on a feedthru on BRSX vacuum vessel. 

I tested that we are able to run the motor from the MEDM, and made an adjustment to the mass on the BRSX to try to get us better centered. I can now recenter the BRS from my office, nice.

The pico controller screen is accessed through LSC->Picmotors->End X (X PICO A) screen. To recenter push "Enabled", push the  wait for the err codes to clear, set the step size and speed. "Sprint" is fine for the speed, for the BRS at least. The motor is, I think, 10k steps per revolution, when UW was here, we used 1250 steps per move. Today, I went +2250 steps, saw that was the wrong direction, then -5000 steps from and then a final +1000 steps to disengage the mass adjuster drive yoke. The net move was -1750 steps. This is recorded in epics as H1:SYS-MOTION_X_PICO_A_CURRENT_X_POSITION, but that seems to depend on the channels selected on the chassis.

 The buttons on the pico motor screen are kind of confusing " < " makes positive steps, and " > " is negative. This moved the BRSX driftmon channel about -20k counts, and it just barely on the bottom edge of it's readout range. I've turned the voltage on the heater down to bring it back to center, but that will take a day or two to settle out, so I've taken the BRS out of loop for now.

Summary (highlights) of findings during the DQ Shift

-Engineering run with ongoing commissioning
-Numerous lock-losses due to earthquakes/seismic activity
-BNS range averages between 145 and 150 Mpc. Sometimes there are fluctuations to 130 Mpc.
-Glitchgrams show a reoccuring glitch around 40 Hz that occurs a few times a day.
-The h(t) spectrogram showed sporatatic increases of amplitude relative to the median in low-frequency range. Anomalies can be associated with drops in BNS range.
-Fscans are unavailable at this time.
-A few pyCBC triggers with new SNR > 10 (short) occured 2 out of 7 days.
-PEM, ASC and LSC type channels showed often as round winners in Hveto associated with low-frequency noise around 40 Hz and 50 Hz respectively.
-New channels of interest include IMC and HPI. Channel activity can be linked to sustained winds. Also could be round winners because of shorter observation runs.



DQ Shift Report: 

https://wiki.ligo.org/DetChar/DataQuality/DQShiftLHO20240318

WP 11799.

Patrick T., Ryan S.

The PSL Beckhoff PLC has been updated to match the changes made at LLO to add a watchdog channel. This is the code from 'TwinCAT Projekt_LIGO_240124.zip' in https://dcc.ligo.org/E2100235. The now running version is in git at https://git.ligo.org/cds/ifo/beckhoff/lho-psl and commit 9b6ad86cb26660b136242b334bd4831a683f1c5b. It appears there was also a change to fix a bug where toggling the noise eater tripped the NPRO. Ryan verified that this now appears to be fixed. I also removed the ${IFO}.PSL.LASER.MAIN.SHUTTER alias of the bEnable_Shutter2 variable, which according to LLO is not being used. The channel was not being exported to EPICS.

There were two minor issues. The first was that the version of a couple of the terminals found from a scan of the EtherCAT bus did not match what was in the IO configuration, and this appeared to be the cause of an INIT error for a couple of terminals (presumably those) when the system was put into run mode. This was fixed by changing the IO configured versions to match the scanned versions. The other was that I forgot to comment out the inversion of the signal from laser PD1, which was not realized until we started everything and were wondering why the power was reading as negative for that PD. I fixed that in a new commit and restarted the code again.

FAMIS 20022

The main chiller alarm was active last night, so this morning I added 100mL of water to stope the low-level alarm.

We plan on doing a remote PMC alignment tomorrow during maintenance to address the recent rise in PMC reflected power.