This could be a bad readback, or real, we are not sure.  The signal should be 23dB, but the readback is showing 16.8dB on NDSCOPE and has been trending down for a while, it is throwing an error flag in beckhoff.  This signal is located in ISC-R1 slot U16 on the left side.  The LO signal is 9MHz and is fed through a delay line chassis, there are also baluns, cabling, etc...  We were not able to determine the source of the readback error before noon, so this will have to wait until a maintenance window.

M. Pirello, F. Mera, D. Sigg

This morning I flipped the sign (after talking to the Operator) of the ETMX ESD lock bias voltage from -450V to +450V for 3hrs to reduce the charge build up in the optic.

After 3hrs I reversed the sign and set it to nominal.

ndscope trends are attached below.

Per WP11795 I looked at the EX Accelerometer Power Conditioner based on what Robert said about one of the channels not functioning.  Visually, there is no power on channel 8, the power good light is off.  We moved the signal to channel 9 and will go back to fix channel 8 when time permits.  This channel was working when the chassis was installed.

Lights off, paging system unplugged, WAP stayed off. Some unused extension cords unplugged, genie lift charging block unplugged. Turned off SQZ laptop and monitor. 

All else looked good, followed T1500386

WP 11799.

Patrick T., Ryan S.

The PSL Beckhoff PLC has been updated to match the changes made at LLO to add a watchdog channel. This is the code from 'TwinCAT Projekt_LIGO_240124.zip' in https://dcc.ligo.org/E2100235. The now running version is in git at https://git.ligo.org/cds/ifo/beckhoff/lho-psl and commit 9b6ad86cb26660b136242b334bd4831a683f1c5b. It appears there was also a change to fix a bug where toggling the noise eater tripped the NPRO. Ryan verified that this now appears to be fixed. I also removed the ${IFO}.PSL.LASER.MAIN.SHUTTER alias of the bEnable_Shutter2 variable, which according to LLO is not being used. The channel was not being exported to EPICS.

There were two minor issues. The first was that the version of a couple of the terminals found from a scan of the EtherCAT bus did not match what was in the IO configuration, and this appeared to be the cause of an INIT error for a couple of terminals (presumably those) when the system was put into run mode. This was fixed by changing the IO configured versions to match the scanned versions. The other was that I forgot to comment out the inversion of the signal from laser PD1, which was not realized until we started everything and were wondering why the power was reading as negative for that PD. I fixed that in a new commit and restarted the code again.

Tue Apr 02 10:06:37 2024 INFO: Fill completed in 6min 34secs

Jordan confirmed a good fill curbside

Picket Fence was updated.  The only major change is that one channel "HLID" was dropped for another, "HWUT".

We have an upgrade slated today for the PSL Beckhoff computer; I captured a screenshot of the settings table in case it gets overwritten and am posting it here for posterity.

Camilla, Vicky

Screenshot of how we found the SQZ_FC and SQZ_OPO this AM. Two separate issues.

Only to-do might be fixing the SQZ_FC issue in FIND_IR. I hope the SQZ_OPO_LR issue is solved, but we will watch out for it.

More description of the issues from this morning/last night and how to fix it:

• SQZ_FC -  same stuck in FIND_IR issue as yesterday 76862. See the execution time over-run circled in today's guardian screenshot. 
   
  • To fix: brought SQZ_FC to "DOWN". That was it, SQZ_MANAGER took over the rest.
     
  • To-do: Unsure why we have gotten multiple timeouts in the past 2 days, try to resolve this. 
    • Maybe in SQZ_FC, on Line 112, we need to wrap the cdu.getdata() function call in TJ's timeout checker function. 
    • In the meantime, I have added a log statement to Line 111 just before cdu.getata() is called, and reloaded SQZ_FC. So if FIND_IR gets stuck again, we can confirm whether it is really this function call that times out the guardian.
       
• SQZ_OPO_LR - did not scan to high enough PZT voltage to find co-resonance. Hopefully we have now increased the scan range enough to fix this for O4b. Steps taken today:

  1. From it being STALLED and MANAGED, I took SQZ_OPO_LR to AUTO. This worked to un-stall it, and let it continue locking. If the scan is fixed, this should be enough to solve the problem.
      --> Not sure why OPO guardian was "STALLED" when it jumped to "IDLE" state, but I think this is related to the guardian "jump" to IDLE. I'm not sure if we even want to fix this -- leaving the SQZ_OPO_LR guardian in STALLED was probably the correct move, otherwise it would have spent hours scanning the PZT without finding co-resonance. Better to fix issues with not finding co-resonance, maybe not much to do here.

  2. OPO couldn't lock on co-resonance b/c that was at 104V, but PZT1 was only scanning up until 100V -- I changed the scan offset.

     • OPO did not find co-resonance within its scan range, then it timed out, jumped into IDLE, and the guardian STALLED. See sqz locking screenshot -- increasing the scan range let it find co-resonance at 104V, and then SQZ_MANAGER easily went back up to FDS.

     • I increased H1:SQZ-OPO_PZT_1_OFFSET from 0V to 15V, and saved this 15V offset in SDF. We cannot increase H1:SQZ-OPO_PZT_1_SCAN_STOP above 100V in slowcontrols, so we have to increase the scan offset to get the PZT there.

       • Note, back when OPO_PZT2 was at 0V (before 76688), I think the OPO would often find co-resonance between 100-110 V during O4a. 

     • Based on a previous opo co-resonance issue, 76642 seemed to want to the scan starting around ~50V. Today's issue shows we want to scan beyond 105V, probably up to 110V or even 115V (the max). Based on OPO cavity scans (e.g. screenshot from LHO:75285), I think OPO PZT1 is not super effective (in terms of um/V scanning) below 50V or so.

     • Let's try scanning from 55V - 115V to guarentee we find co-resonance. This is how it's currently set. This corresponds to the first screenshot, where H1:SQZ-OPO_PZT_1_OFFSET = 15V (saved in SDF), H1:SQZ-OPO_PZT_1_SCAN_START = 40, H1:SQZ-OPO_PZT_1_SCAN_START = 100V.

TITLE: 04/02 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
    SEI_ENV state: MAINTENANCE
    Wind: 5mph Gusts, 3mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.25 μm/s
QUICK SUMMARY:

H1 lost lock 1458 (Camilla thinks the in-lock charge measurement might have run...but caused the lockloss due to New-DARM.).

MAINTENANCE has begun!

Paused TCS_ITM_CO2_PWR guardians so that when the IFO unlocks the CO2s will stay on so we can take absorption measurements of the ITMs using HWS (need ITMs and SR2 to stay aligned for 1 hour after lockloss). 

Workstations were updated and rebooted.  This was an OS package update.  No Conda packages were changed.

TITLE: 04/02 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 155Mpc
INCOMING OPERATOR: None
SHIFT SUMMARY:

IFO is in NLN and OBSERVING as of 23:00 UTC (8hr 36 min lock)

Nothing else of note

LOG:

Here are the highlights of my data quality shift report for the week of 3/25-3/31.

• Duty cycle of 49.7% for the week between ~140-160 Mpc
• Roughly hour long periods of very low frequency (<30 Hz) noise in the strain which could not be reliably seen in seismic BLRMS or PEM overviews (see figure 1) 
  • Associated with range drops
  • Seen only on Tuesday
  • LASSO should be rerun on this day to try to identify correlations
• The SNR distribution of glitches was unexpected on Friday and Saturday - there is an overabundance of SNR ~10 Omicron triggers (see figure 2)
  • Hveto was not successful in vetoing this population
• There is a population of moderate (order 10) SNR glitches at 1100 Hz that appeared in the second half of the week (see figure 3)
  • The central frequency is apparently constant over time, so I suspect there is a common mechanism for this glitch
  • They start at roughly midnight UTC on Friday (3/29)
  • No clear morphology in Qscans
• There were several days where the number of spectral artifacts was over the alert threshold. It's unclear to me whether that is due to a lack of observing data or poor data quality

The full report can be found here: https://wiki.ligo.org/DetChar/DataQuality/DQShiftLHO20240325

IFO is in NLN and OBSERVING as of 23:00 UTC  (4hr 45 min lock)

Secondary microseism is steadily decreasing.

J. Kissel
ECR E1700387
IIET 9392
WP 11743

Here, this week we tackle the last of the suspension types that need suspension watchdog upgrades after having finished the End Station, Corner BSC, and HAM Triple suspensions this past month (LHO:76269, LHO:76545, and LHO:76712, respectively).

WP 11797
While doing so, I'm also going to address the OMC ASC routing bugs found in the h1susifoout top-level model and its OMCS_MASTER library part (which used to live in h1susomc.mdl), and fix those too.

As such, I'm doing a linear combination of the following changes:
    (1) In library part, removed sending watchdog out to top level
    (2) In library part, removed ODC parts from DAMP and LOCK banks
    (3) In library part, removed never-usered software shutter system
    (4) In library part, upgraded watchdog trigger generator to actual functional RMS
    (5) At top level, removed USER DACKILL
    (6) At top level, removed wires to ground (i.e. things never used) at top-level for WD, SHUTTER, and ODC outputs and SHUTTER input
    (7) At top level, cleaned up receiving of ISC ASC signals (see discussion in LHO:76842)

Below is the list of models, their respective library parts, and the optics in that model that this impacts, followed by a call out of which of the above changes happened
h1susim       HSSS_MASTER    IM1, IM2, IM3, IM4      (1), (2), (4), (5), (6)
h1sushtts     HSSS_FF_MASTER RM1, RM2                (1), (2), (3), (4), (6)
h1susifoout   OMCS_MASTER    OMC                     (1), (2), (4), (5), (6), (7)
              OFIS_MASTER    OFI                     (1), (2), (4), (6)
              HSSS_MASTER    OM1, OM2, OM3           (1), (2), (4), (5), (6)
h1sussqzout   HSSS_MASTER    ZM4, ZM5, ZM6           (1), (2), (4), (5), (6)
h1sussqzin    HDDS_MASTER    ZM1, ZM3                (1), (4)
              OPOS_MASTER    OPO                     (1), (2), (4), (6)
              HSSS_MASTER    ZM2                     (1), (2), (4), (5), (6)

I've test-compiled all of the above listed models, so they're ready for re-compile, install, and restart of the front-end process tomorrow morning.
All model changes have been committed to the userapps repo, either under
    /opt/rtcds/userapps/release/sus/h1/models/
    /opt/rtcds/userapps/release/sus/common/models/
where the above mentioned models live.

Screenshots describing the changes will be posted in the comments.

J. Kissel, [Corroborated by J. Wright and J. Driggers]
WP 11797

While upgrading the OMCS_MASTER.mdl library part for the OMC Suspension (which now lives in the h1susifoout.mdl top-level model), I found two distressing things:
    (1) The OMC ASC signals that are sent from the h1omc.mdl model -- running at 2kHz -- are not anti-imaged at all before being sent to the OMC SUS DAC -- running at 16 kHz. So, we're sending unnecessary imaging noise to the SUS DAC.
    (2) The OMC ASC L, P, amd Y signals are being sent through a filter bank that does NOT past through the DRIVEALIN matrix, rendering 2019's attempt at diagonalization (LHO:47488) unused.

These bugs have been long standing.
I see it now as I touch the h1susifoout.mdl and the OMCS_MASTER library part for watchdog upgrades, but I'd previously identified the issue when 
 - I was upgrading the front-end models for O4 (i.e. when the OMC SUS transitioned from the h1susomc.mdl to the h1susifoout.mdl) in 2021 (LHO:59652), and
 - When we were identifying which SUS models used the broken "TrueRMS" bloick -- there's a preceding note in the simulink model that suggests this has been around since 2015 or earlier.

We can fix this tomorrow, with *only* a change to the h1susifoout.mdl and OMCS_MASTER.mdl models, and we're already doing that anyways for the SUS watchdog upgrade.


Attached are several series of screenshots showing:
    (1) The signal path from the h1omc.mdl model where the OMC ASC control signals begin, to where they land in the OMC SUS driving in the EULER basis (again WITHOUT going through the DRIVEALIGN MATRIX)
    (2) MEDM Screens showing that in practice we only drive L, P, and Y (as one should for an ASC signal) and we don't use T, V, or R drive and likely never will.
    (3) The existing filters that will need to be copied over, and the DRIVEALIGN gain values that will likely need to be re-addressed -- since effectively we've NOT been using them.

Note -- I checked on how this is done at LLO, and they *do* drive through the LOCK banks and DRIVALIGN banks (BUT they don't have any anti-imaging filters in pace either, even though their ISCINF FM1 banks are ON... whoops)...
So, when we change the OMCS_MASTER library part, this won't affect them.