Summary: adding a digital offset to get a flat sensing function does not correspond to SRM detuning phase of 90, as modeled in FINESSE.

recap: I have modeled the SRCL DOF detuning due to mode mismatch. Nominally, we want the detuning phase of the SRM to be ø = 90˚ for resonant  sideband extraction conditions, where there are no optical spring effects. However, mode mismatches affect this phase, and consequently we can see optical spring effects showing up in the DARM sensing function.

On site, after locking we can add a digital offset to the SRM phase to flatten the sensing function, but then the question is “does the digital offset that flattens the DARM sensing function correspond to  ø = 90˚?”

And the answer is “no”, at least as modeled in FINESSE. 

I have added an offset to SRCL and looked at the corresponding sensing function, and it is clear that a ø = 90 is not necessarily a flat response (starting from 5 Hz), as the plot attached shows. 

In this case I’m looking at the RF readout (from DARM to AS45_Q), and where the average mismatch between all cavities is 0.01% (the start locking point is ø = 90.03˚). This also demonstrates that even a tiny mismatch can show optical spring effects.

TITLE: 05/22 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
    SEI_ENV state: MAINTENANCE
    Wind: 11mph Gusts, 6mph 3min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.09 μm/s
QUICK SUMMARY: HAM1 pumping continued overnight, but starting about 1.5 hours ago the pressure started rapidly climbing. No cause determined as of yet. VAC team has been contacted.

EDIT: Had the wrong channel trended, corrected in current attachment.

Starting around 6am today the vacuum pressure in HAM1, which had dropped to 6e-06 Torr, started rising rapidly. At time of typing it is up to 9.8e-04.

Richard is heading to the LVEA to investigate.

Jordan, Travis, Janos

HAM1 pumpdown continued. The new Agilent leak checker is backing the turbo pump. We are waiting for the He-signal to be at least <5E-9 Torr, so we could start leak checking. After evaluating the trends, it is likely that tomorrow morning this happens. Also, regardless this happens, the Ion-pump (IP13) will be possible to be opened up. Also, an interesting note, that after achieving molecular flow in HAM1, the corner pressure suddenly decreased by ~5-6E-9 Torr, that suggests a E-3 Torrl/s gas load through the HAM1/HAM2 septum viewport O-rings, so there is indeed some communication between HAM1 and the rest of the corner.
The overall pressure in HAM1 in the end of the day is ~9E-6 Torr, which aligns well with fully gutted HAM chamber pumpdown speeds in the past.

The large Gate Valves, first GV7, then GV5 was opened. The fully open position was achieved at ~45 and ~47 psi, respectively. No issues were encountered during the process. The corner pressure went down quickly to ~3.5E-8 Torr. RGA scans are continuously being taken.

FRS34149

Camilla and I removed the broken Model#VS35S2T0 Serial#3179 shutter and replaced it with Model#VS14S2T0 Serial#8796. The replacement unit had a slightly smaller outer diameter so we traded some posts with beam dumps that were on the side of the table not in use to get the correct height.

 Y-End TX Module Maintenance was performed today.
Everything went fairly well while following the instructions laid out in T1600436 -V12 Except the AOM Rejected Power measurement. This measurement inbetween PBSC2 and beam dump 2  is measuring the AOM power that we were dumping on to the beam dump. It didn't ever give us a static value like all the rest of the measurements. The max was a 18.5 mW  jump right after the we opened the shutter. Then the power would decay down to a lower value of 13.6 mw but over several minutes. The decay rate seemed to get longer after a while. The excitations were indeed turned off. We had zeroed the low power sensor with a shuttered background measurement. I'm not sure why that seemed to not be a steady number. We ended up taking the number that seemed like it had initially asymptote to before we saw the slow fall off.

The Beam Spots all look great. TX Sphere might be very slightly  to the right of perfectly center, but I think it's fine.
 

OLTF went well, Setup was well documented.

TITLE: 05/21 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: None
SHIFT SUMMARY: Busy day today! Arm gate valves are open, all HAM HEPI's are unlocked, and HAM1 continues to pump down. More activities below:

• After h1susb123 crashed (alog84500), all 18-bit DACs were upgraded to 20-bit: alog84508
  • 4x gain added to COILOUTFs on upgraded DACs: alog84509
  • MEDM screens updated: alog84514
• PSL PMC/FSS alignment: alog84512
• ALS-X beam profiling: alog84513
• h1sush7 crash and recovery: alog84521
• HAM1 cabling: alog84525

LOG:

The HAM1 floor cable tray near the PSL was reinstalled. Removed for the ISI installation. All cables on flanges D4 and D6 are dressed in front of the SEI cross beam. Cables are usually pulled behind the cross beam to avoid cables resting on the beam and to provide strain relief. More work needed to finish cable dressing/strain relief.

The following whitening controls concentrators D2400263 were installed:

ISC-R4, slot U30 S2400747
ISC-R4, slot U27 S2400748

ISC-R6, slot U33 S2500400
ISC-R6, slot U15 S2500401

F. Clara, J. Figueroa

J. Kissel

During today's upgrade of susb123's DACs (LHO:84519), and as Ryan and I were adding calibration change compensation in various OUTF banks (LHO:84509) I realized that we've never accounted for the gain loss when upgrading the PI DACs from 18-bit DACs to 20-bit DACs.

That "gain loss" more explicitly:
    - Assume you tune the open loop gain of the PI damping loop to be some magnitude |G|, with a 18-bit DAC, which has gain calibration of 20V/2^18 = 7.6294e-5 [V/ct].
    - Then you upgrade the DAC to a 20-bit DAC, which has gain calibration 20/2^20 = 1.9073e-5 [V/ct], a factor of 0.25x, or a factor of 4.0x less [V/ct].
    - This drops the loop gain |G| by the same factor of 4x, if not accounted for digitally.
Note, this is a scale factor at all frequencies. So, even if the DAC output is high-passed at 10 kHz, the gain above 10 kHz is still lower. 
I say this because they are high-passed at 10 kHz in both the ETM and ITM low-noise ESD drivers; see D1400301 block diagrams and circuit drawings D1500016 and D1600122 for ETM and ITM.

In every other SUS DAC chain, we compensate for this with a "20bitDAC" filter which is a "gain(4)" filter so that any upstream loop that uses that DAC chain doesn't have to be re-designed upon the transition from 18- to 20-bit DAC.

Here's the history of when each PI DAC was upgrade, including all the different names / numbers of how they're referred:
    IO Chassis    Card Name    "The nth DAC"   Slot    When Upgraded       aLOG 
    h1susex         DAC4           5th          7        Oct 2018          LHO:44918
    h1susey         DAC4           5th          7        Jun 2020          LHO:56217
    h1susb123       DAC6           7th          9        May 2025          LHO:84508

However, the PI models and MEDM interfaces aren't built with convenient places to apply a DAC gain calibration fix. An inconvenient but logical location to account for this would in each and every MODE's UPCONV_UC[n]_SIG bank, and I see no evidence of such a gain filter. If you wanted to be sneaky, you increase the OUT_MTRX coefficient from +/-1.0 to +/-4.0, but that has also not been done. I don't see any other evidence an additional gain of 4x along any mode's digital chain. 

I do note that we developed "EXTREME_PI_DAMPING" Dec 2024 LHO:81909 in hopes to get more oomph out of the ETMY PI system.
I wonder if we can get more oomph by just increasing the loop gain by a factor of 4x ...

I'm using this aLOG to mildly advocate for an ECR to implement some ESDOUTF filter banks just upstream of the DAC outputs, like we do for the audio-band DAC requests on the rest of the suspensions.

13:25:11 Wed 21 May 2025 PDT all models on h1sush7 stopped running. The SWWD to h1iopseih7 started its countdown with 100% IPC receive errors.

I set a bypass time of 999999 on h1iopseih7 to interrupt the countdown.

We first restarted all the models. This did not clear the error, and we found a PCI bus issue whereby only one of the four ADCs could be found.

As a precautionary measure we fenced h1sush7 from the Dolphin fabric.

Next we power cycled the computer. This fixed the PCI bus issues, all cards are visible and all models starting running.

I cleared the SWWDs and handed the system over to the control room.

Note, there were TIMING error flashes during this recovery time which we have not traced down.

[Wed May 21 13:25:11 2025] h1iopsush7: ERROR - An ADC timeout error has been detected, waiting for an exit signal.
[Wed May 21 13:25:11 2025] h1susauxh7: ERROR - An ADC timeout error has been detected, waiting for an exit signal.
[Wed May 21 13:25:11 2025] h1sussqzin: ERROR - An ADC timeout error has been detected, waiting for an exit signal.
[Wed May 21 13:25:11 2025] h1susfc1: ERROR - An ADC timeout error has been detected, waiting for an exit signal.
 

J. Kissel
E1900216 (IIET:13232)
E2100485 (IIET:20828)

In light of the last 24 hours of 20-bit DAC upgrades (see LHO:84519), it's worth updating my Oct 2023 (LHO:73518) list of priorities for DAC upgrades:

    (a) [2x] Upgrade PRM M2/M3, and PR2 M2/M3. 
        DAC5 (Slot 8, card_num = 5) on h1sush2a for PRM
        DAC4 (Slot 7, card_num = 4) on h1sush34 for PR2
    (b) [2x] Upgrade PR3/SR3 M2/M3 DACs
        DAC6 (Slot 9, card_num = 6) on h1sush2a for PR3
        DAC3 (Slot 6, card_num = 3) on h1sush56 for SR3
    (c) [1x] Upgrade ITMX and ITMY UIM DACs
        DAC4 (Slot 8, card_num = 4) on h1susb123
    (d) [3x] Finish out susex IO chassis upgrades (which is ETMX M0 / R0 and TMSX M1)
        DAC0, DAC1, DAC2 on h1susex
    (e) [3x] Finish out susey IO chassis upgrades (which is ETMY M0 / R0 and TMSY M1)
        DAC0, DAC1, DAC2 on h1susey
    (f) [5x] Finish out susb123 IO chassis upgrade (which is ITMX M0/R0, ITMY M0/R0, and BS M1)
        DAC0, DAC1, DAC4, DAC5, DAC9 on h1susb123
    (g) [3x] MC2 M2/M3, MC1 M2/M3, MC3 M2/M3
        DAC3 (Slot 7, card_num = 3) on h1sush34 for MC2
        DAC3 (Slot 6, card_num = 3) on h1sush2a for MC1
        DAC4 (Slot 7, card_num = 4) on h1sush2a for MC3
    (i) [3x] MC1/MC3/PRM/PR3 top masses
        DAC0 (Slot 2, card_num = 1) in h1sush2a for MC1/MC3 M1
        DAC1 (Slot 4, card_num = 1) in h1sush2a for MC3/PRM M1
        DAC2 (Slot 5, card_num = 2) in h1sush2a for PRM/PR3 M1
    (j) [3x] MC2/PR2/SR2 top masses
        DAC0 (Slot 2, card_num = 0) in h1sush34 for MC2/PR2 M1 
        DAC1 (Slot 4, card_num = 1) in h1sush34 for PR2/SR2 M1
        DAC2 (Slot 5, card_num = 2) in h1sush34 for SR2 M1
    (k) [4x] SRM/SR3/OMC top masses
        DAC0, DAC1, DAC2, DAC3 in h1sush56
    (l) [2x] IMs
        DAC0, DAC1 in h1sush2b

25 cards left to go!

J. Kissel, F. Clara, E. von Ries, D. Barker, E. Dohman, O. Patane, R. Short

After fixing MEDM screens, I was able to validate that all expected drive signal channel assignments reach the expected DAC outputs for H1SUSITMY, H1SUSITMX, and H1SUSBS. 

As such, I restored the function of top mass damping loops, L2 to R0 tracking for the ITMs, and optical lever damping for the BS.

We've confirmed that alignment offsets have been restored.

With the SUS happily damping, I then restored the seismic isolation system, using the SEI guardians to bring HEPIs and ISIs back to FULLY_ISOLATED for the ITMs and FULLY_ISOLATED_NO_ST2_BOO for the BS.

Executive Summary of the last 24 hours of events regarding h1susb123 DAC card failure:
    - LHO:84500 h1susb123's Slot 4, DAC 2 -- an 18-bit DAC -- died and caused the front-end to crash, which eventually tripped all BSC1, BSC2, and BSC3 seismic systems via Software Watchdog (SWWD) 
    - LHO:84506 Having agreed that we should take the opportunity to enact ECR E1900216 (IIET:13232) and upgrade all the DAC cards in the chassis to 20-bit DACs -- we prepped the h1susb123 user and IOP models for the change 
    - LHO:84508 Erik and Fil replace all 18-bit DAC cards with 20-bit DAC cards, Dave compiles, installs, and restarts the h1susb123 front-end's models with the new code.
    - LHO:84509 With the models running, Ryan and I installed updates to the COILOUTF filters to account for the calibration change between 18 and 20 bit DACs.
    - LHO:84514 I cleaned up the MEDM interface for the SUS-ITMs and SUS-BS to ensure they accurately show signal flow from user model output to the DAC.
    - (this aLOG) Having validated signal flow, we restored the SEI and SUS systems to full nominal functionality.

This post summarizes some features of the 106.06736 comb’s history over O4 so far. This comb was found in the run-averaged spectrum (Fig.1) while updating the lines lists. Its cause is unknown. 

• There are several peaks over the course of the run in the 106 Hz region. Not all of these peaks occur at the same time.
• For some of these peaks, it appears that the peaks entirely disappear during breaks occurring at least one month later. For others, the peaks continue after the breaks, as evidenced by the attached weekly spectrogram (Fig.2).

Attached are static images of plots displaying the comb’s structure and behavior. Fig.1 is the run-averaged spectrum of the comb and the other two (Figs.2 and 3) are spectrograms of it. The blue spectrogram in Fig.2 is weekly over the ongoing O4 run and the green spectrogram in Fig.3 is daily.

Temporary HAM1 gauge EPICS channel

In lieu of the Beckhoff IOC, the HAM1 gauge raw voltage signal was converted to a pressure channel by a new EPICS IOC. This channel was added to the DAQ for trending, EDC+DAQ restart was needed

DAQ Restart

For above HAM1 vac channel. Also H1EPICS_PWRSTRIP.ini chans were added.

WP12562

Jonathan, Erik, EJ, Jeff, Ryan, Fil, Dave:

As part of the h1susb123 crash investigation, after power cycling the computer and IO Chassis the next thing to do was to replace the 2nd 18bit-DAC card which was no longer visible on the bus.

During the 08:30 vent meeting it was discussed which option we should pursue: replace the suspect 18bit-DAC with another 18bit-DAC, replace this one card with a 20bit-DAC, replace all six 18bit-DACs with 20bit-DACs. The third option was chosen.

Hardware:

Keep the exisiting two 20bit-DACs, replace the six 18bit-DACs with 20bit-DACs. Put aside the second 18bit-DAC as a suspect card, put the other five cards into the O4 spare pool.

Sofware:

h1iopsusb123:

replace the DAC area with eight 20bit-DACs from CDS_PARTS, card numbers 0-7. The SWWD system already DACKILLs all eight DACs (card_num 0-7) irregardless of which suspension trips, so no change needed.

No change to INI file.

h1susitmx, h1susetmx, h1susbs, h1susitmpi:

See Jeff's alog for model changes. DAC part naming was chosen such that there were no changes to INI files.