Wed Jun 19 10:07:48 2024 INFO: Fill completed in 7min 45secs

1402849079

Ended a 20 hour lock. Looking at the plots it seemed to be a very fast lock loss. I see that LSC-DARM wiggle that we often see, but the magnitude of it is a bit smaller than what I have been looking at in the recent past.

TITLE: 06/19 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 148Mpc
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 14mph Gusts, 10mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.07 μm/s
QUICK SUMMARY: Locked for 18.5 hours, the range has been decreasing a bit the last hour or two. I'll run some checks to see what's going on.

There is a loud whining in/on the OSB that can be heard from the parking lot and alos in the control room. I think it might be coming from the roof or somewhere in the ceiling. I'll see if I can localize it a bit before calling facilities.

TITLE: 06/19 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
INCOMING OPERATOR: Tony
SHIFT SUMMARY: We've been locked for 11:50, calm night. The trace on the bottom of nuc5 isn't showing up.

23:25 -23:27 UTC We dropped out of observing to run SCAN_SQZ_ANG to improve SQZing which did not look good following its last relock.

If we lose lock and relock ISC_LOCK will need to be taken to INJECT_SQUEEZING then back to NLN to reset the ISC_LOCK notification about SQZ_MANAGER.

Rahul ran the OPLEV charge measurement this morning for ETMY so I processed the measurement.

TITLE: 06/19 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Observing at 150Mpc
We've been locked for almost 8 hours, the wind has died down a bit.

Last checked in alog78384

Closes FAMIS 28358

While the SUS_CHARGE guardian LOG shows no errors in the measurement, ITMX, and ITMY failed to be processed. ITMX complained of bad coherence and or failed measurements:

" UserWarning: Cannot calculate alpha/gamma because some measurements failed '             'or have insufficient coherence!
  warn("Cannot calculate alpha/gamma because some measurements failed ' \ "

ITMY had the following error when trying to load the data:

"ValueError: no Fr{Adc,Proc,Sim}Data structures with the name H1:SUS-ITMY_L3_ESDAMON_DC_OUT_DQ"

As shown in the attachment, SQZ unlocked for unknown reason at 2024/6/18 23:12:09 UTC and the range got much worse after it relocked. We ran the SCAN_SQZANG and the range was recovered. The optimal SQZ angle changed from 225 to 203 after relocking and it seems due to the different OPO PZT voltage.

TITLE: 06/18 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 122Mpc
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Quieter maintenance day. Relocking was fully automated after and initial alignment. We have been locked for 3.5 hours. The SQZer has unlocked a few times since the team finished their table work, and Naoki has been looking into this.
LOG:

• 1505 Intentional lock loss for maintenance
• 2035 Observing
• 2044-2059 dropped observing when the SQZer unlocked. Naoki fixed the issue
• 2312-2315 dropped observing when the SQZer unlocked.

TITLE: 06/18 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Observing at 151Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 12mph Gusts, 9mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.07 μm/s
QUICK SUMMARY:

• We've been locked for just under 3 hours
• The wind is a low/moderate

Andrei, Terry, Naoki

We aligned pump AOM and fiber following 72081.

We set the ISS drivepoint at 0V and measured the AOM throughput. The throughput was 30 mW/60 mW = 50%. We aligned the AOM and got 57 mW/60 mW = 95% throughput.

Then we set the ISS drivepoint at 5V and aligned the AOM to maximize the 1st order beam. The 1st order beam improved from 3.5 mW to 11.5 mW. Then we set the ISS drivepoint at 0V and checked the AOM throughput again. The throughput was only 35mW/60 mW = 58%. We found that bottom screws of AOM for vertical alignment maximized the 1st order beam, but the 0th order beam significantly reduced, which means there is a clipping. This was the case in the last AOM alignment in 77508.

To avoid the clipping, everytime we aligned the AOM and improved the 1st order beam, we made sure that the 0th order beam is not clipped. We ended up with 10.5 mW of 1st order beam and 55 mW of 0th order beam.

After AOM alignment, we aligned the pump fiber, but the fiber alignment did not improve much.

We reverted the OPO trans setpoint from 70 to 80, which was done in 78467.

WP11924 TW0 offload

Dave:

The past 6 months of raw minute trend were archived on TW0 and the NDS process on h1daqnds0 was restarted to serve these data from this temporary location while the transfer is ongoing.

The copy of these files from TW0 SSD-RAID to permanent HDD-RAID storage on h1ldasgw0 was started at 09:39 PDT. As of 13:26 PDT, 37 of the 256 directories had been copied. This gives an ETA of Wed 19th 11:50 PDT.

WP11937 DMT Sensemon2

Jonathan, Jamie, John, Dave:

The dmt2epics IOC was updated to add 4 new channels:

+[H1:CDS-SENSMON2_BNS_EFF_RANGE_CLEAN_MPC]
+[H1:CDS-SENSMON2_BNS_EFF_RANGE_CLEAN_MPC_GPS]
+[H1:CDS-SENSMON2_BNS_RED_SHIFT_CLEAN]
+[H1:CDS-SENSMON2_BNS_RED_SHIFT_CLEAN_GPS]
 

DAQ+EDC restart was needed.

Replacement of broken Pulizzi power control unit

Ryan S, Dave, Erik:

Ryan discovered that pwr-cer-pem0 Pulizzi unit which powers the PEM magnetic coil located in the inner cleaning room (between gowning and LVEA, on the wall it shares with the CER) was unresponsive with an error message "Error: telnet to host pwr-cer-pem0 fail". This was a strange failure in that the unit was not pingable or telnetable, but the sw-lvea-aux port it was attached to was not reporting a comm error. Ryan compared this Pulizzi with a working one an nothing suggested it had failed.

Trending back this unit could have failed mid-May, 14th May was the last time it ran successfully.

Erik replaced this unit with a good spare. Ryan is looking to modify the magnetic injection script to verify correct Pulizzi control.

DAQ Restart

Jonathan, Dave:

The DAQ was restarted for the DMT change. This was unremarkable other than gds0 needing a second restart.

Maintenance activities finished up just before noon, and the SQZ team has transitioned the LVEA back to hazard to work on SQZT0. I was able to start initial alignment early and we are now on DC readout.

J. Kissel

I'm on a slow-but-steady adventure to characterize the performance of the digital anti-aliasing system for the new 524 kHz ADC readout of the OMC DCPDs photocurrent. 
The big picture goal is too look at the 524 kHz data, and make sure that it's filtered enough before down sampling to 65 kHz and then 16 kHz, such that none of "fun and interesting" features that are real DARM signals (or analog or digital artifacts of the ADC) in the 10 - 100 kHz region appear down-converted at low frequency.

As usual, when I *look* at something for the first time, hoping for a simple "yup, works as expected," I instead get 60 new questions.

Executive summary: 
While I don't see anything obviously detrimental to the 16 kHz data, I see a confusing, frequency-independent noise floor in the filtered version of the 524 kHz OMC DCPD A channel *only* in NOMINAL LOW NOISE that I can't explain as either ADC noise, spectral leakage, or limits of numerical precision. Apparently, in NOMINAL LOW NOISE, we're not getting nearly as much digital anti-aliasing as designed.

Plots and their Explanation, Exploration, and Conclusions:

First, I compare the amplitude spectral densities of the two 524 kHz channels available for DCPD A during NOMINAL LOW NOISE vs. when DARK (the IMC OFFLINE), "as the are" 
    - H1:OMC-DCPD_A0_OUT (calibrated into milliamps [mA] on the DCPDs),                        # HAS 526-to-65 kHz and 65-to-16 kHz digital AA filters
    - H1:OMC-PI_DOWNCONV_SIG_OUT (a copy of the DCPD A channel, also calibrated into [mA]),    # DOES NOT HAVE digital AA filters

The first attachment, H1OMCDCPDs_0p1HzBW_10avgs_524kHzSignals_NLN_vs_DARK_AprtHz.png shows this comparison. 
These are the *output* of the respective filter banks, so the front-end filter banks already do *most* the work of inverting the frequency response of the TIA and Whitening filters, below 10 kHz.
That front-end calibration also already accounts for the fact that the analog voltage coming into the ADC is copied 4 times and summed, and does the "divide by four" gain of 0.25 to create an average of the voltage copies.
Importantly, the TIA and whitening's analog 11130 & 10170 and 44818.4 Hz poles are *not* inverted, so they remain a part of the frequency response of the readout -- and thus they remain uncompensated for in the plot.
The *only* calibration applied in DTT for these channels is a simple 1e-3 [A/mA].
    NOMINAL LOW NOISE, "reference" data for the two channels -- shown in CYAN and MAGENTA -- was taken a last week on 2024-06-12 20:45:46 UTC.
    DARK noise, "live" data for the two channels -- shown in RED and BLUE -- was taken this morning, with the IMC offline at 2024-06-18 15:10:28 UTC.

One immediately notices several "interesting" things:
    (1) The CYAN, OMC-PI_DOWNCONV_SIG_OUT version of the (4x copy average of the) DCPD A signal -- that doesn't have digital anti-aliasing filters applied -- shows lots of "fun and interesting" features on the DCPDs above 8 kHz in NOMINAL LOW NOISE. 
    (2) Comparing the CYAN NOMINAL LOW NOISE data with BLUE DARK data, we see that a lot of these "fun and interesting" features in the 8 kHz to 100 kHz region are real features from the detector. My guess is that this is the forest of acoustic modes of optics that appear in the DARM signal.
    (3) MAGENTA OMC-DCPD_A0_OUT version shows that most of these features *are* filtered out by 30 kHz by the digital AA filtering, BUT -- they hit some frequency-independent noise floor at "1e-12 [A/rtHz]."  This noise floor is obviously *not* a real noise floor on the DARM light coming in on the PDs, nor some sort of current noise, given how feature full the CYAN unfiltered version is. As such I will for the remainder of the aLOG put the quantitative numbers about this noise floor in quotes. What is this noise floor?
    (4) Comparing MAGENTA NLN trace and RED DARK trace, we see that when the DCPD is DARK, we see the full expected frequency response of the digital AA filters. Why are the filtered data's DARK and NLN noise floors so different?

Hoping to understand this noise floor better, I calibrated the traces into different units -- the units of input voltage in to the low-noise, 18 bit, 524 kHz ADC.
The second attachment, H1OMCDCPDs_0p1HzBW_10avgs_524kHzSignals_NLN_vs_DARK_CastAsADCInput_VprtHz.png, shows this different version of the ASD.

To calibrate into ADC input voltage units, I calibrated all the DTT traces by the following calibration (with detailed poles and zeros pulled from the calibration group's DARM loop model parameter file, pydarm_H1.ini from the 20240601T183705Z report): 
    zpk([2.613;2.195;6.556],[5.766+i*22.222;5.766-i*22.222;32.77;11130;10170],219.65,"n") # The TIA response [V/A], normalized to have a gain of 100e3 [V/A] at 1 kHz
  * zpk([0.996556],[9.90213;44818.4],1,"n")                                               # The Whitening Response [V/V]
  * gain(0.001)                                                                           # The inverse of the [mA]
where notably, I continue to exclude the two 11130, 10170 Hz poles and the 44818 Hz pole from the TIA and whitening filter's analog response. 

To compare against our model of the noise of the low-noise, 18-bit ADC sampled at 524 kHz (derived from the data in LHO:61913, shown to be comparable in G2201909), I took that model and divided by sqrt(4) = 2.0 to account for the 4 copies of the ADC noise that appear during the digitization of the 4 copies of the analog voltage. 

I also show the RMS voltage (color coded to match the corresponding ASD) in case that's important for discussions of noise floors.

This exposes more interesting things, and rules out one thing that this 1e-12 [A/rtHz] noise floor might be -- it's *not* the ADC noise floor.
    (5) Both version of the DARK noise traces, BLUE and RED show that the data agree with the noise model below 5 Hz, which gives me confidence that the scale of the model is correct across all frequencies.
    (6) The MAGENTA trace shows that, when recast into ADC input voltage, the filtered nominal low noise data's "1e-12 [A/rtHz]" noise floor is "1e-6 [V/rtHz]," which is a factor of ~4 above modeled ADC noise floor.
    (7) The BLUE trace shows the dark noise -- without the digital anti-aliasing filters -- is *also* above the ADC noise, is frequency-dependent, and *below* the MAGENTA filtered NOMINAL LOW NOISE data in the 20 to 200 kHz region. 
(5), (6), and (7) all give me confidence that this frequency independent noise is *not* ADC noise
    (8) The ADC input voltage 
        (a) during NOMINAL LOW NOISE data spans 5 orders of magnitude (0.2 [V_RMS] total in the CYAN NLN unfiltered trace, w.r.t. this 1e-6 [V/rtHz] high frequency noise floor in the filtered MAGENTA trace), and
        (b) the DARK data spans 6 orders of magnitude (5.4e-4 [V_RMS] of the BLUE unfiltered trace, w.r.t the 1e-10 [V/rtHz] at the lowest 65 kHz notch in the RED trace). But BOTH are still above the dynamic range of where floating point precision would be causing problems -- dynamic range of 8 orders of magnitude. So, I don't think it's a numerical precision issue, unless I'm misunderstanding how that works.
    (9) Also, given that the NOMINAL LOW NOISE data spans less orders of magnitude that the DARK noise data, and yet the filtered RED data does *not* show any such frequency-independent noise floor, I also don't think it's an issue what window I've chosen on the ASDs (it's the default Hanning window, for the record).

Just to add another view in further different units, to explore (8) a little better, I attach a third version of the plot, but cast into the 18 bit ADC's counts.
This is the third attachment, H1OMCDCPDs_0p1HzBW_10avgs_524kHzSignals_NLN_vs_DARK_CastAsADCInput_ctprtHz.png.
The further calibration is a simple multiplication of the ADC Input Voltage by a further 2^18/40 [ct/V]. 
(The inverse of this ADC voltage calibration has already been done in the "raw" channels in [mA], and the factor of 4 for the number of copies being summed has already been accounted, so I *can* treat these channels like a single ADC channel).
    (10) Not much more interesting here, the MAGENTA trace showing "1e-12 [A/rtHz]" or "1e-6 [V/rtHz]" high frequency noise floor of the filtered NOMINAL LOW NOISE data, now in ADC counts, lands at an uninteresting "7e-3 [ct/rtHz]", compared to an RMS of ~1000 [ct_RMS]. So, it's not like we're bottoming out on ADC counts or ADC precision.

I'd really like to understand this noise, since it's present during NOMINAL LOW NOISE, and it's indicating that there's some flaw in our digital anti aliasing.

Camilla, Andrei, Naoki, Sheila

There have been many examples of times when the filter cavity length locking error signal peak to peak seems to be correlated with worse range over the last 3 months, I've attached some screenshots of examples. This was true before the May 16th change in the status of the CO2 ISS channel , 78217, and before the output arm damage that happened April 22nd.

Some of these times correspond to times when there is a whistle visible in the FC WFS signals 78263, others do not.  These whistles in the FC WFS channel have been present throughout all of O4, but they do go away sometimes for several days, this last week they do not seem to have been present.  Andrei has identified a candidate wandering line in the IOP channels for these WFS that was last week ~10 kHz away from the 105kHz RLF-CLF signal, today that line seems to be gone.

Last week, the filter cavity error signal peak to peak became much noisier than it was previously (screenshot), until June 15th at around 15 UTC when things returned to the previous state.  Camilla identified that this started a few hours after the time of the ringdown measurement attempt, 78422, and that there haven't been any ZM1/2/3 alignment changes.  During that time period, the FC error signal from around 0.7-9 Hz has higher and variable, in addition, the low frequency noise was changing and varriying the RMS as it has done before.  The attached screenshot shows some of the typical low frequency variation (compare yellow to blue), a whistle in the yellow trace, and in red a time during last week's elevated noise when the low frequency was relatively quiet but there is elevated noise from 0.7-9Hz.