I had to make a temporary modification to DIAG_CRIT to ignore the ISI_ETMY_ST1_SC node. While we are doing the wind fence work, we pause this node to avoid it automatically putting out super sensor back in. The test in DIAG_CRIT watches all of the SC and BLND nodes ACTIVE channels (history of this addition - alog51606). Jim is working on another solution and then we can remove this modification. While I was in DIAG_CRIT I also updated the SQZ subordinate nodes.

As of right now we monitor 119 Guardian nodes to be OK. Current exclude list for IFO node is below. On top of the nodes in this list are the seismic nodes that end in SC (sensor correction) or BLND (blend) since these will change with earthquake mode, an approved changed in observing back in O3.

EXCLUDE_NODES = [
    'BRSX_STAT',
    'BRSY_STAT',
    'DIAG_MAIN',
    'H1_MANAGER', # This node should get taken out of this list after testing
    'HIGH_FREQ_LINES',
    'IFO_NOTIFY',
    'IFO_UNDISTURBED',
    'PEM_MAG_INJ',
    'SEI_CONF',
    'SEI_DIFF',
    'SEI_ENV',
    'SQZ_SHG',
    'SQZ_OPO_LR',
    'SQZ_CLF_LR',
    'SQZ_LO_LR',
    'SQZ_FC',
    'SR3_CAGE_SERVO',
    'SUS_CHARGE',
    'TEST',
    'VIOLIN_DAMPER',
    ]

A few notable nodes in this list are SUS_CHARGE and PEM_MAG_INJ. Both of these cannot ever be in an OK state without being overly hacky, but since they run excitations from models that are monitored, we will get taken out of Observing when they start.

J. Kissel

In order to reduce the layers of confusion when a "headless" command line use of the dedicated excitation channel "reserved" (named after) driving the QUADs for calibration purposes ($(IFO):SUS-$(OPTIC)_$(STAGE)_CAL_EXC) fails (e.g. what happened overnight, see LHO:69086), I've added a link to a new sub-screen underneath each of the dark purple "$(STAGE) CAL EXC" buttons (see QUAD_OVERVIEW snippit) that have never been buttons before. The new screen is quite simple, see screenshot. Perhaps most importantly, it indicates -- directly on the screen -- the exact commands to enter into a terminal to *kill* the excitation. Note, there's some smarts in that instruction set -- the "tp clear $(FEC) *" line, indeed, has an $(FEC) macro variable that updates correctly according to *which* QUAD the screen corresponds to.

The following screens,
    /opt/rtcds/userapps/release/sus/common/medm/quad/
        SUS_CUST_QUAD_ITM_OVERVIEW.adl   # Updated with link to new screen below
        SUS_CUST_QUAD_OVERVIEW.adl       # Updated with link to new screen below

        SUS_CUST_QUAD_CAL_EXC.adl        # New Screen
have been updated and/or committed to the that location in the userapps repo.

Last night they had trouble locking, specifically ALS_COMM couldn't find IR (alog69063) , and found that if they avoid SDF_REVERT that they could lock. This afternoon when we were relocking we found the same issue. We went through CHECK_SDF and then after finding IR I looked through all of the different SDF diffs and trended ones that I though might be related to see if they were reverted last time we went through that state. The H1:ASC-{X|Y}_TR_{A|B}_WHITEN_GAIN and GAINSTEP seemed to be the issue and once I saved the values seen in my screen shots, we could go through SDF_REVERT and find IR without issue.

Randy, Tyler, Jim, Mitchell

Today the last of the cables were strung and tensioned. Further adjustment of cable tension may be needed. This will be determined after the panels have been installed.

Thu Apr 27 13:17:51 2023 INFO: Fill completed in 2min 51secs

Thu Apr 27 14:05:57 2023 INFO: Fill completed in 5min 56secs

The first fill ended prematurely, tripping at -90C before the discharge line pressure could increase. This is due to the warm up of the ambient temperature as we go into spring.

We tried a second run at 14:00 with lowered trip temperatures at -100C. This was probably too soon after the first run and the TCs had not settled below -60C, resulting in another premature trip.

At this point we had an total fill time of 8min 47secs which is most probably sufficient for today. We will run again tomorrow with the new -100C trip levels (nominal settings attached)

Mid day Update!
Big Red was moved with out breaking lock!

IFO Status : Re-locking after a localized spike in ground motion caused a lockloss.

Issues relocking.... Lockloss at FIND_IR, Seesm like a very similar issue to last nights locking issues.
We are trying to resolve the issue, and relocking will commence.



 

SQZ Team. Last night we left the ADF on at 1300Hz while the IFO thermalized. See attached plot for how ADF calculated phase changes 25degrees over the first 5 hours of the IFO lock. We plan to repeat this tonight with the ADF nearer 100Hz (where H1 range is effected by SQZ changes more) and expect less change. See alog 69055 for optimizing SQZ angle during first ~1hour of lock. SQZ angle H1:SQZ-CLF_REFL_RF6_PHASE_PHASEDEG = 140.33 throughout.

Remeasured and retuned the PRCL feed-forward.

The old filter wasn't subtracting much: it turns out PRCL coupling is about 2x higher than what measured before

Daniel, Naoki, Nutsinee

A following up of LHOalog68991 we injected frequency dependent squeezing with high CLF into DARM and looked at the coherence between our sqz laser intensity noise and DARM. In the end we increased CLF power by a factor of 8 and made DARM noise around 1.3kHz worse by 10.8%. Multiplying our previous CLF intensity noise projection of 3.7e-21m/sqrt(Hz) by sqrt(8) then add to our DARM noise floor at nominal CLF power of 2.95e-20 m/sqrt(Hz) in quadrature you get

sqrt((2.95e-20)^2+(10.46e-21)^2) = 3.13e-20 m/sqrt(Hz)

that is off from what we observed, 3.27e-20 m/sqrt(Hz), by 4.5%. It's not too crazy. Daniel's theory was intensity noise tumbles in phase but the ADF peak doesn't. If we tumble the phase of the ADF peak and project our noise floor from the mean value instead we might be closer. Something we intend to do during the next lock.

A coherence measurement between DARM and SQZ intensity noise demodulated at 3.125 MHz doesn't show much coherence but it's not nothing. The coherence is small but it does go up with CLF power. We are not sure what this means but the measurement could be an indication that our high CLF power could be intensity noise related.

To get a calibration factor for Anamaria in order to run coincident magnetic injections at both sites in the future, I drove the CS vertex coil with a gain of 600,000 between 18:28 and 18:33 UTC. Response in DARM and my injection settings are attached.

I accepted the three FM3s in the susprocpi sdf.  These filters are turned on by the SUS_PI guardian, so they are already correctly saved as not on in the safe.snap, but needed to be accepted as on in the observe.snap.

I went through the sdfs for lscaux.  I have accepted all of the diffs (shown in screenshots below) in the Observing.snap file.

These are the most recent settings that we had been using for monitoring various signals through thermalization and TCS tests, so we want them to still be there should we need them. They are of no particular consequence as long as the amplitude is zero (which it is right now, since turning these lines on is removed from the guardian).

J. Kissel

As indicated in LHO:69065, we lost lock in the middle of a headless run of the calibration sweeps. What we didn't see until this morning was Louis' instruction that we need to clear test points after we ctrl+C out of a failed headless run of the sweeps (LHO:69072). As such, an unintented, 5.6 Hz ETMX UIM excitation had been running since yesterday afternoon's lock loss. 5.6 Hz merely because that's where the ETMX EXC sweep was at the point of lock loss.

I've now cleared that test point at of 2023-04-27 ~17:22 UTC.


jeffrey.kissel@zotws20:~$ diag -l
testpoint_client 1.0.0
ndshosts: h1daqnds1 and h1daqnds0
getting host by name: h1daqnds1
found host
supported capabilities: testing  testpoints  awg  
diag> tp clear 88 *
test point cleared
diag> exit
EXIT KERNEL
jeffrey.kissel@zotws20:~$ 

I'll work with the pydarm measure authors to make sure we have a graceful exit of headless operation if/when we have a lock loss in the middle of the measurement.

Betsy, TJ, Rahul

I have accepted or unmonitored a bunch of settings on SUS SDF for the OBSERVE STATE. I trended all the settings difference and accepted everything which didn't change in the last few months. I have also set the offsets (for COILOUTF) to zero for all the QUADs and R0 chain after going through all of them. There were a few things which were changed by the GRD, hence I have either accepted them or unmonitored (things like TRAMP) them.

Attached below are the screenshot of all the SDF which original existed when I started yesterday morning. Hence if changes were accepted by mistake then please compare it with the screenshot and make the necessary changes.

Still there are a few SUS SDF remaining and we are planning to tackle this today (currently ongoing).

When the gain of the ISS second loop was changed, the input offset was not adjusted. This results in the output of the 2nd loop ISS to saturate after a lock loss, until the offset slowly gets compensated by the offset loop. But, this now typically takes like 5 minutes. I adjusted the offset (H1:PSL-ISS_THIRDLOOP_OUTPUT_OFFSET) from 587.3 to 1272. We will have to check after the next lock loss that this actually works.

Adrian, Camilla, Gabriele, Fil, Richard

While trying to track down the source of the 4 Hz bumps in DARM, Richard noticed the cosmic ray photomultiplier tube (PMT) power supply voltage display numbers were jumping around. When he reset the dial, we noticed the 4 Hz oscillations in H1:PEM-ADC_5_29_2K_OUT_DQ disappeared for a few minutes. We unplugged the power supply to the PMTs at 19:05:00 UTC and have not seen 4 Hz features in the ADC channel or DARM since. Plots of the change in ADC channel timeseries behavior when Richard first touched the power supply knobs as well as the ADC channel behavior after the power supply was unplugged are attached. A spectrum comparing the ADC power supply plugged in (blue) and unplugged (red) is attached. We had about 40 minutes to watch DARM before some commissioning tasks started and we didn't see any more bumps. Another plot is forthcoming for this...

(edit for spelling...)

JoeB, JeffK, TonyS, DriptaB, RickS

Using Dripta's python script (/ligo/svncommon/CalSVN/aligocalibration/trunk/Projects/PhotonCalibrator/scripts/O4/Pcal_calibration_EPICS.py) to calculate the required EPICS values and write the text file with the caput commands (/ligo/svncommon/CalSVN/aligocalibration/trunk/Projects/PhotonCalibrator/O4_EPICS_calibration/Pcal_LHO_CAPUTfile_O4run_2023-04-25.txt), we updated the EPICS values required by JoeB's new front-end code to calibrate the Pcal power sensor outputs.  A copy of the script that generated the .txt, Pcal_calibration_EPICS_2023-04-25.py, is written in the O4_EPICS_calibration directory with the .txt file.  Note that it must be run from the scripts directory.

Images of the new PCAL_END_FORCE_COEF.adl MEDM screens are attached below.

The caput commands that were executed are:

caput H1:CAL-PCALX_FORCE_COEFF_RHO_T 8284.5616
caput H1:CAL-PCALX_FORCE_COEFF_RHO_R 10628.1901
caput H1:CAL-PCALX_FORCE_COEFF_TX_PD_ADC_BG 8.8131
caput H1:CAL-PCALX_FORCE_COEFF_RX_PD_ADC_BG 0.2599
caput H1:CAL-PCALX_FORCE_COEFF_TX_OPT_EFF_CORR 0.9935
caput H1:CAL-PCALX_FORCE_COEFF_RX_OPT_EFF_CORR 0.9946
caput H1:CAL-PCALX_XY_COMPARE_CORR_FACT 0.9991

caput H1:CAL-PCALY_FORCE_COEFF_RHO_T 7114.8962
caput H1:CAL-PCALY_FORCE_COEFF_RHO_R 10588.6196
caput H1:CAL-PCALY_FORCE_COEFF_TX_PD_ADC_BG 17.5161
caput H1:CAL-PCALY_FORCE_COEFF_RX_PD_ADC_BG -0.8150
caput H1:CAL-PCALY_FORCE_COEFF_TX_OPT_EFF_CORR 0.9920
caput H1:CAL-PCALY_FORCE_COEFF_RX_OPT_EFF_CORR 0.9931
caput H1:CAL-PCALY_XY_COMPARE_CORR_FACT 1.0011

A pretty big HAM-ISI model update went in this morning and there is a new HAM ISI overview, first screnshot. There are four ECRs:

https://services1.ligo-la.caltech.edu/FRS/show_bug.cgi?id=26731 & https://dcc.ligo.org/LIGO-E2300014 --- Euler basis drive

https://services1.ligo-la.caltech.edu/FRS/show_bug.cgi?id=22101 &  https://dcc.ligo.org/LIGO-E2200025 --- CPS BLRMS

https://services1.ligo-la.caltech.edu/FRS/show_bug.cgi?id=26050 &  https://dcc.ligo.org/LIGO-T2200406 --- Blend fader

https://services1.ligo-la.caltech.edu/FRS/show_bug.cgi?id=26778 & https://dcc.ligo.org/E2300028 ---- HAM St0 Senscor

The first is adding infrastructure to allow doing Euler basis suspoint drive to the ISI, allowing drives on the table that align with the Euler basis DOFs of the top stage of each of the suspensions. THis is done by loading in the appropriate elements of the suspoint align matrix, now link on the ISI overview under the ST1 BLEND  block. Clicking on the suspoint drive flag will open the filter banks that serve as the inputs for the excitations.

Second ECR is some new blrms code for monitoring cps noise. Still need to think about tests for monitoring.

Third is the biggest part, adding the channel fader blend switching like HAM8 has been using for a while now. This gives much more flexibility with the blend filters, blends can use more than one filter module and there is no settling time now for the blends, because each super-sensor (CPS+seismometer+filters) is being continuously computed. Blend switches now take 7-seconds, instead of 1-2 minutes. This needed some new blend screens (second  & third screenshots). Second is the overview for all the blends, clicking on "engage"  will start the switch to the new blend, so keeping this screen open could be dangerous. I need to add lights to expose more of the filter bank state for each blend. Third screen is the supersensor filter screen, opened by clicking on any of the grey related window buttons next to the blend bank name.

Added a path for sensor correction from the St0 L4Cs on HAM 4-5-7-8. Not commissioned yet, still need time to work on this, so not much to share here, but no part of it is turned on yet.

A similar update is coming for the BSCs next week, learned a lot today, so that should go smoother.