WP11540 Remove SDF safe->OBSERVE exceptions for h1seiproc and h1brs

TJ, Jim:

TJ made changes to guardian to remove the exception that h1seiproc and h1brs do not transition to OBSERVE.snap. h1brs OBSERVE==safe, h1seiproc has a separate OBSERVE.snap which Jim will verify is correct.

WP11546 New FMCS STAT code base

Erik:

Erik rewrote the FMCS STAT IOC to use this new softIOC python module. The code was restarted.

New Guardian node

Camilla, Dave:

Camilla started a new Guardian nodel called SQZ_ANG_ADJUST. I updated the H1EPICS_GRD.ini file. DAQ+EDC restart was required.

DAQ Restart:

Jonathan, Erik, Dave:

The DAQ was restarted to include the new GRD channels into the EDC. This was a good restart with no issues.

cdsioc0 reboot

Erik, Dave:

Erik updated and rebooted cdsioc0. There were some issues getting Picket Fence running again, which Erik resolved. We were reminded that I was running a temporary HWS ETMY IOC in a tmux session, Erik switched this over to a systemd service via puppet.

Jordan

We ran the functionality test on the main turbopumps in MY and EY during Tuesday Maintenance (11/28/23). The scroll pump is started to take pressure down to low 10^-02 Torr, at which time the turbo pump is started, the system reaches low 10^-08 Torr after a few minutes, then the turbo pump system is left ON for about 1 hour, after the hour the system goes through a shut down sequence.
 

MY Turbo:

Bearing Life:100%

Turbo Hours: 208

Scroll Pump Hours: 74

EY Turbo:

Scroll pump made a grinding sound after getting to ~ 5E-2 Torr. I closed all valves and stopped the test. The scroll pump only has 200 hours on it so it will be disassembled to figure out the source of the noise. I have swapped the scroll pump with a new ISP250, but did not have time to run the turbo test. I will resume next tuesday and add a comment to this alog with the EY results.

Closing WP 11544 and FAMIS 24917

F. Mera, D. Sigg, M. Pirello

Per WP11548:

The Squeezer 4 rack was modified duplicating the EL3692 terminals in the positions M13, M14, M19 and M20. Wiring was modified to have readings in CH1 only.

TwinCAT solution was updated accordingly, the system was restarted and succesfully verified with the new configuration.

SQZ 4 rack serial: S2101205

Clean room over Ham 6 lights were turned off
Wifi was turned off at the corner and both arms, except MSR.

LVEA has been swept.

Tuesday maintenance activities are wrapping up - locking thereafter.

• Tony doing LVEA Sweep
• Gerardo wrapping up FCTE valve work (in contact)

Fil Marc Daniel

Today we pulled the lond d-sub cables going from the CER to the squeezer racks. This completes the electronics installtion for the SQZ PMC with the exception of the on table devices.

Closes FAMIS 25967, last checked in alog74134

BSC:

Script reports the following are elevated: ITMX_ST2_CPSINF_{H3, V1, V3 }

ITMY_ST1_CPSINF_{V2, V3}, ETMX_ST1_CPSINF_V2 are elevated at high frequency

HAM:

The HAMs look fine

Workstations were updated and rebooted at about 1500 UTC.  This was an OS update.  conda packages were not affected.

Tue Nov 28 10:03:40 2023 INFO: Fill completed in 3min 37secs

TC-A min = -90.1C

TITLE: 11/28 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 3mph Gusts, 2mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.19 μm/s
QUICK SUMMARY:

IFO was locked (but not observing due to Tues magnetic injections). Sheila unlocked IFO through planned DARM configuration change.

PSL 102 dustmon is still stuck and giving warning: "dust counts did not change please investigate". See Oli's eve (11/27) summary alog 74421 to follow the story on this.

TITLE: 11/28 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Earthquake
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY: Relocked twice today, both locklosses (74432, 74438) being due to earthquakes. Had some issues with the first relocking, needing to go to GREEN_ARMS_MANUAL before running the initial alignment, and after still going through PRMI. Second relocking I also needed to touch both arms and we ended up going through CHECK_MICH_FRINGES on the way up. We are currently at LOWNOISE_COIL_DRIVERS. Other than that, everything was relatively normal.

We actually just got a verbals call out for an incoming earthquake from Fiji that is in the middle of the green zone on the EQ Response Zones script so hopefully we will be able to stay up!
LOG:

00:00UTC Detector holding in DOWN due to earthquake

00:40 Started Locking
    - Needed to align ALSX manually, then on to an Initial Alignment
    - 01:30 Initial Alignment completed, on to Locking

02:22 NOMINAL_LOW_NOISE
02:25 Observing

06:30 Earthquake mode activated
06:31 Lockloss due to earthquake                                                                                                                                                            

Lockloss 11/28 @ 06:31UTC due to earthquake from Alaska

Link to full report

Summary of the report:

• observing for 74.8% of the week
• many summary page plots are not displaying properly for this week, especially the glitchrate plots and hveto
• range was mostly around 150-160 Mpc except for fuziness on Thursday and Friday. A cause was not identified.
• there were a couple of locklosses with unidentified causes. EQs, wind, and microseism may be factors.

Observing at 160Mpc and Locked for almost 2 hours now

TITLE: 11/28 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
    SEI_ENV state: EARTHQUAKE
    Wind: 1mph Gusts, 0mph 5min avg
    Primary useism: 0.32 μm/s
    Secondary useism: 0.21 μm/s
QUICK SUMMARY:

Detector went down due to an earthquake(74432) and we are currently having another earthquake pass through so we will be holding in DOWN for a bit longer.

Sheila, Camilla, Jenne, Austin

We droped out of observe for a few minutes to start running the sqz angle adjustment guardian state using the ADF that Naoki wrote on Wed (74376).  This changed the sqz angle by a few degrees in the first 2 minutes it was on, which improved the sqz blrms slightly.  

Because Vicky put some code in ISC_LOCK that set the SQZ_MANAGER request to it's nominal state, the only change we had to make was to change the nominal state of SQZ_MANAGER, and this should now be used in future locks. The new SQZ_MANAGER state is ADJUST_SQZ_ANG_ADF state number 110

TITLE: 11/23 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 157Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:

The last hour has shown a drops in range while being in OBSERVING.  In the last 30min the range drops have been less severe, but this is an odd one.

Had a few bumps out of observing due to the squeezer, but it would only be for about ~2min as the squeezer relocked itself.  L1 came back to Observing this shift.
LOG:

• Drops from OBSERVING due to the Squeezer unlocking this shift:
  • 2356 - 2359 utc (end of Ryan C.'s shift)
  • 0004 - 0006 
  • 0624 - 0626
• 0244 Short GRB alert
• 0712 Short GRB alert
• 0700 H1 range begins taking a hit with drops and variations.  Have not received any alarms for this & have been in OBSERVING the whole time. Environmentally we look good at first glance.

WP 11533

Checked the HAM4 ISI Coil Driver Chassis. Issue reported last week of noisy fan. Fan is spinning and noise reported last week has not returned. Will leave WP open another week.

Artem, Gabriele, Sheila, Louis

Plots and Jupyter notebook with these results are also available here.


 ESD quadratic noise 

This is a follow up on investigation of potential non-stationary noise at Handford introduced by the ESD drive, reported in 73913 and previous entries linked from there. The idea is that from the ESD signal, Vs, and bias, Vb, voltages we can reconstruct ESD force applied to the mirror, using equations in LIGO-T1700446. The force component linear in Vs gets subtracted by the control loop and therefore does not affect DARM. But the force component quadratic in Vb and both linear and quadratic in Vb can couple to DARM. Therefore in this report, we are calculating respective force components and projecting them to DARM. Then respective ASDs and spectrograms are compared.

 Selection of input channels 

The ESD drive voltages are recorded by Monitors, H1:SUS-ETMX_L3_LVESDAMON_LL_OUT_DQ for Vs and H1:SUS-ETMX_L3_ESDAMON_DC_OUT_DQ for Vb. Important note here: we initially tried channel H1:SUS-ETMX_L3_ESDAMON_UL_OUT_DQ to get Vb, but it looks like it's not connected to anything and recording ASD noise... The H1:SUS-ETMX_L3_ESDAMON_DC_OUT_DQ seems to be fine. Also, LIGO-T1700446 specifies calibrations for voltage channels (equations 13 and 16), but currently the channels mentioned above are already calibrated to voltage.



 Calculating force terms 

Calculation is based on equation 3 in LIGO-T1700446, but this equation is modified in the following way:

Bias voltage is decomposed into DC and varying part, to avoid propagating DC term through the calculation (DC term will not affect DARM). So the following substitution is applied: Vb = Vb_DC + dVb. The term Vb_DC is simply median of Vb time series, and dVb is Vb time series with median subtracted
Brackets are then opened to gather all the terms linear in Vs, quadratic in Vs, linear in dVb, quadratic in dVb, and mixed dVb*Vs term. Noise term is then quadratic in Vs +  linear and quadratic in dVb + mixed dVb*Vs


Here are the new equations:

# new equations with dVb and rearrange terms
F_quadratic_Vb_term = (alpha+gamma)*(dVb_ts**2)
F_quadratic_Vs_term = (alpha+gamma)*(Vs_ts**2)
F_mixed_term = (gamma-alpha)*2*dVb_ts*Vs_ts
F_linear_Vb_term = (alpha+gamma)*2*Vb_DC*dVb_ts + beta2*(dVb_ts+Vb_DC)  + beta*(dVb_ts+Vb_DC)
F_linear_Vs_term = (gamma-alpha)*2*Vb_DC*Vs_ts  + beta2*(-Vs_ts) + beta*(Vs_ts)
F_total_term = F_quadratic_Vb_term + F_quadratic_Vs_term + F_mixed_term + F_linear_Vb_term + F_linear_Vs_term 

# noise term has everything _linear_ in Vs and both _linear and quadratic_ in Vb
F_noise_term = F_quadratic_Vb_term + F_quadratic_Vs_term + F_mixed_term + F_linear_Vb_term

Coefficients alpha, beta, beta2 and gamma are taken from this measurement: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=56613.



 Results: 95%CL ASD and spectrograms for ESD noise converted to equivalent strain 


Force is converted to acceleration by dividing by the mirror mass (40 kg).
Subsequently acceleration is divided by omega^2 to convert to displacement.
Finally displacement is converted to equivalent strain by dividing by arm length (4 km). Now the spectrum can be compared to DARM channel.





 Replacing Vb with white noise at typical level, to avoid projecting Monitor's ADC noise 

Gabriele made a plot showing ASD of Vb for different levels of bias and when the bias is off. What we can see is a shoulder of additional noise which we believe is coming from ADC of the monitor. To get rid of that, we generate white noise time series for dVb at the level of Vb where it is not dominated by additional noise. The new dVb term is then plugged into the equation for the force term and a new set of plots is produced.





 Results: 95%CL ASD and spectrograms for ESD noise converted to equivalent strain (dVb replaced with white noise - see above)