FAMIS tasks 24873, WP 11771

Procedure checklist for both stations completed.  No issues were identified at this time.

MX: Scroll pump hours: 213.5

       Turbo pump hours: 123

       Crash bearings: 100%

Jeff K, Dave B, Oli P

All of the optics (except OFIS, OPOS, and HXDS) were originally built with a DACKILL that will only trip if all of that suspension's Watchdogs trip. This has resulted in some problems (74545, 74622). Because of this, there has been a push to update the logic for each suspension to have an OR instead of an AND, so that the IOP DACKILL trips as soon as any one of the stages trips.
The Watchdogs for these suspensions also rely on an outdated BLRMS to figure out if the stages are saturating, and people have been wanting this to be updated since 2017(FRS9392). This update to the watchdogs would remove the need for a USER DACKILL, so now is a perfect opportunity to do both at the same time! Suspension models created after 2019 (OFIS, OPOS, HXDS, and the new HXTS) were originally made with the updated watchdogs and without the USER DACKILL, and we haven't had any issues. The changes that we made line up with what was done for these previously changed models.
Today we updated the models for the ETMs and TMSs(76305), and we will gradually be updating all of the models.
Changes made:
QUAD MASTER and TMTS Master sus models (QUAD_MASTER.mdl, TMTS_MASTER.mdl):
    - Top level: Removed USER DACKILL and WD block output flags(attachment1)
    - Inside each block with watchdogs: removed the watchdog flags that connect the WD RMS to the DACKILL(attachment2)
    - Inside each WD block within stage blocks: swapped the old WD BLRMS block with the better BLRMSLP block(attachment3)
Individal sus models (h1susetmx.mdl, h1susetmy.mdl, h1sustmsx.mdl, h1sustmsy.mdl):
    - Removed top level WD_2_ISI output flags and connection of WD_2_ISI out to ISI model(attachment4)
Individual sus pi models(h1susetmxpi.mdl, h1susetmypi.mdl):
    - Found out during installation of these updated models that a WD value that we had removed in the individual models went out to the pi models. The sus pi models were updated to include constants as inputs in lieu of the removed WD values(attachment5)
ISI QUAD models (h1isietmx.mdl, h1isietmy.mdl):
    - Since the USER DACKILL connects to dolphin, we also changed the ISI QUAD inputs that feed into SUSWD_2_PAYLOAD from ground to both be 0 for WDMON_STATE_DOLPHIN and WDMON_STATE_DOLPHIN_ERR respectively(WDMON_STATE_DOLPHIN_ERR needs to be 1 because it later gets inverted)(attachment6)
Changes to medms (ETMs,TMSs adl files):
    - Removed the IOP DACKILL button on suspension medm and TO ISI arrow(attachment7)
    - Inside the watchdog subscreens: added in the filter banks for the low pass and removed the diagrams showing the USER DACKILL logic(attachment8).
All changes have been committed to svn.

No SQZ time taken this morning 15:29UTC to 15:51UTC. There was small glitches at 15:35:37 and 15:41:53. Best stretch of time is 15:42:00 to 15:51:00. Took no sqz time with hot OM2 in 74834 / 74640, Cold OM2 Dec 20th 18:00- 18:18UTC 74935.

Followed Naoki's 70642 instructions. Pushed to aligoNB git. There was a small EQ passing through just before I started this. Had ~3dB of constant SQZ.

Last ran and analyzes aligoNB injections with hot and cold OM2 in December 74943 / 74788. We haven't measured coupling of frequency and intensity noise since August 2023: 72140.

Fil, Dave:

I've updated the dewpoint.adl MEDM to show the decommissioning of HAM 4. Fil found yesterday that the Modular Container #2 sensor is misreading and will be replaced.

TITLE: 03/13 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
    SEI_ENV state: USEISM
    Wind: 4mph Gusts, 3mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.64 μm/s
QUICK SUMMARY: H1 has been locked for the past 6 hours, BNS range currently at 135Mpc.

• Wind is low, microseism is elevated
• EDC shows 12 disconnected chanlles all related to nuc28 (currently powered off as the monitors are being replaced)

Very low frequency angular lines are being injected in the OMC ASC controls starting at 14:39 UTC. The longer they run the better, but please feel free to stop them if other tests are ongoing.

awggui's are running on cdsws26.

This is a test to see if we can find a better OMC alignment position, and if this has any effect on DARM noise.

I was able to request and transition to the new DARM state without issue. ETMX still saw a pretty large kick. I'll have to circle back re: how large compared to previous attempts. We will also want to take a look at any effects moving the integrator on L2 LOCK L had.

Note: We do get a few SUS_PI warnings shortly after transitioning to this state.

Quiet time to monitor for non-stationarity:
Start GPS: 1394344327
Stop GPS: 1394346169

Turned off cal lines at GPS 1394346770.39

Elenna started a Bruco after the cal lines were turned off.  here is a screenshot of DARM in the new configuration. 20-90Hz looks high and below 15Hz looks low. It's hard to tell how much of this is real until CAL-CS is calibrated & that calibration is propagated to the DTT template.

I started an L2 LOCK IN1/IN2 injection using noise recorder at 1394347034.426. We used a tuned broadband measurement that Craig and I put together. Apparently it was too strong because we lost lock from this injection. It also tripped EX. I requested DOWN and the EX trip alarm reset. 

I tried a new DHARD Y filter that should reduce the noise injected from DHARD Y between 10-30 Hz. It's about 6 dB improvement. I ran 10 minutes quiet DARM after the test starting Mar 13 4:36 UTC (Mar12 21:36 local). No apparent change in the low frequency noise in DARM.

The filter engagement works, so I added it to the "LOWNOISE ASC" state of the guardian. It's in FM8 and engages with other DHARD Y boosts.

Gabriele ran a bruco for me, looks like this was an improvement-

before dhard y change bruco

after dhard y change bruco

[Gabriele, Elenna]

Since we saw high coherence betwen DHARD_Y and DARM, we tried to improve the A2L. In summary, the current value of 1.65 is the best for angular coupling at ~30 Hz, so we didn't change it.

To do the measurement, we injected two lines, one at 15.1 Hz and one at 30.3 Hz, and used a python script to demodulate the lines in DARM. The lowest coupling at 15.1 Hz is obtained for a larger value around -2.4, but this makes the coupling at 30.3 Hz larger. The initial value is the best for minmize the coupling at 30 Hz, where DHARD is limitign DARM.

As a part of several suggestions to address the clipping on IM4 trans and the ISS QPD, Keita suggested I check the POP QPDs to see if things have changed there since the vent. Here are two ndscopes comparing the PIT, YAW and NSUM channels of both POP A and POP B QPDs. The two screenshots show the last lock in January and the recent lock yesterday. It appears that the overall power on the QPDs is about the same, but the alignment on POP B in particular has changed significantly in pitch, close to -1 and probably to falling off.

We servo on POP A during DRMI lock, but do not use POP B for any control (to my knowledge).

[Evan, Gabriele, Louis, Georgia]

We struggled to properly align the IFO. We saw low flashes in the y arm during ALS locking and had to move the beamsplitter to increase the flashes. Then, after completion of initial alignment, the ALS Y arm alignment would be poor again. After some trial and error and fighting with the "increase flashes" guardian state, Evan saw that PR3 has drifted in yaw 0.4 microradians. When he moved PR3, it significantly improved the Y arm flashes and the input alignment. We don't know why PR3 drifted and it had so much effect on the Y arm but not necessarily the X arm. Our third initial alignment succeeded and locking is proceeding well.

General Compute work successful and complete. Split the fibre optic pair that was a single pair - 2 strand connection to ESNet and split the fiber to 2 single strand full duplex circuits to ESNet. We now have BGP Peer redundancy and Seattle route as preference. We are using BiDi connectors now over standard SFP+. One route to Boise and one route to Seattle. Seattle preferred due to latency, Boise is redundant. Duration of total outage was approximately 30 minutes, and duration to full restoration was about one hour. 

1500: Work Started 
15:15 Internet connection cut
15:45 Initial restoration achieved, but not the Seattle link.
16:15 Seattle link is restored when fiber patch found incorrectly connected. This was corrected and full circuit restoration was seen.
16:30 All secure from this evolution. Hands off all systems check shows complete operation, BGP peering to Seattle that is established, and IP Flows are in the Seattle route destinations.

TITLE: 03/12 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
INCOMING OPERATOR: None
SHIFT SUMMARY: Tuesday maintenance today, mostly a lot of VAC and CDS work. We has issues getting ETM and TMS model working(76305), but we were finally able to get them up and are currently working on locking green arms.
LOG:

15:00UTC Starting Tuesday maintenance
15:38 Turned sensor correction off                                                                      

J. Kissel, O. Patane, D. Barker
ECR E1700387
IIET Ticket 9392
WP 11743

More details to follow in the comments, but today started the campaign of SUS / SEI watchdog improvements. This required us to restart the
    h1susetmx
    h1susetmy
    h1sustmsx
    h1sustmsy
    h1susetmxpi
    h1susetmypi
    h1isietmx
    h1isietmy
front-end models.

The update makes the following changes:
    (1) Establish a sensible band-limited RMS user watchdog trigger (where it had been some old broken RMS code prior), where the OSEM trigger signals are now calibrated into microns (um_RMS), and the threshold has been adjusted to match it.
    (2) Entirely removed the USER DACKILLs from the SUS, so we no longer AND together the output triggers for any stages' user watchdog and shut down the output of DAC. In other words, we'll no longer kill damping and offsets if we drive the L2 stage too hard with ISC control.
    (3) Entirely removed the connection between the ETM and TMS DACKILLs to the ISI and PI models, and replaced the input to the ISI's PAYLOAD flag with "always good" constants.

As originally promised in ECR and IIET ticket -- this drastically reduces the unnecessary complexity and redundancy of the watchdog system, it now behaves in a much less trigger-happy fashion, and delightfully, we now can answer the question "how much motion trips the watchdog?"

I've accepted the threshold at 25 um_RMS over the 0.1 to 10 Hz band. We can and should adjust this threshold as we gather more experience.

All changes to settings have been accepted in the corresponding SUS safe.snap such that the work in calibrating the OSEM trigger signal won't disappear with the ISC_LOCK guardian's SDF_REVERT state.
    

Sheila, Camilla, Jennie W, Keita remote

Stefan and Daniel suspect that our excess noise around 100 Hz might be due to intensity noise, and we did have a large shift in alignment of the beam transmitted through IM4 (76291) 76241.  I moved pico 1, first to center the beam on the ISS QPD, which made the power on the ISS array PDs drop, and didn't improve the spectrum of the ISS Sum inner or outer channels (we did this with the loop open).  We reverted this, then looked at the QPD position in O4a when we had 60W input power, went to 60W input power and pico'd to bring the beam to the same location as O4a.  This also didn't improve the spectrum, so we brought the pico back to where we started.