TJ, Sheila

TJ is having difficulty locking in the high winds this afternoon.  We've been running without DRMI ASC other than the BS, so that operators have to manually adjust PRM and SRM sometimes.  This afternoon we added back in INP1, PR1 and PRC2 loops, but left SRC ASC off.  This seems to be helping with POP18, although the operators may still need to adjust SRM by hand.

Maintenance activies completed around 30 min ago. I ran an initial alignment at 1100 PST, then started locking at 1206 PST. The wind is really starting to pick up and we just lost lock at the state Resonance. We will keep trying but this might be a tough relock.

Since it hasn't been done since before the vent, and the output of the PMC has continued to slowly drop, before locking today I calibrated the PSL rotation stage following the steps in alog79596.

The measurement file, new calibration fit, and screenshot of accepting SDFs are all attached.

Now that CIT's ldas-pcdev2 machine is back up following last week's data center troubles, my CW hardware injection monitoring shows no apparent signals since the return to observing mode on Saturday. Is that by design, or did the injections restart fall through a crack? Thanks.

J. Oberling, R. Short

The FSS RefCav TPD had been trending down again, and Ryan did not get enough improvement from a remote alignment tweak last week.  So today we went into the enclosure to do an on-table alignment of the FSS RefCav beam path.  As usual, we began with a power budget:

• FSS In: 282.9 mW
• AOM In: 283.2 mW
• AOM Out, SP: 174.4 mW
  • SP Diffraction Efficiency: 61.6%
• AOM Out, DP: 122.8 mW
  • DP Diffraction Efficiency: 70.4%
• EOM Out: 121.4 mW

No, we're not getting magic amplification between the FSS In and AOM In power measurements; that kind of small discrepancy is normal with the small stick power meter head we use for this alignment (they are also very AOI dependent, so it's entirely possible we had the head better aligned to the beam for the AOM In measurement).  Immediately we see the single-pass diffraction efficiency is lower than it normally is (generally hangs out in the lower 70% range these days), so we began by adjusting the AOM alignment to improve it.  This also means we have to then adjust M21 to improve the double-pass diffraction efficiency.  After the adjustments we had:

• AOM Out, SP: 199.4 mW
  • SP Diffraction Efficiency: 70.4%
• AOM Out, DP: 143.3 mW
  • DP Diffraction Efficiency: 71.9%

This is the lowest we've seen the diffraction efficiencies of this AOM, and suggests maybe it's time to swap the AOM for a new one (pretty sure this is the same AOM installed with the PSL in 2012, would have to go back through the alog to see if it's been swapped since install).  As usual, we had to do some small tweaks to the FSS EOM (provides the 21.5 MHz PDH sidebands for RefCav locking), and had to tweak the beam alignment into the RefCav using the input iris.  Once done, the RefCav locked without issue and our picomotor mounts were used to maximize the RefCav TPD; we began with a TPD of 0.57 V and ended with a TPD of 0.82 V.  We've had it higher in the past, but we have less power out of the PMC than usual (due to our slowly increasing PMC Refl that we have yet to figure out the cause of), and less power out of the PMC means less power available for the FSS RefCav which means lower maximum RefCav TPD.  To end we realigned the beam onto the RefCav RFPD and took a visibility measurement:

• Unlocked: 1.162 V
• Locked: 0.221 V
• Visibility: 81.0%

This finished the FSS RefCav tune-up, so we left the enclosure.  We left the ISS OFF while the enclosure returns to thermal equilibrium; it will be turned back ON once that equilibrium is reached, and Ryan will do a rotation stage calibration.  This closes LHO WP 12057.

WP12061 Upgrade RCG h1susex, remove LIGO-DAC delays

EJ, Erik, Jonathan, Dave, Daniel, Marc:

h1susex RCG was upgraded to a custom 5.3.0 specifically for the IOP to remove a delay in the new LIGO 28AO32 DAC. We compliled all of the user models with this RCG as well.

Our first restart was a simple model restart, but we got Dolphin errors. So our second restart was to fence h1susex from the Dolphin fabric and power cycle h1susex via IPMI. After this the models started with no issues.

No DAQ restart was required.

Code path is /opt/rtcds/rtscore/advLigoRTS-5.3.0_dacfix

WP12063 Alert System

Dave:

The locklossalert system code was modified to permit an alert window which spans over midnight, needed for the new owl shift hours.

Note that the business/every day filter is applied after the minute-in-day filter, so a window starting Friday evening and extending into Saturday morning will cut off at midnight if business days only is selected.

One other change:

For everyone subscribed to Guardian alerts, if for any reason the Guardian EPICS record cannot be read (node down, h1guardian1 down) the alert will now default to SEND.

Guardian Reboot

TJ:

h1guardian1 was rebooted at 08:21 PDT. All nodes except TEST came back automatically. TJ worked on TEST and got it going.

MSR Rack Cleanup

Jonathan:

Jonathan removed two test switches from the MSR racks which are no longer needed.

I removed two switches from the h1-daq-1 rack that had been put in for testing.  They are no longer needed in racks.

I also took a moment to sort and clean up a number of SFP transceivers that had been left out on the MSR work area.

Tue Aug 27 08:08:21 2024 INFO: Fill completed in 8min 17secs

Jordan confirmed a good fill curbside.

TITLE: 08/27 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 6mph Gusts, 3mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.09 μm/s
QUICK SUMMARY: Locked for 2.5 hours, seems like an automated relocked before that. PEM magnetic injections are running now. Planned 4 hours of maintenance today.

Workstations were updated and rebooted.  This was an OS packages update.  Conda packages were not updated.

TITLE: 08/27 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY: Observing and have been Locked for 5.5 hours. Quiet evening with nothing to report. Feedforward measurements were run, and I also just tuned squeezing up.

Little comparison on the damping for new PI over the past 5 days (ndscope)
Since starting back up last week, we have been seeing a new PI (79665), and it had caused a few locklosses. On Friday afternoon (08/24 ~00:00UTC - crosshairs on ndscope) it was decided that the easiest way to try and control it for now would be to change our input power from 60W to 61W. This, along with some changes to the damping gain, has helped keep it under control. Comparing the PI's behavior to the length of our lock stretches, we only see the ringups during the first 1.5-2 hours of each lock, so once we're thermalized past a certain point, they stop happening. The changes to damping (I'm not sure what was done besides raising the damping gain) also means that they get caught and damped quickly.
 

LOG:

23:30 Observing and Locked for 2 minutes
04:27 Dropped Observing to take FF measurements
04:46 Feedforward measurements done, running SQZ alignment
04:55 Back to Observing

TITLE: 08/26 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 9mph Gusts, 6mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.07 μm/s
QUICK SUMMARY: Observing and just got to NLN a few minutes ago. Once we're thermalized I'll be taking LSC FF measurements.

TITLE: 08/26 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 139Mpc
INCOMING OPERATOR: Oli
SHIFT SUMMARY: The shift was mostly taken up by commissioning time. There was recovery from a CDS crash this morning, but recovery was pretty straight forward. The reacqusitions from the two lock losses one needed an initial alignment, the other did not. Minor interventions for both.
LOG:

• 1744 Back to low noise, time for commissoining
• 1827 Lock loss
  • Tried relocking without an initial alignment, didn't work. Fully auto IA and main locking
• 2059 Observing
• 2159 Lock loss
  • No initial alignment, but I held at Locking_Arms_Green because it would move to the next state before the WFS would converge and cause a lock loss. We then lost lock at Engage_ASC_For_Full_IFO, and the next time it made it up on its own.                             

I restarted the 4 SUS in lock charge nodes and the PEM magnetic injection node to reestablish the connection to the front ends that can go away after front end crashes, like the one this morning ( alog79708). We wanted to be prepared for these to run tomorrow morning if the IFO is locked.

FAMIS 27796

• TCSx chiller: Level of 29.8; did not add water
• TCSy chiller: Level of 9.8; did not add water
• No water seen in small cup under the "slow leak"
• Values updated in T2200289

Before the vent, we had lowered the DARM offset used at the end of the DARM_OFFSET state for locking the OMC since we had seen the PRG fall off and cause a lockloss with the nominal offset of 9e-05 (see alog79082 for details). When locking this afternoon, I raised the offset from 6e-05 back to 9e-05 after running through DARM_OFFSET, and seeing that the PRG didn't plummet and cause a lockloss, we continued locking. The OMC locked on the first try, something that hasn't been the case recently, so having more carrier light there seems to help. I compared OMC scans from this lock against the last lock, which used the lower DARM offset; attachment 1 shows the scan with the higher offset and attachment 2 with the lower offset. According to the OMC-DCPD_SUM channel, we get ~10mW more carrier light on the OMC when locking with this higher DARM offset.

I've put this DARM offset of 9e-05 back into ISC_LOCK and loaded it. We can watch over the next couple of lock acquisitions to see if the problem wth the PRG dropping off resurfaces.

TJ, Jonathan, EJ, Dave:

Around 01:30 this morning we had a Dolphin crash of all the frontends at EY (h1susey, h1seiey, h1iscey). h1susauxey is not on the Dolphin network as was not impacted.

We could not ping these machines, but were able to get some diagnostics from their IPMI management ports.

At 07:57 we powered down h1[sus,sei,isc]ey for about a minute and then powered them back on.

We checked the IX Dolphin switch at EY was responsive on the network.

All the systems came back with no issues. SWWD and model WDs were cleared. TJ is recovering H1.

We can edit this list as needed. 

We are having trouble locking this afternoon because of the wind, but we have some requests for the operators this weekend if they are able to relock. 

We've changed the nominal locking power to 61W, in the hopes that this might avoid the PI or let us pass through it quickly.

When we first get to NLN, please take the IFO to NLN_CAL_MEAS and run a calibration sweep.  If we stayed locked for ~1.5 hours, please re-run, and if we are ever locked for more than 3 hours please re-run again. 

After the calibration has run, we would like to check the camera set points since we have seen earlier today that they are not optimal and that might be related to our PI problem.  We already updated the offset for PIT1, but we'd like to check the others.   We modified the script from 76695 to engage +20 dB filters in all these servos to speed up the process.   Each DOF should take a little more than 15 minutes.  We'd like these run with the POP beam diverter open, so we can see the impact on POP18 and POP90.  The operators can run this by going to /ligo/gitcommon/labutils/beam_spot_raster and typing python camera_servo_offset_stepper.py 1 -s now  (and once 1 completes, run 2 and 3 if there's time.)

I've added 3 ndscope templates that you can use to watch what these scripts do.  the templates are in userapps/asc/h1/templates/ndscope/move_camera_offsets_{BS, DOF2_ETMX, DOF3_ETMY}.yml  We'd like to see if any of these can increase the arm powers, or POP18.  If there is a better value found, it can be set to the default by updating lscparams.py lines 457 to 465.