J. Kissel, T. Shaffer

As a continued parts of the recovery, even after Tony reset the alignment sliders for all suspensions to a time prior to the dolphin crash, we continued struggling to align the SRC. Trending the OSEMs, we found that OM2 had particularly different align than the beginnings the last successful lock acquisition attempt. OM3 also showed a bit of drift due to the temperature excursion. 

So, we've now restored the alignment to the OSEM values to 2023-05-10 23:11 UTC, and initial alignment seems to be going much better now.

(The attached trend shows pitch on the left, and yaw on the right, for columns of OM1, OM2, and OM3. The horizontal cursor Y1 is set to the 2023-05-10 23:11 UTC values in each, and the Y2 cursor is set to what the values were just before this fix.)

FIXED IT.

J. Kissel, E. Capote, T. Shaffer

We're struggling through alignment of the SRC during lock acquisition, and we looked through some trends of SR3 alignment and we see that it's alignment is quite off in Yaw. TJ said "Oh, well, yeah, the temperature in the VEA has gone bad. That's normal." 

Trusting but verifying, I trended the average of zones in the LVEA, and I disagree -- there as a marked step-change in the LVEA temperature starting at 2023-05-10 13:45 PDT (20:45 UTC), which, over the course of a day has increased the output arm's temperature by 1 deg C, or ~2 deg F.

Perhaps a set point got lost?

J. Kissel

I blame the dolphin crash last night LHO:69492, but I saw the that PCALX RX PD showed some excess noise up on the front wall FOM so I've open and re-closed both PCALX and PCALY optical follower servo (OFS) enable switch.

FIXED IT.

PCALs functional again as of May 11 2023 18:17 UTC.

J. Kissel

While chasing other SDF issues, I found the yellow warning lights on the SDF overview screen indicating that there were front-end models had some channels that were either new and not yet monitored, or removed and no longer found. 
Those models were 
    - h1sqz, h1ioppsl
    - h1pemcs
which, it seems to correspond exactly with what front-end models were changed this morning -- see LHO:69502.

I've cleaned them up -- all new channels are now monitored, and removed channels have now been purged from the SDF safe.snap.
we may have to run through this again once the SDF comparison switches over to the OBSERVE.snap.

Tony had a rough night after teh Dolphin crash. I started by trying an initial alignment after we did some opportunistic model restarts and DAQ restarts. We've been having issues with the XARM_IR state recently, and Jeff came in and immediately guessed what it was (alog69499) and when we reverted to pre May 5th settings, it locked right up. The rest of initial alignment was straight forward. I did have to moce SRM quite a bit for the SRC step to increase H1:H1:ASC-AS_A_DC_NSUM highs and lows as usual procedure.

Since initial alignment the IFO has been catchin DRMI in an awkward mode and I haven't been able to fix it in time. We've now locked a PRMI to help with the BS, and while that helped it still didn't fix DRMI enough. At this point there must still be a misalignment in the output arm.

Thu May 11 10:05:03 2023 INFO: Fill completed in 5min 2secs

Gerardo confirmed a good fill curbside

Mini-Maintenance Morning.

WP11183 h1omc remove unused ADC channels

Daniel, Dave:

A new h1omc model was installed. The old model was reading some ADC channels and immediately terminating them. These have been removed. No DAQ restart was required.

WP11188 PSL FSS add NPRO TEMP fast channel to DAQ

Jason, Robert, Adrian, Dave:

The PSL FSS common model (pslfss.mdl) was changed to add the output of the NPRO_TEMP filter module to DAQ at the lowest rate of 256Hz. DAQ restart required, but since this is just an INI file change I did not need to restart the model, I did a DAQ LOAD instead.

++: fast channel H1:PSL-FSS_NPRO_TEMP_OUT_DQ added to the DAQ

WP11190 PEM add HAM7 FC1 ACC to h1pemcs and DAQ

Robert, Adrian, Fil, Dave:

A new h1pemcs model was installed. We have repurposed 3 channels from the 6th ADC to read the ACC_HAM7_FC_[X,Y,Z] channels. These were originally read by generic filter-modules and not acquired by the DAQ.

The X, Y, Z channels are being acquired by DAQ at 8kHz.

Latest SQZ models

Daniel, Dave:

Daniel's latest h1ioplsc0 and h1sqz models were installed. DAQ restart was required.

DAQ Restart

Dave:

The DAQ was restarted to support the above changes. FW0 was restarted at 08:55 and at 09:00 when it was attempting to write all three types of frame file its daqd crashed and restarted. It has been stable since.

GDS0 needed a second restart. 

No problems with the 1-leg restart.

R. Kumar, J. Kissel, T. Shaffer

TJ let me know that Rahul found that the RT and SD OSEM sensors have died on Jan 16 2023 08:15 am PST (16:15 UTC). Thankfully, we don't use any alignment offsets on the OSEMs, so this will have only impacted the damping loops.

The HAM56 wiring diagram (D1002740) suggests that these two OSEMs are on the same signal chain, and alone on that signal chain, so this should be a relatively easy swap.

J. Kissel, T. Shaffer

I came in this morning and TJ was having trouble with initial alignment, and having trouble in the same steps -- ALIGN_IFO guardian's XARM IR LOCKED state -- that he says operators have been struggling with "all week." Namely, the ALIGN_IFO / INIT_ALIGN guardians would suggest that the XARM was locked, but it was not -- it would trigger but not hold. The practice in that past week was to "just skip" ALIGN_IFO guardian's XARM IR LOCKED state, where we use POP_A_45 to lock the XARM. 

I connected the dots between "trouble with POP" and us needing to decrease the POP_A_45 analog gain to allow for an increased SRCL offset during normal, nominal low noise last week LHO:69350.

As such, TJ and I tried reverting the analog gain, H1:LSC-POP_A_RF45_WHITEN_GAIN, to +21 dB, and then compensated the analog gain increase with the corresponding digital gain compensation -- switching back to FM4 in the H1:LSC-POP_A_RF45_I and H1:LSC-POP_A_RF45_Q banks.

Reverting to higher gain immediately restored the functionality of the XARM IR LOCKED.

FIXED IT.

So, we'll code this adjustment up in the initial alignment guardian sequence.

TITLE: 05/11 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Aligning
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 8mph Gusts, 6mph 5min avg
    Primary useism: 0.06 μm/s
    Secondary useism: 0.20 μm/s
QUICK SUMMARY: Been locking on PRMI for a large chunk of the night, but the output arm must be off. Since we aren't locking well at the moment we are going to opportunistically restart some models that have been pending.