TITLE: 09/13 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 146Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 10mph Gusts, 8mph 5min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.14 μm/s
QUICK SUMMARY:

IFO is in NLN as of 7:24 UTC (16 hr lock) and OBSERVING (after Wednesday commissioning) as of 23:12 UTC

TITLE: 09/13 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY: Locked for 15.5 hours, we just finished up 4.5 hours of commissioning and calibration time.
LOG:

• 1830 Out of observing for planned commissioning
• 2312 Observing

J. Kissel (after consult with J. Driggers)

While poking around the front-end computed response function systematic error yesterday and today (LHO:72848 and LHO:72863), I was reminded that one of the calibration lines used to measure the response function systematic error has been showing completely bogus large numbers for "a long time."

It's the 102.13 or 104.23 Hz PCALX calibration line.

I realize the systematic error measured at this frequency is wrong because we never updated the frequency in the DARM loop model parameter file. 
(See details below as to why this matters for both GDS and CALCS computation of the response function systematic error at this frequency.)

For reasons I outline below, it's easier to fix the problem by changing the calibration line frequency *back* to 102.13 Hz, so I've done so as of 2023-09-13 20:17:45 UTC, and the error estimate has returned to functional values around unity magnitude and zero phase by 2023-09-13 20:19:45 UTC (i.e. after the 2-minute gating, medianing, averaging algorithm catches up.)

All of this was done while we're were *out* of observing during today's commissioning window, so the *next* observation ready segment will pick up this change around 2023-09-13 ~23:00 UTC.

%%%%%%%%%%
Details & Motivation
%%%%%%%%%%

2023-06-21 Calibration updates to reflect power down to 60W PSL input, 20230621T211522Z pydarm_H1.ini file correctly has 102.13 Hz in the list of cal_line_sys_pcalx_frequencies. LHO:70693

2023-08-04 Friday afternoon, SRCL Feed-forward filter re-tuned, informed by a measurement that mistakenly has all calibration lines still ON, including 102.13 Hz line. So, FF measurement fitter installs a super high Q zero:pole pair to "compensate" for it. The resulting filter is installed in the SRCL FF path, starting a period of huge interference between this high-Q zero:pole pair in the SRCL FF and the 102.13 Hz calibration line itself. LHO:71961

2023-08-08 Next week Tuesday, DetChar Team finds "some feature" at 102.12833 Hz, and folks blame the calibration line itself since it's originally identified as appearing Saturday morning. LHO:72064

2023-08-09 That same week Wednesday, out of panic and as well as starting a "grasping at straws" test, we moved the 102.13 Hz PCALX calibration line to 104.23 Hz LHO:72108 -- but in our rush, forgot/didn't realize we needed to update down-stream computation that is impacted by this change:
    (a) The front-end demodulator that turns this line into something useful in CAL-CS needs its local oscillator frequency to change to match,
    (b) The cal_line_sys_pcalx_frequencies list within pydarm_H1.ini needs updating as well -- or else the "DARM loop model transfer function values at calibration line frequencies" EPICs records are wrong for the new frequency, which means both GDS and CAL-CS are interpreting this line incorrectly.

2023-08-29 Problem with SRCL FF filters is finally identified and removed, and the story about what happened on 2023-08-04 finally becomes clear. LHO:72537

2023-08-31 Calibration updated in order to account for the past two month's worth of commissioning and ETMX ESD actuation strength drift, but the 20230830T213653Z pydarm_H1.ini file incorrectly still has 102.13 Hz in its cal_line_sys_pcalx_frequencies list. LHO:72594
    >> I didn't aLOG it explicitly, but on this day, I noticed the (a) mistake above, and changed the CAL-CS DEMOD local oscillator frequency to 104.23 Hz for the front-end demodulation and estimation of systematic error. Because (b) was still going on, it didn't fix the problem.

2023-09-13 TODAY Realized the issue with front-end systematic error calculation was still wrong because of (b); the cal_line_sys_pcalx_frequencies list in the pydarm_H1.ini file, and reasoned it would be way easier to change the PCAL OSC and DEMOD OSC frequency back to 102.13 Hz than to go through the entire rigamarole of updating the calibration and "pushing new EPICs records." So, I moved calibration line *back* to 102.13 Hz changing the PCALX OSC7 frequency, and remembered to update the LINE8 DEMOD LO frequency to match.

First attachment: The graphical MEDM interface of where I changed the EPICs records that define the PCAL OSC7 calibration line frequency, and the CALCS LINE8 DEMOD LO frequency.
Second attachment: The PCAL OSC7 frequency change has been saved in the H1CALEX SDF system. (Only the OBSERVE.snap is show, because the h1calex model's safe and OBSERVE.snaps are the same file soft-linked together.)
Third attachment: The CALCS LINE8 DEMOD LO frequency change to match has been saved in the H1CALCS SDF system. (Only the OBSERVE.snap is show, because the h1calcs model's safe and OBSERVE.snaps are the same file soft-linked together.)
Fourth attachment: A graphical representation of the above mentioned timeline

%%%%%%%%%%%%%%
Calibration Line List Update
%%%%%%%%%%%%%%

Freq (Hz)   Actuator                   Purpose                      Channel that defines Freq             Changes Since Last Update (LHO:72108)     
15.6        ETMX UIM (L1) SUS          \kappa_UIM excitation        H1:SUS-ETMY_L1_CAL_LINE_FREQ          No change
16.4        ETMX PUM (L2) SUS          \kappa_PUM excitation        H1:SUS-ETMY_L2_CAL_LINE_FREQ          No change
17.1        PCALY                      actuator kappa reference     H1:CAL-PCALY_PCALOSC1_OSC_FREQ        No change
17.6        ETMX TST (L3) SUS          \kappa_TST excitation        H1:SUS-ETMY_L3_CAL_LINE_FREQ          No change
33.43       PCALX                      Systematic error lines       H1:CAL-PCALX_PCALOSC4_OSC_FREQ        No change
53.67         |                            |                        H1:CAL-PCALX_PCALOSC5_OSC_FREQ        No change
77.73         |                            |                        H1:CAL-PCALX_PCALOSC6_OSC_FREQ        No change
102.13        |                            |                        H1:CAL-PCALX_PCALOSC7_OSC_FREQ        FREQUENCY CHANGE; THIS ALOG
283.91        V                            V                        H1:CAL-PCALX_PCALOSC8_OSC_FREQ        No change
284.01      PCALY                      PCALXY comparison            H1:CAL-PCALY_PCALOSC4_OSC_FREQ        No change
410.3       PCALY                      f_cc and kappa_C             H1:CAL-PCALY_PCALOSC2_OSC_FREQ        No change
1083.7      PCALY                      f_cc and kappa_C monitor     H1:CAL-PCALY_PCALOSC3_OSC_FREQ        No change
1153.2      PCALY                      Old "cancelling" line        H1:CAL-PCALY_PCALOSC9_OSC_FREQ        Added to the list, but has been on for all of O4.
n*500+1.3   PCALX                      Systematic error lines       H1:CAL-PCALX_PCALOSC1_OSC_FREQ        No change (n=[2,3,4,5,6,7,8])

Simulines start:

PDT: 2023-09-13 11:36:48.648354 PDT
UTC: 2023-09-13 18:36:48.648354 UTC
GPS: 1378665426.648354
 

Simulines end:
PDT: 2023-09-13 11:58:49.591980 PDT
UTC: 2023-09-13 18:58:49.591980 UTC
GPS: 1378666747.591980
 

Files:

2023-09-13 18:58:49,243 | INFO | File written out to: /ligo/groups/cal/H1/measurements/DARMOLG_SS/DARMOLG_SS_20230913
T183650Z.hdf5
2023-09-13 18:58:49,260 | INFO | File written out to: /ligo/groups/cal/H1/measurements/PCALY2DARM_SS/PCALY2DARM_SS_20
230913T183650Z.hdf5
2023-09-13 18:58:49,269 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L1_SS/SUSETMX_L1_SS_20
230913T183650Z.hdf5
2023-09-13 18:58:49,279 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L2_SS/SUSETMX_L2_SS_20
230913T183650Z.hdf5
2023-09-13 18:58:49,289 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L3_SS/SUSETMX_L3_SS_20
230913T183650Z.hdf5

Starting with a calibration measurement.

Last night we had some alignment issues after a lock loss (alog72854) and Jenne noticed that our jitter noise is higher (alog72853). Comparing two DARM spectra from the summary pages in the first attachment from two days ago to now, the ~120Hz noise is worse, though our higher freqency noise seems better now.

In terms of alignment, Ryan C's Misaligned GUI shows ITMY P being further off compared to near the end of the Sept 11 lock before maintenance, but I'm not seeing that in top mass OSEM values as seen in the 3rd attachment. Interestingly, L1 P seems to be slightly shifted from the reference time, but this isn't seen in M0 or the oplevs.

This GUI also points to SR2 being slightly off in P and Y, which it does seem to be compared to the reference time, but overall doesn't seem that far off from where it normally moves (6th attachment). PRM is another contender with larger alignment moves, but nothing out of the normal between locks (7th attachment).

The ETMs see similar movement as well (ETMX ETMY). Perhaps obvious, the most movement we see is during our move spots [508] and max power [520] states.

FC2 has been moving much more in the last 5 days. Not sure if this is a symtom or a cause, but perhaps this might point to some of our SQZ issues. FC1 shows a similar story, though doens't seem to be as drastic.

All of this to say that I don't notice any major alignment differences, aside from FCs, from before maintenance to now, but there are a few interesting bits to spend some more time looking at.

Wed Sep 13 10:06:26 2023 INFO: Fill completed in 6min 22secs

Jordan confirmed a good fill curbside.

TITLE: 09/13 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 151Mpc
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 2mph Gusts, 1mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.14 μm/s
QUICK SUMMARY: Locked for 7.5 hours, calm environment.

CDS Overview OK, no alarms.

There are two issues to address in this entry regarding the configuration of the corner station AHU1.

1: persistent flooding of the condensate tray's was occurring which at times caused water to reach the floor of the AHU requiring that it be vacuumed up. The cause of improper draining was suspected to be a clogged drain line. The line was flushed both with compressed air and with pressurized water. Following that, a vacuum was pulled on each of the two drain lines with a wet/dry shop-vac type vacuum. Once the trap at the drain is primed, the drain line should now function without issue.

2: the likely cause of moisture to be present in the first place is in the way that the AHU was moving air. Unlike the neighboring AHU (2), The face bypass dampers in AHU1 above the cooling coil were OPEN and the dampers directly below which allow flow across the coil were CLOSED. The attached screen grab illustrates the desired configuration. The cause for the bypass dampers to have flipped was found in the supply temperature settings which were changed from a temp control mode of "auto reset" at a 52F setpoint to a temp control mode of "manual" with a 45F setpoint which now mirrors AHU2. 

This change has been in effect since midday Monday 9/11.

B. Gateley T. Guidry 

TITLE: 09/13 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Austin
SHIFT SUMMARY:

IFO is LOCKING and is at LASER_NOISE_SUPPRESSION as of submission (07:21 UTC)

Lock Acquisition (not smooth at all)

While attempting lock acquisition, DRMI would not lock, PRMI woul not lock and none of the POP18 abd POP90 traces were anywgere near where they were supposed to be (both just showing noise about 0 counts). Guardian went to check MICH fringes about 5 times before I decided to do an initial alignment. I attemped to manually request/pause at ACQUIRE_PRMI to play the whole BS beam movement game but I couldn't see a second beam after some movement in both yaw and pitch (and guardian kept beating me to it by going into check MICH fringes). There were some strange interference patterns in the AS AIR camera that I hadn't seen before while I was attempting this.

I was checking the troubleshooting documentation throughout this, but found no relevant details to assist with this problem.

Initial Alignment (6:10 UTC)

I had issues getting guardian to perform an automatic (ish) initial alignment. It kept getting stuck trying to lock ALS when ALS had been locked steadily for a few minutes. I manually offloaded green arms and went into the next stage of initial alignment, where it started behaving and moving automatically. During initial alignment, I saw both beams overlaying in the AS AIR camera for the first time since lockloss so that is a good sign (I think). Initial alignment successfully finished so now onto locking!

Post Initial Alignment Lock (6:38 UTC):

DRMI has finally locked. This was definitely some weird alignment issue that got fixed because this time, we didn't even go through PRMI (or MICH Fringes). I expect guardian to manage locking manually.

What was the issue?

I do not know what the root of the lockloss or the lock acquisition was. As said in alog 72852 the wind had died down and there were no significant earthquakes. This leaves the BLRMs glitch lockloss coindcidence.

LOG:

Random lockloss - wind and seismic acitivity look okay at time of lockloss.

DRMI unlocked as soon as a glitch (maybe 10th one of the night) showed up on BLRMs screen. This is similar to my last few evening locklosses.

IFO is in NLN and OBSERVING as of 21:00 UTC (Sep 12) - 6hr and 35 minute lock.

Nothing else to report.

Weekly (missed week 9/4-9/8) In-Lock SUS Charge Measurements (also includes more recent 9/12 measurements).

Closes FAMIS 26056

J. Kissel, D. Barker
WP #11423

Dave has graciously compiled, installed and restarted the h1calcs model. In doing so, that brings in the bug fix from LHO:72820, which fixes the issue that the front-end, CAL-CS KAPPA_UIM library block was receiving the KAPPA_TST uncertainty identified in LHO:72819.

Thus h1calcs is now using rev 26218 of the library part /opt/rtcds/userapps/release/cal/common/models/CAL_CS_MASTER.mdl.

I'll confirm that the UIM uncertainty is the *right* uncertainty during the next nominal low noise stretch later today (2023-09-12 ~20:00 UTC).

Elenna, Gabriele, Camilla 

This afternoon we updated the MICH Feedforward, it is now back to around the level it was last Friday, comparison attached. Last done in 72430. Maybe need to be done so soon because of the 72497 alignment changes on Friday.

The code for excitations and analysis has been moved to /opt/rtcds/userapps/release/lsc/h1/scripts/feedforward/

Elenna updated in guardian to engage FM1 rather than FM9 and sdf accepted. New filter attached. I forgot the accept this in h1lsc safe.snap and will ask the operators to accept MICHFF FM1 when we loose lock or come out of observe(72431), tagging OpsInfo.

Attached is a README file with instructions.

This morning between 9:20am and 10:00am I injected some noise to retune the MICH and SRCL feedforward. New filters have been uploaded with name '8-4-23'. Unfortunately the IFO lost lock before I was ready to test them.

We shoudl wait for the IFO to thermalize after relock, and then test the two new filters