Modification of FC2 M1/M3 crossover

Naoki, Vicky, Gabriele

Recently Gabriele improved the M1/M3 crossover of PRM as reported in alog68864 and alog68899. Since the FC2 and PRM are the same HSTS suspensions, we imported his offload filter from PRM to FC2. The FC green can be locked with new filter. We will check if IR can be locked and see the long term stability.

The previous FC2 offload filter was designed by Sheila as reported in alog66092. Fig 1 shows the FC2 M1 LOCK filter bank. The previous filter was using FM1,3,4,6,10. We imported the FM5,7,8,9 filters from PRM M1. The gain of PRM M1 is -0.02 and the gain of PRM M2 is 10, so we set the gain of FC2 M1 as -0.2. Here is the summary of FC2 M1 filter.

old: FM1,3,4,6,10 ON, gain 0.5
new: FM1,5,7,8,9,10 ON, gain -0.2

Fig 2 shows the FC2 M3 LOCK filter bank. We are not sure about the FM2,9 in FC2 M3, but the CLP300 in FM9 seems imported from MC2 M3. Since we want to keep the crossover of green SUS/VCO, we kept these FC2 M3 filters.

Fig 3 shows the crossover of FC2 M1/M3 (purple: old, blue: new). The M1/M3 crossover frequency is 0.8 Hz. In the previous crossover, there was a peak at 1.3 Hz which required us for the wig1.3Hz filter for stable crossover. With the new filter, the peaks at 1.3 Hz and 3 Hz are removed and the crossover looks more reasonable. 

We also measured the FC2 actuation response when FC is down (Fig 4). The template is saved in userapps/sqz/h1/Templates/dtt/FC2/M1M3total.xml. The peak at 3 Hz with old M1+M3 is removed with new M1+M3. 

OFI Transverse shaking: evidence of backscattering from the squeezing port

Eleonora, Xinghui

0. OFI suspension diagonalization attempt

The OFI suspension OSEM matrices turned out to be not diagonalized. We realized this while trying to perform noise injections in the longitudinal (L) and transverse (T) directions in order to study the effect of scattered light.

Refer to Fig. 1 for a schematic of the OFI axis and sensor definitions. The OFI is outfitted with three OSEMs: SD (side), LF (left) and RT (right). Linear combinations of these three OSEMs are used to drive and read out L,T and Y (Yaw, not used here). The transformation for driving L, T and Y is saved in the EUL2OSEM matrix, whereas the transformation to read-out in L,T and Y basis is saved in the OSEM2EUL matrix.

 

We sent noise to the OFI suspension coils at a frequency = 0.65Hz. This is true for the whole entry, unless explicitly mentioned.

By sending a current in L, we coupled in T by 20%. By sending a current in T, we excited L 5 times more than T.

We tried:

• to change the gains in the driving filters in L, T, Y to reduce the coupling by checking at the ndscope signals (H1:SUS-OFI_M1_DAMP_{L,T,Y}_INMON)
• to update the sensing matrix by computing a new matrix based on geometrical considerations

however, finding a configuration that acted on L only, didn't yield a decoupled T. Instead, driving T in this configuration resulted in read-out signals with equal amplitudes in L and T. We concluded that the computed L and T directions did not correspond to the true L and T directions. 

Therefore, we decided to check more in detail the input/output. We set the sensing matrix (OSEM2EULER) in a way that we were directly reading the 3 coils LF (left), RT (right), SD (side) instead of L, T, Y: [ 1 0 0;

                                                                                                                                                                  0 1 0;

                                                                                                                                                                  0 0 1 ].

Then, we tried to drive only in the tranverse direction by sending an excitation to H1:SUS-OFI_M1_TEST_T_EXC in order to check the balancing between LF and RT wrt the COM and decouple them from L.

We set the driving matrix (EULER2OSEM) to be [ 0 -1 0;

                                        0 1 0;

                                       0 0 1 ] .

The minus sign is due to the fact that the magnets of the OSEMs have inverted poles (see Fig. 1). By changing the gain on the driver filters (H1: OFI M1 COIL OUTPUT FILTERS) we didn't improve the decoupling, so we left all the gains to 1.

The LF, RT, SD signals should be already calibrated in um.

We measured first the residual motion sensed by the OSEMs without injecting noise: LF = 0.06 um, RT = 0.06 um, SD = 0.03 um).

Then we measured the residual coupling by sending a transverse signal with a gain of 5000: LF = 10.6 um, RT = 9.6 um, SD = 0.2 um. LF and RT are 10% coupled, while SD is couple 2% with LF and RT, which is good enough for our measurements.

In the following, we report 3 different OFI transverse shaking measurements.

1. OFI T shaking with SQZ (injection gain 5000)

During our diagonalization attempts, we noticed (thanks also to the commissioning team) an excess of noise between 6 and 15 Hz that we believe to be scattered light coming from the squeezing port.

• The first evidence was between 17h55 UTC and 17h58 UTC, injecting noise on T with a gain of 5000. In Fig. 2, the signal read by ndscope (NB the plot channel names are INCORRECT due to the change of OSEM2EULER matrix mentioned in (0)) are shown. The effect of the scattering noise on DARM is shown in Fig. 3.

• The second evidence was between 18h20 UTC and 18h30 UTC, injecting noise on T with a gain of 5000 and unbalancing the driver gains (gain_LF = -0.9, gain_RT = 1.5) to have a bigger amplitude: LF = 13.7 um, RT = 12.7 um, SD = 0.2 um. In Fig. 4 the signal read by ndscope (NB the plot channel names are INCORRECT due to the change of OSEM2EULER matrix mentioned in (0)) are shown. The effect of the scattering noise on DARM is shown in Fig. 5.

2. OFI shaking without SQZ (injection gain 15000)

We did the same injection with a higher gain and saw a different response on DARM, which came out from the fact that SQZ was NOT injected in the interferometer.

We injected T noise with a gain of 15000, with balanced driving gains, corresponding to a motion of LF = 31 um, RT = 29 um, SD = 0.55 um.

The noise injection lasted from 23h37 UTC to 23h48 UTC.

Without squeezing injection, we don't see the same scattered light effect as with SQZ, but we introduce some noise between 20 and 30 Hz as shown in Fig. 6.

3. OFI shaking with SQZ with different injection gains

We performed this measurement for G = 5000, 15000, 17000 and 19000. The suspension was saturating for G = 20000. The results are shown in Fig. 7.

• G = 5000, corresponding to a motion in LF = 10.6 um, RT = 9.6 um and SD = 0.2 um ( yellow curve ), between 0h47m40 UTC and 0h57m40 UTC (22nd April 2023)
• G = 15000, corresponding to a motion in LF = 31 um, RT = 29 um and SD = 0.5 um ( mint curve ), between 1h01m20 UTC and 1h13m10 UTC (22nd April 2023)
• G = 17000, corresponding to a motion in LF = 35.5 um, RT = 32.5 um and SD = 0.6 um ( green curve ), between 1h13m15 UTC and 1h24m00 UTC (22nd April 2023)
• G = 19000, corresponding to a motion in LF = 39.7 um, RT = 36.6 um and SD = 0.7 um ( blue curve ), between 1h25m40 UTC and 1h35m40 UTC (22nd April 2023)

The scattering shelves are very evident and the cutoff frequencies scale with the scatterer velocity as expected. Moreover, we notice some structures with 3.2 Hz spacing in the plateau region of the shelves, as shown in Fig. 8.

We will analyze the data to extract the amount of recoupled scattered light coming from the squeezing port into DARM.

 

Very noisy slow control power supply fan

The sound of the fan chugging was audible from downstairs (this power supply lives on the mezzanine). I didn't hear this yeaterday. Just in case DARM gets noisy over the weekend this guy might be a suspect.

DARM Spring Measured

I went to NLN_CAL_MEAS and used the code in  gitcommon/noise_recorder to measure the DARM spring (~1hour after powerup). Plot attahced, the SRCL1 offset was set at -165.

I edited them functions to record a directory name including the time so that on another lock we can schedule the 'python pcal_noise_injection_caller.py; python darm_noise_injection_caller.py' command multiple times in 1 day. This did accidently put the data in two directories but that's an easy fix later.  Hope we could script this to run mutilple times while thermalizing to give more evidence for an evolving DARM spring as Vicky theorized (alog68853) or while doing CO2 steps. Alterative would be lines as Jeff suggests we add next week. 

Ops Day Shift Summary

TITLE: 04/21 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
SHIFT SUMMARY: Jim has a new ISI filter for ETMY to mitigate the motion at EY from the wind fence work; we were able to leave it on all day successfully.

• Lockloss @ 16:06 from accidentally turning on PRCL UGF servo incorrectly

Lock Acq #1:

• Relocking via PRMI
• Paused during OMC_WHITENING to damp violin modes
• NLN @ 18:26
• Lockloss @ 20:58 from attempting to switch to new PRM offloading in-lock (it did not work)

Lock Acq #2:

• Turned nominal sensor correction back on at EY as wind fence work had finished
• Adjusted both TMSX and ETMY alignment to lock ALS
• IR DIFF found by hand
• Again paused during OMC_WHITENING to damp violin modes
• NLN @ 23:01

LOG:

Start Time	System	Name	Location	Lazer_Haz	Task	Time End
15:15	FAC	Karen	Opt/Vac Labs	-	Technical cleaning	15:28
15:15	FAC	Randy, Mitchell	EY	-	Wind fence work (Tyler joining @15:51)	18:54
15:30	ISC	Elenna	CR	-	Freq noise inj, CARM meas, DARM spring, CO2 step	16:13
15:47	EE	Fil	MX	-	Vacuum pump wiring	17:11
16:18	SUS	Rahull	Remote	-	SRM	17:04
16:39	VAC	Travis	EY	-	Install wheels on dolly	17:39
17:42	SQZ	Nutsinee	LVEA - SQZ	-	Picking up equipment	17:46
17:48	FAC	Karen, Cindi	FCES	-	Technical cleaning	18:31
19:58	VAC	Travis	MX	-	Hepta pump work	20:26
20:01	FAC	Randy, Mitchell, Tyler	EY	-	Wind fence work	21:07

 

 

Strange DARM Behavior at ADS_TO_CAMERAS

Both times locking today, the DARM diaggui has done some strange things starting somehwere between LASER_NOISE_SUPPRESSION and ADS_TO_CAMERAS (see attached). This "glitch" bumps everything up, then a few seconds later it will bump up again. This appears to be an issue with GDS-CALIB_STRAIN, as restarting the template without that channel included returns the picture to normal.

HSTS mode cleaner and recycling cavity mirror damping loops updated

I have updated all mode cleaner mirrors (MC1, MC2, MC3) and recycling cavity (SR2, SRM, PR2, PRM) HSTS damping loops to the Level 2 design that Jeff Kissel developed, and we worked on testing and implementing. You can read alog 65310 for a reminder of the implementation of these.

We have done this once before, but faced many stability issues related to the change. However, Jeff and I do not think the problems we faced were actually related to the damping loops. We think the various issues might have been related to the incorrect PRM M1/M3 crossover that Gabriele has noticed (and now fixed), as well as the ITM reaction chain motion. There is a whole saga related to this, but two things remained evident:

• When we reverted the damping loop changes, instability problems persisted
• Measurements of the M1 to M1 and M1 to M3 transfer functions were nominal, and did not indicate any issues with the damping loops themselves (see alog 67266 for more info)

Therefore, Jeff and I think these are fine to implement again, especially given that Gabriele has solved the crossover issue on PRM (and is currently working on updating SRM).

All HSTS damping loops should be using FM1, 2, 4, 5, 9 and 10. All gains should be -1. I have SDFed these filters and gains accordingly. The "old" filters are located in FM6, 7, 8, and 9. (Note: FM9, the BR notches, is common to both configurations.)

We are relocking and there is no sign of the issues we experienced during the past saga of implementation.

Fri CP1 Fill

Fri Apr 21 13:24:44 2023 INFO: Fill completed in 9min 44secs

Gerardo confirmed a good fill curbside.

POP beam diverter close back in guardian

We commented out the POP beam diverter close in CLOSE_BEAM_DIVERTERS during TCS changes. I have put that back in the guardian state.

End Y Hepta Installation Complete

The mechanical and electrical assembly of the Hepta roughing pump at End Y is complete.  We still need to do the full suite of controls tests, but we have successfully powered up the Hepta remotely via the turbo pump station controls.  This is a lengthy process, so will likely take a couple of maintenance days to complete.  

Comparison between Coil Driver Noise Monitor Designs Using SRM M2/ M3

J. Kissel
ECR E2300035
WP 11130

Executive Summary: we may not want/need the D1900052-style upgrade for the M2 and M3 stage of SRM. Looking for opinions. I'll continue working on this in the background; next step is to calibrate these monitor signals in DAC output voltage in the front-end -- and then move on to looking at crafting the story for the Beam Splitter to see if there's the same conclusion there.

This past Tuesday, we changed SRM's M2 and M3 triple acquisition coil driver noise monitor circuits from D070480-style "broad band" boards to D1900052-style "narrow-band" boards. In this aLOG, I cover the "before" vs. "after," uncalibrated empirical results from the boards. For both "before" and "after" data sets, I'm comparing the answers when the detector is in Nominal Low Noise, where
    :: "before" is 2023-04-16 18:00 UTC -- i.e. the data from LHO:68735, and
    :: "after" is last night, 2023-04-21 10:00 UTC.
At both times, there is no additional characterizing drive, just the normal requested drive on the M3 stage during nominal low noise. Also as before, there is *no* requested drive from the M2 stage so the M2 data is representative of DAC noise without drive.

2023-04-21_H1SUSSRM_M3_CoilDriverMonitorUpgrade_TFs.png shows the change in transfer function between digital DAC request "excitation" (from the "MASTER_OUT_DQ" channels) and the noise  (from the "NOISEMON" channels). While I don't have a model the frequency response of the D070480-style "broad band" boards, one can see the expected shape from the newer D1900052-style "narrow-band" boards based on the modeling shown in pages 10 and 11 of G2300866. There's MUCH less gain between 0.1 and 1 Hz in the narrow-band filter, and a much faster roll-off above 100 Hz. but only a little more gain between 20 and 100 Hz.

2023-04-21_H1SUSSRM_M2M3_CoilDriverMonitorUpgrade_ASDs.png shows a comparison between the amplitude spectral density of both the MASTER_OUT drive request (top panels) and the NOISEMON monitor signal (bottom panels). 
    :: The top panels show that the normal, Nominal Low Noise drive request is much the same between the two times (I don't know why the "before" time has more 10-200 Hz request, my guess is the retuning of the SRCL feed forward, on Apr 15 2023, LHO:68709).
    :: The bottom two panels show the output of the noise monitor, which had changes that are reflective of the change in frequency response of the two boards. In those bottom panels, I also compare the two times' data against our standard model of the 16-bit ADC noise.

Perhaps interestingly, a lot of the motivation for switching to these narrow-band noise monitor circuits was because to "avoid saturation under normal under realistic actuation conditions" (see original ECR that created these boards, E1900049). 
    - Looking at the M3 stage (lower right), while there's clearly tons more gain on, the "realistic actuation conditions" -- i.e. most of the DAC request is still well below saturation. In fact, the RMS is completely dominated by the 253.1 Hz LSC monitor line at 253.10 Hz (+/- 0.017 Hz) -- which is a temporary feature of commissioning the PRCL and SRCL loops -- see the conclusion to turn these OFF this morning *after* I took the "after" data, Apr 21 2023 LHO:68886).

2023-04-21_H1SUSSRM_M2M3_CoilDriverMonitorUpgrade_ASDs_250HzZoom.png shows a zoom around this feature. (I thought this was interesting until I found out that the line is well known and now off.)

It may be note worthy that H1 still has a factor of 10 more analog gain in its triple acquisition drivers (see latest discussion in LHO:64114), so the *digital* DAC request is a factor of 10 less than if we reverted to the standard TACQ drive, and the noise monitor pick-off is up-stream of the gain increase, so we see a factor of 10 less signal here that we might have if we reverted the driver, but we'd still have plently of ADC head room. 

    - Looking at the the M2 stage (lower left), I think the story is the same here. The change we see in the signal is entirely expected given the change in frequency response, and indeed -- with the narrow-band noisemon, we're resolving the un-driven DAC noise about the ADC noise floor *less* than with the broad-band board. Where the narrow-band board gains, it's only a factor of 1.5 to 2 at 20-25 Hz. Not that exciting.

So, in conclusion -- see above executive summary.

Redesigned PRM offload

Following up on the discovery that the 3.4 Hz instability is due to the PRM offload, I redesigned the M1/M3 offload for PRM. The old offload has a crossover at 0.5 Hz, which I believe was too large. I targeted a cross-over at 0.1 Hz. This mean that we the new crossover, the PRM equivalent actuator will have less gain at low frequency, but there's plenty of room there.

The new design is shown in the first attached figure, showing the target crossover of 100 mHz. The second plot compares a model of the new crossover design with the old crossover. The model of the old situation reproduces very well the measured PRM actuation. The thrid plot compare sthe old filter with the new filter (without including the band stops).

I implemented the new crossover in the H1:SUS-PRM_M1_LOCK_L filter bank FM7. This is to replace FM4+FM9 (I don't think we need the 10 Hz elliptic low pass).

We tested the new crossover during lock. It works fine and as expected:

• the PRCL 3.4 Hz peak is disappeared
• the PRCL error signal has less suppression below 0.5 Hz (we can fix it with a boost in the PRCL loop if desired)
• the PRCL control signal is larger below 0.5 Hz to compensate for the reduced actuation gain
• with the IFO down, the measured PRM actuation shows the effect of the new offloading.

We'll automate this new configuration.

I suspect that SRM needs a similar treatment, to reduce the 3.5 Hz peak, and maybe also to tackle that huge 1 Hz peak.

 

PRCL UGF servo trends

We now had several locks with the PRCL UGF on. In each lock there were different TCS settings and different thermalization going on. Nevertheless the behavior is quite consistent.

The PRCL gain needs to be increased by a factor 3.8 in about 400 seconds.

A fit of one of the trends shows a good match with an exponential thermalization with a time constant of 100 minutes.

We could switch off the PRCL UGF servo (and the 52.1 Hz line) and hard code a gain change as above.

Loud Glitch Rate in O2 and O3

Gaby, Shania 

We have been investigating the potential sources of loud glitches. At the request of Robert Schofield we calculated the daily rate per hour of loud glitches (SNR >1000) for O2 and O3 at both sites (see attached pdf). The rate of loud glitches increased in both detectors from O2 to O3. This may potentially rule out dust as a soure as we expected the glitch rate to decrease over time as the dust settled. 

	O2 Rate/hr	O3 Rate/hr
LHO	0.14	1.20
LLO	0.09	0.69

 

Ops Day Shift Start

TITLE: 04/21 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 7mph Gusts, 4mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.22 μm/s
QUICK SUMMARY: H1 has been locked for 2.5 hours at NLN. We are running with BRSY on this morning during wind fence work to test Jim's new ISI filter. At the next lockloss, there are plans for a slow controls power supply swap and SRM drive measurement.

• CDS, SEI, SUS all okay
• No alarms overnight
• All dust monitors okay

Debugging TDCFs: Mysteries with the PCAL Line DEMOD Phase vs. 16 kHz Clock-cycles from IPC Delays SOLVED

J. Betzwieser, L. Dartez, E. Goetz, J. Kissel

We've circled the wagons to try to understand why we're consistently getting things wrong when we try to update the "reference model DARM Loop transfer function values at calibration frequencies" (aka what the calibration group typically refers to as simply "EPICs Records") and get time-dependent correction factors (TDCFs) to the IFO's sensing and actuation functions that are *very* inconsistent with MCMC fit values for corresponding parameters (overall gain of the sensing function [often loosely referred to as the "optical gain"], coupled cavity pole frequency, and the actuation strengths for each of the three ETMX actuator stages). Namely, yesterday's attempt by Louis, LHO:68856 convinced us that we've fundamentally forgotten about *something* between O3 and O4, and it's manifested in how the latest version of pyDARM is written.

While we can't say that we've solved *all* of the issues, we have found and remembered the *a* major thing, and it has to do with the differences between how GDS calculates TDCFs and CAL-CS calculates TDCFs, and our valiant efforts to change the infrastructure for for O4, such that both are doing the same thing. 

Here's the fundamentals of the TDCFs:
    (A) In order to determine things like the (relative) optical gain, cavity pole, we always need to find the transfer function between of DARM_ERR and PCAL, i.e. DARM_ERR / PCAL.

    (B) Both GDS and CAL-CS use demodulation techniques to find this transfer function, taking some version of the in-loop signals, DARM_ERR|_OMC and PCAL|_END, band-passing them, and beating them against a "local" oscillator to find I and Q, or real, and imaginary components of those demodulated signals, and then taking the coherent, complex ratio of the two sets to form the transfer function.

    (C) Both pipeline use the same reference model transfer function values (EPICs records) to divide out known stuff about the electronics, suspension dynamics, etc.

    (D) Both pipelins use the CAL-CS copy of the in-loop DARM_ERR signal, DARM_ERR|_CS. This signal is delayed by one 16 kHz clock cycle, exp(-1i*phi) as its shipped over inter-process communication from the OMC model to the CAL-CS model,
        DARM_ERR|_CS  = DARM_ERR|_OMC * exp(-1i*phi)        (1)
           or
        DARM_ERR|_OMC = DARM_ERR|_CS  * exp(+1i*phi)        (2)

    (E) There's been a constant point of confusion (discrepancy, or loss of institutional memory) that CAL-CS reads PCAL from corner station, PCAL|_CS, where GDS reads PCAL from the end-station, PCAL|_END,
        PCAL|_CS  = PCAL|_END * exp(-1i*phi)                (3)
           or 
        PCAL|_END = PCAL|_CS * exp(+1i*phi)                 (4)

    and reconciling that between the two pipelines. In the O2-O3 era, we *thought* we reconciled the discrepancy by modifying the CAL-CS pipeline's PCAL demod phase, adding a negative phase that corresponded to the value (phase) of a one 16 kHz clock cycle delay at each calibration line frequency,
        phase = 180/pi*angle(exp(-1i*2*pi*linefreq*(1/16384 Hz)))
      You can see this adjustment every time we've had to mode a PCAL calibration line frequency, and enter in a new phase value LHO:48534, LHO:44005, LHO:31748, etc. 

    (F) In April 19 2019, when we got bit one last time by this loss of institutional memory, and we vowed to "fix all this" and demand that "every part of both the GDS and CAL-CS pipeline should ingest the same thing, and compute the same thing, and use the same EPICs records" -- see IIET Ticket 12638, pydarm issue #17.

    (G) Between the end of O3 and now (yes, for 3 years), Evan's been championing a new version of pyDARM, and among many other improvements, we'd that hoped to resolve the issues with the computation of the EPICs records. In the last couple of attempts to "push" those EPICs records (LHO:68856), as well as attempts to move TDCF calibration line frequencies LHO:66268, the resulting TDCFs were wildly different from the MCMC fit values that we expect -- most typically in the optical gain, kappa_C, and the coupled cavity pole frequency, f_cc.
    
    (H) In July 2022 (LHO:64005), I added a channel to the frames, H1:CAL-DELTAL_REF_PCAL_DQ, which is a corner station copy of which ever PCAL we're using as our reference, i.e. PCAL|_CS -- such that the GDS pipeline *could* switch over to using the this version of the PCAL.

Now you're caught up with yesterday, when Louis and I sent out the bat signal to circle the wagons.

After comparing old EPICs records (not updated since O3) against new EPICs records (recently computed by modern pyDARM and a modern pyDARM model parameter set), 
CLUE 1: Joe found that the *newly* computed reference model transfer function value for the so-called "PCAL Corrections" for the 410.3 showed a difference, only in phase, of 18.03 deg against the old values. It smelled suspiciously like double the phase delay expected to be put in the DEMOD PHASE, 9.02 deg,
    180/pi*angle(exp(-1i*2*pi*410.3*(1/16384))) = -9.0154,
    180/pi*angle(exp(-1i*2*pi*410.3*(2/16384))) = -18.031
Suspicious that we were double counting this delay, we went back to the O3 code that computed the PCAL corrections.

Remember, PCAL corrections are the offline corrections one needs to perform on any copy of any PCAL RX or TX PD channel in true DARM displacement caused by PCAL, DELTAL_PCAL, which includes the stuff that the front-end doesn't do to the channel before storage,
    * arm sign                  # a conversion of pcal force (which is in the minus 
                                # longitudinal direction, in test mass suspension 
                                # euler basis) to DARM force (PCALY makes negative 
                                # DARM, PCALX makes positive DARM)              

    * 1/f^2                            # the "rest" of the TST-stage force to TST-stage 
                                       # displacement transfer function, which *acts* 
                                       # like a "dewhitening" filter

    * 1 / (analog AA)                         # high frequency inversion of the response 
                                              # of the 65 kHz analog AA filter for any 
                                              # readback signal

    * 1 / (digital 65 kHz to 16 kHz delay)    # high frequency inversion of the response 
                                              # of the 65 kHz to 16 kHz downsampling filter
But also, critically, where the copy of the pcal signal is used -- the corner station or end-station -- the pcal correction must also either include or exclude the exp(-1i*phi) clock cycle delay, as defined above in Eqs. (3) and (4).

The new pydarm code (as of git hash 20668eb4), has a method that produces this correction, compute_pcal_correction. Importantly, in that version, it has a boolean flag, where the user of the method can determine whether the user is correcting a PCAL|_CS or a PCAL|_END channel. When "endstation=False," then it's to be used for a cornerstation copy of the PCAL, PCAL|_CS, and it divides by one 16 kHz clock-cycle delay, which would create the equivalent of equation (4) from above, 

    (pcal_sign *
     pcal_dewhitening /                 
    (pcal_analog_aa_hi_f * pcal_digital_aa) /
     end_to_calcs_response)

where end_to_calcs_response is either a transfer function of 1 (when endstation=True), or a one 16 kHz clock-cycle delay (when endstation=False).

Stumped, because this makes sense to us, we went to old O3 codesensing.py, line 215,

    pcalCorrection = signal.freqresp(pcaldewhitening, 2.0*np.pi*freq)[1] /              # 1/f^2
                     ( signal.freqresp(AAanalog, 2.0*np.pi*freq)[1] *                   # 1/(analog AA)
                     AAanalog_delay/abs(signal.freqresp(AAanalog, 2.0*np.pi*0.01)[1])   # (Some crap to fix a bad model of the analog AA)
                     * totalSensingDigital_freqresp                                     # 1/(digital AA)  
                     * omcDelay_freqresp )                                              # A poorly named variable that resolves to a TF of 1 

CLUE #2: we find that O3's pcal correction did not contain any advance or delay. We collectively recall: this variable was designed to be the correction to PCAL|_END channels, and did not consider the possibility of a corner station PCAL|_CS channels. 

So "how did it all ever even work??" that drove us to the O3 code that made the EPICs record itself, which *used* pcalCorrection, in , lines 431 - 434 

    EP9 = pars.Cpars.pcal.Pcal2DARMactSign *                   # arm sign
          sensProd.pcalCorrection[4] *                         # (1/f^2)*(1/analog AA)*(1/digital AA)
          np.exp(-2.0 * np.pi * 1j * freq[4] * 1/16384.0)      # exp(-1i*phi)

which reveals ... there's a delay in here! But ... GDS computes its TDCFs from PCAL|_END channels, so it doesn't need Eqs. (3) and (4) and shouldn't need any delay. And yet, both CAL-CS and GDS use this same EPICs record to compute TDCFs... CLUE #3: OH YEAH Eqs. (1) and (2) -- this so-called "PCAL_CORRECTION" EP9 (and their equivalent EPs at other frequencies) isn't actually *just* correcting PCAL, it's a correction to the DARM_ERR|_CS / PCAL|_END transfer function, because GDS reads in DARM|_CS, not DARM|_OMC, and thus needs Eq. (2).

This drove us to remind ourselves how these EPICs records are actually applied in the front-end code. the first attachment shows that it gets even nastier because CLUE #4: the "PCAL corrections" EP9 (and the like), are *divided* into the DARM|_CS / PCAL|_CS transfer function.

If you're not pulling our hair out already by this point, don't worry, Joe, Evan, Louis, and I have *definitely* pulled out enough hair today already for you. 

This means that this is the math that's happening in the GDS algorithm,

    DARM_ERR|_CS      1        DARM_ERR|_CS                                     1
    ------------  *  ---   =   ------------  * --------------------------------------------------------------------
     PCAL|_END       EP9         PCAL|_END     (arm sign) * (1/f^2) * (1/analog AA) * (1/digital AA) * exp(-1i*phi)
                               
                               DARM_ERR|_CS                         exp(+1i*phi)
                           =   ------------  * -----------------------------------------------------
                                 PCAL|_END     (arm sign) * (1/f^2) * (1/analog AA) * (1/digital AA)
                               
                                                DARM_ERR|_CS * exp(+1i*phi)
                           =   -----------------------------------------------------------------
                                 PCAL|_END * (arm sign) * (1/f^2) * (1/analog AA) * (1/digital AA)

                               DARM_ERR|_OMC
                           =  ---------------
                               DELTAL_PCAL

And in order for the CAL-CS pipeline to use the same EPICs record and produce the same answer -- but with the PCAL|_CS -- CAL CS has to install a "fudge" by altering the phase of the PCAL DEMOD, i.e. in the denominator channel before the transfer function is taken, CLUE #5: but that DEMOD phase has the opposite sign than you expect:

    DARM_ERR|_CS             1              1        DARM_ERR|_CS                                             1
    ------------ * -------------------- *  ---   =   ------------  * ------------------------------------------------------------------------------------------
     PCAL|_CS      (DEMOD exp(-1i*phi))    EP9         PCAL|_CS     (arm sign) * (1/f^2) * (1/analog AA) * (1/digital AA) * exp(-1i*phi) * (DEMOD exp(-1i*phi))
                               
                                                     DARM_ERR|_CS * exp(+1i*phi)                              1                    
                                                 =   -----------------------------   * -----------------------------------------------------
                                                     PCAL|_CS  (DEMOD exp(-1i*phi))   (arm sign) * (1/f^2) * (1/analog AA) * (1/digital AA)

                                     THE DENOMINATOR ONLY BECOMES PCAL|_END IF (DEMOD exp(-1i*phi)) == exp(+1i*phi)
                               
                                                                            DARM_ERR|_OMC 
                                                 =   -----------------------------------------------------------------
                                                     PCAL|_END * (arm sign) * (1/f^2) * (1/analog AA) * (1/digital AA)

                                                    DARM_ERR|_OMC
                                                 =  -------------
                                                     DELTAL_PCAL

So in in the most convoluted way, with delays turning into advances by a series of confusing divisions across disparate code, front-end models, and hacks on hacks, and a confusing convention with the front end's DEMOD phase system, somehow it all ended up working in O3.
Also notice, that if one computes the transfer function with both channels in the CAL-CS model, then both experience a one 16 kHz clock cycle delay, and the delays, or advances cancel, and it's like you're reading in the channels from their original location,

    DARM_ERR|_CS      DARM_ERR|_CS    exp(+1i*phi)     DARM_ERR|_OMC
    ------------   =  ------------ *  -----------   =  -------------
      PCAL|_CS          PCAL|_CS      exp(+1i*phi)        PCAL|_END

This fact was exploited a few times early on when folks first were debating who should have what delay when and how, and we ended up confusing ourselves by not including explicitly including advances or delays at all.

So, let's now zoom forward back again to O4, where we've got new code from Evan, where he's pre-included the option to envoke computer_pcal_correction with either the endstation set to True (no delay/advance) or False (dividing by a delay exp(-1i*phi), or equivalently multiplying by an advance exp(+1i*phi)).

How is the EPICs record computed? Now we head to the compute_epics_records in the calcs.py module (last changed at git hash version 60d261c4), to find CLUE #6: there's an error in this code, in that it does not apply the extra exp(-1i*phi) delay here:
        PCAL_LINE2_CORRECTION = pcal_correction[4]          # used to be EP9 from above

and for brevity we'll call this version "CORR".

On top of *this* CLUE #7: Louis and I removed the CAL-CS fudge delay yesterday, thinking "Oh, well, every pipeline is computing the DARM / PCAL transfer function ratio with the 'fixed' EPICs records with endstation set to False, so we shouldn't need the DEMOD phase fudge.

That means the CAL-CS pipeline as of yesterday instead has been computing

    DARM_ERR|_CS          1           1        DARM_ERR|_CS                            (DEMOD exp(0))
    ------------ * ------------- *  ----   =   ------------  * --------------------------------------------------------------------
      PCAL|_CS     (DEMOD exp(0))   CORR         PCAL|_CS      (arm sign) * (1/f^2) * (1/analog AA) * (1/digital AA) * exp(+1i*phi)
                               
                                               DARM_ERR|_CS * (DEMOD exp(0))                               1
                                           =   -----------------------------    * -----------------------------------------------------
                                                  PCAL|_CS * exp(+1i*phi)         (arm sign) * (1/f^2) * (1/analog AA) * (1/digital AA)
                               
                                             DARM_ERR|_CS                              1
                                         =   ------------ * ---------------------------------------------------
                                               PCAL|_END    (arm sign) * (1/f^2) * (1/analog AA) * (1/digital AA)

                                             DARM_ERR|_CS
                                         =   ------------ 
                                             DELTAL_PCAL

                                              DARM_ERR|_OMC
                           (DOES NOT EQUAL)  ---------------
                                               DELTAL_PCAL

Where one can see that, when one sets endstation to False, and you don't include any fudge phase in the PCAL DEMOD, there's a now missing advance that propagates the CAL-CS copy of DARM_ERR|_CS back to DARM_ERR|_OMC, and this messes up all computation of TDCFs that involve PCAL, which y'know is ALL of them. We see it manifest the most on the 410.3 Hz line, and thus in the calculation of the (relative) optical gain kappa_C and coupled cavity pole frequency, 180/pi*angle(exp(-1i*2*pi*(410.3 Hz/16384 Hz)) = -9.015 is MUCH larger than the 17.1 Hz line's phase modifier of 180/pi*angle(exp(-1i*2*pi*(17.1 Hz/16384 Hz)) = -0.3757.

This also explains why, when we compare EP9 with one delay, against CORR with one advance, we see a 2x abs(9.015) = 18.031 deg change.

By this point, Evan, Joe, Louis, and I all agree -- because the transfer function (DARM_ERR|_CS / PCAL|_CS) * (1/CORR) is missing an advance to propagate DARM_ERR from CAL-CS back to the OMC, or DARM_ERR|_CS * exp(+1i*phi) = DARM_ERR|_OMC, Eq. (2) from above, we can either
    (i) Fudge the phase of DARM_ERR DEMOD with +9.015, to create the missing advance, or
    (ii) Continue with the hot mess of convolution and fudge the phase of the PCAL DEMOD with -9.015

and in the mean time, Evan will edit O4-era pyDARM to include a darm_advance in the computation of CORR, and rename the EPICS record variable to be something like DARM_OVER_PCAL_CORR or something.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

OK, all of the above was all the whiteboard math and history, but as we all know -- you can math 'til your blue in the face, but when it comes to minus signs, multipications vs. divisions, or delays vs. advances -- you've got a 50/50 shot, and the interferometer will tell you very quickly if you've got it right.

So, as a quick test, Louis and I went with option (ii), ignored CLUE #5 'cause "who would believe that??," and installed the "appropriate" DEMOD phase fudges, exp(-1i*phi), in all the PCAL line DEMOD phases.

    PCAL line
    Frequency (Hz)    Demod Phase Channel                           Phase (deg)
    7.93              H1:CAL-CS_TDEP_PCAL_LINE4_PCAL_DEMOD_PHASE    -0.174
    17.1              H1:CAL-CS_TDEP_PCAL_LINE1_PCAL_DEMOD_PHASE    -0.3757
    401.3             H1:CAL-CS_TDEP_PCAL_LINE2_PCAL_DEMOD_PHASE    -9.01538
    1083.7            H1:CAL-CS_TDEP_PCAL_LINE3_PCAL_DEMOD_PHASE    -23.812

THE IFO LAUGHED AT US, SAYING "HA! YOU FORGOT ABOUT CLUE #5!"

See Second attachment.
With a brief moment in time that we installed these negative phase rotations, the already nonsensical ~310 Hz coupled cavity pole frequency and 1.12 relative optical gain shot to ...let's just say "off the map bad.
So, we installed positive phase fudges...

    Frequency (Hz)    Demod Phase Channel                           Phase (deg)
    7.93              H1:CAL-CS_TDEP_PCAL_LINE4_PCAL_DEMOD_PHASE    +0.174
    17.1              H1:CAL-CS_TDEP_PCAL_LINE1_PCAL_DEMOD_PHASE    +0.3757
    401.3             H1:CAL-CS_TDEP_PCAL_LINE2_PCAL_DEMOD_PHASE    +9.01538
    1083.7            H1:CAL-CS_TDEP_PCAL_LINE3_PCAL_DEMOD_PHASE    +23.812

and the cavity pole frequency and optical gain settled on a MUCH more reasonable 430 Hz, and (relative) optical gain of 0.95!

The cavity pole is consistent with the MCMC fits to the swept sine measurements and the change in optical gain is totally reasonable because we're *still* changing alignment and TCS settings.


WE FIGURED IT OUT, TEAM. WE DID IT.
We can trust the answers from front-end, CAL-CS computed TDCFs again...
and
There's something that doesn't make sense about the phase convention for the standard CDS DEMOD part, but we can figure that out later.

So looking at the cdsPhase part, it uses a rotation matrix of:
[I']      [ cos(phi)  sin(phi)]  [I]
      =                        *
[Q']      [-sin(phi)  cos(phi)]  [Q] 
        

This corresponds to a rotation clockwise in phi, with phi > 0.  Normally

You can imagine this matrix as multiplying the input I + 1i*Q vector by exp(-1i*phi) = cos(phi) - 1i*sin(phi).
Or explicitly, the I' + 1i*Q', the I and Q after rotation or after multiplication by this delay, are:

I' =  I*cos(phi) + Q*sin(phi)
Q' = Q*cos(phi) - I*sin(phi)

So to apply the delay Jeff needed, we need to use a positive phi into the cdsPhase rotation part.

Just as further documentation, I note GDS and the front end are multiplying by exp(-1i*wt), where w is the LO frequency, which means in all cases, for increasing time, phase rotates clockwise.  So we all have this same choice of handed-ness, which also matches what pydarm expects.