On June 28 2023, Sheila and Naoki took one hour of no-squeeze data with the hot OM2 alog 70930.
Directly afterwards, I took a DARM OLG and PCAL injection using noise_recorder, with good coherence > 0.9 everywhere between 6 and 700 Hz.

With these three inputs, we can construct the correlated DARM noisebudget directly from measurements in a model-independent way.
This eliminates most of the dependence on the calibration model when reconstructing correlated DARM.


Results
The interactive correlated DARM noisebudget .svg is hosted here:
https://lhocds.ligo-wa.caltech.edu/exports/craig.cahillane/gitcommon/correlated_noise/figures/20230628_195416_utc/correlated_darm_calibrated_zoomed_binwidth_0p1.svg
The same .pdf is posted as the first attachment.

Most of the noisebudget terms are from the latest LHO noisebudget squeezed time, taken from 
/ligo/gitcommon/NoiseBudget/aligoNB/out/H1/lho_darm_noisebudget/lho_darm_noisebudget.hdf5.

Notes
0) The noisebudget terms come from a time with squeezing injected, so for my correlated noise both shot noise and radiation pressure noise should be elevated compared to what is plotted.  I have included the Unsqueezed shot noise estimate here.

1) The correlated noise seems to be dominated by thermal noise in the bucket, around 200 Hz.  See Evan's alog 68482 with a different method for a comparison.

2) At low frequencies, the noisebudget expected is higher than what is measured.  This is true for the normal squeezed DARM measurement in the NB as well.  Not sure if this NB is out of date, or if I'm including old terms that shouldn't be included anymore (like this RadiationPressure term or PUMDAC term).

3) At high frequencies, we are hitting some unknown correlated noise floor, just as we have in the past.  I am sure we have resolved the actual correlated noise floor because I've used logbinning (which gives you many more effective averages at HF), and I ran at a few different binwidths (0.1 Hz, 0.25 Hz, 1.0 Hz) and this correlated noise floor became better resolved and did not go lower.

4) At very high frequencies (> 4 kHz), correlated noise is limited by frequency noise.  The precise mechanism of HF frequency noise coupling to DARM is not well-understood, but it can be made to go extremely low given a good mode-matching thermal state, e.g. red Freq/DARM TF here.

4) There were significant glitches during this hour of data, as there usually are.
The second pdf shows the difference between the glitchy vs "PSD-rejected" spectral densities, where I threw out abnormally high PSDs.
 
5) Sheila has made these plots already in alog 70978, but was struggling because the correlated noise and DARM noise were having some 7% error in the totally-correlated regime (~20 Hz), casting doubt on the rest of the results (only at the few-percent level).  I don't understand the source of those issues, but I don't think I am having the same issue.  I zoomed way in at 20 Hz and there was no difference (PDF 3).  

6) I did not do any DCPD balance matrix corrections because the current matrix has very small values ( SUM = 0.9996 * A + 1.0004 * B ).


Code
I've revamped the code in 
/ligo/gitcommon/correlated_noise/code/plot_cross_correlation.py
to be easily usable with labutils env.
plot_cross_correlation.py needs access to the noise_recorder DARM measurement data and the aligoNB lho_darm_noisebudget.hdf5 file, so this code will only run on CDS machines.

To run, 
1) open plot_cross_correlation.py  
2) go to the USER PARAMETERS section (~line 453 as of this writing), and make sure the parameters look correct.  You can even change the binwidth, (I found 0.1 Hz, 0.25 Hz, and 1.0 Hz all actually worked, but 0.3 Hz did not just because of annoying array length problems).
3) In terminal, run 
conda activate labutils
cd /ligo/gitcommon/correlated_noise/code/
python plot_cross_correlation.py


Details
I calibrated the PCAL using a gain of 1 and two poles at 1 Hz.  This is our ultimate meters reference.
I used the DARM OLG data directly to remove the DARM loop sensing noise correlating effect.  I extrapolated in the low and high frequency regimes using a 1/f and 1/f^2 function, respectively.
The DARM sensing function is just a simple gain and pole fit using scipy.curve_fit().  At this time, there was very little detuning and so no need for a more complex sensing model.
"PSD-rejection" is just the process of finding the median PSD value at some frequency, assume an exponential distribution, and throw out any PSD that comes in above a certain threshold.  It's sort of like median-averaging meets glitch-gating without the headaches.


alogs
Sheila's June 21th (cold OM2) and June 28th (hot OM2) correlated noise plots: alog 70891
Naoki's No-SQZ times for June 28th: alog 70930
Sheila resets the OMC DCPD matrix, and does some DCPD math: alog 70453
Evan turns down and bandstops the DARM loop at 100 Hz to avoid DARM loop correlations: alog 68482
May 2021 correlated noisebudget: alog 59073
O3 correlated noisebudget: alog 57284