J. Kissel
I've been working for about a month now on trying to assemble the uncertainty budget for what I hope will be the only O3B darm loop "calibration" model parameter set,
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/params/
modelparams_H1_20200103.py rev 9128
installed in to the online CAL-CS pipeline on 2019-01-13 (see LHO aLOG 54473, 54476).
I've reached a point where I'm stumped, because all of the hard work organizing and culminating the huge collection of measurements out there, and understanding/accounting for each measurements' idiosyncrasies, the "final answer" uncertainty budget does not agree with the "final answer" transfer function of PCAL/GDS. The level of disagreement is shown in the final attachment, 2020-02-04_H1_PCAL2GDS_BB_meas1262990871_on_2020-01-13_vs_model_uncertainty_for_2020-01-03.pdf.
I have a couple of options in order explore what's wrong from here on, but I fear that those explorations will take me in to deep rabbit holes of even more of a mess, and I'd rather show my progress thus far, and work with others to understand where I should best proceed from here.
Please note that:
- this budget disagrees statistically signficantly with measurement, so is likely wrong in some subtle way, it will change, and should not yet be used for astrophysical analysis,
- this is the uncertainty budget for only the reference time (at 2019-12-26) it does not cover all of O3B
- this budget is comprised of data taken only during times when the IFO was well thermalized, and it does not yet cover how "bad" the systematic error is a low frequency while the sensing function is evolving as it thermalizes.
- the collection data used to create the estimate of unknown systematic error does not include data beyond 2020-01-20 that is now available, though I don't expect this to significantly impact the results
- for a look at how much this estimate has changed, and/or what we're currently using for astrophysics, please see G1901479.
For a very long, detailed record of my process -- currently in text file, core dump, format -- see T2000006.
I attach a huge collection of plots in order to support from where this uncertainty budget comes. Eventually, when I figure out what's going on, these plots and the detailed notes will turn in to a review presentation in T2000006. But, for now, bear with me, because I think this still very raw snapshot will become important.
I attach them in "order of operation:"
[1] Creating the uncertainty on the model parameters themselves which come from,
[a] posterior distributions of an MCMC fit to the reference measurement for the sensing function, C
2019-12-26_H1_forreference_fmin20Hz_sensingFunction_MCMCresults.pdf
[b] posterior distributions of an MCMC fit to each stage of the actuation function, A_UIM, A_PUM, and A_TST, which form the total actuator, A
2019-04-03_H1_20200103Model_REF_UIM_actuationMeasurement.pdf
2019-04-03_H1_20200103Model_REF_PUM_actuationMeasurement.pdf
2019-04-03_H1_20200103Model_REF_TST_actuationMeasurement.pdf
[2] Assembling the large collection of data that will be used to find any residual, unknown systematic error in
[a] the frequency dependence of the model for C,
2020-02-04_GPR_Run_H1_C_allO3BMeas_collection_PCALSYSERR_nofsQcorr_MCMCfmin20Hz_GPRfmin20_lengthscaleZp35_withPCALXHFdata_meas2020-01-20_sensingFunction_referenceModel_vs_allMeasurements.pdf
[b] the frequency dependence of the models for A_UIM, A_PUM, and A_TST
2020-01-13_H1_2020-01-16_GPR_Run_PCALSYSERR_collection_UIM_actuationMeasurement_allresiduals_MCMCInput.pdf
2020-01-13_H1_2020-01-16_GPR_Run_PCALSYSERR_collection_PUM_actuationMeasurement_allresiduals_MCMCInput.pdf
2020-01-13_H1_2020-01-16_GPR_Run_PCALSYSERR_collection_TST_actuationMeasurement_allresiduals_MCMCInput.pdf
accounting for known time-dependent correction factors and the presence of known PCAL systematic error (in measurements prior to 2019-11-12)
[3] The preferred GPR fit to that processed collection of data in [2], for the estimate of the unknown, frequency dependent systematic error
[a] in C, and its associated uncertainty
2020-02-04_GPR_Run_H1_C_allO3BMeas_collection_PCALSYSERR_nofsQcorr_MCMCfmin20Hz_GPRfmin20_lengthscaleZp35_withPCALXHFdata_model2020-01-03_sensingFunction_GPR.pdf
[b] in A_UIM, A_PUM, and A_TST
2020-01-16_GPR_Run_PCALSYSERR_collection_H1_A_model2020-01-03_lastmeas2020-01-13_UIM_actuationMultiGPR_f7-250Hz_L0p1.pdf
2020-01-16_GPR_Run_PCALSYSERR_collection_H1_A_model2020-01-03_lastmeas2020-01-13_PUM_actuationMultiGPR_f7-500Hz_L0p2.pdf
2020-01-16_GPR_Run_PCALSYSERR_collection_H1_A_model2020-01-03_lastmeas2020-01-13_TST_actuationMultiGPR_f10-1000Hz_L0p5.pdf
[4] The "final answer" uncertainty budget computed for the reference time only, when measured TDCFs are either 1.0 for the scalars like \kappa_C, \kappa_UIM, etc. or the exactly the model value for f_cc.
2020-02-04_O3_LHO_GPSTime_1261415486_ref_RelativeResponse1SigmaUncertainty.png
[5] The "final answer" uncertainty budget compared against the two versions of measurements which should be a cross-check and equivalent.
2020-02-04_H1_model20200103_deltaL_over_pcal.pdf
2020-02-04_H1_PCAL2GDS_BB_meas1262990871_on_2020-01-13_vs_model_uncertainty_for_2020-01-03.pdf
In the forthcoming comments below, I'll
- list the scripts that I wrote in order to create these plots.
- quote below the command line used in order to produce the uncertainty estimate.
- note the versions of the pyDARM universe that I committed in order to support the above scripts.
