jeffrey.kissel@LIGO.ORG - posted 16:39, Thursday 02 April 2020 - last comment - 16:39, Thursday 02 April 2020(55852)
H1 O3 B, C01 Chunk 1 Uncertainty Budget -- Still Disagrees with BB PCAL Injection
J. Kissel, Now that we've got DCS C01 h(t) data for O3 B Chunk 1 (and other associated calibration data products like TDCFs) successfully produced, it's time to pick up where I've left off on producing an uncertainty budget for it. Chunk 1 is covering data from the start of O3 B, Nov 1 2019 15:00 UTC to Jan 14 2020 18:00 UTC. Recall, the last time I touched this was on Feb 20 2020, when I was still confused as to why the broadband PCALY injection to h(t) disagreed with the estimated uncertainty envelope -- see LHO aLOG 55203. The hope was that, with C01 data, the static model, and the time-dependent correction factors would be "super right," and it would adjust the *uncertainty budget's* predictions of median and 68% confidence interval envelope and it would all nicely line up, as has been true in the past (e.g. in LHO aLOG 52145). Sadly, using C01 TDCFs made no difference in modifying the median or envelope. See attached .pdf. In it, we see a bode plot of the measurement, /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/GDS_BB_plots/H1_C00_over_CAL-PCALY_RX_PD_OUT_DQ_1262990871-153.txt against the uncertainty budget produced in three ways ? each with the same input MCMC and GPR .hdf5 files, but requesting to use: (a) No TDCFs (i.e. --version=ref --gpsTime=1261415486) (b) With C00 TDCFs (i.e. --version=C00 --gpsTime=1262998218 --GetData=GWPY) (c) With C01 TDCFs (i.e. --version=C01 --gpsTime=1262998218 --GetData=GWPY ) where the parenthetical statements are the differing arguments to our uncertainty budget code, /ligo/svncommon/CalSVN/aligocalibration/trunk/Common/pyDARM/RRNom.py I'll add a comment below with the exact commands elucidating what other parameters were used, including the MCMC and GPR .hdf5 files. I'll at least say here, that I'm using measured sweep data collected for GPR analysis up to 2020-02-24, when we have 15 sensing function measurements, and 6 to 7 actuation function measurements, so we're in the region of diminishing returns. You'll notice that all three uncertainty budgets, (a) through (c), look virtually identical. [EDIT] Lilli reminds me that the *uncertainty budget* is *unaffected* by the values of the TDCFs. (i)We correct for "all" TDCFs (except for f_{s} and Q_{s}) in the live h(t) data, i.e. in the GDS/C00 and DCS/C01 data streams. That means that the TDCFs are *no longer* systematic error error. Thus, we *don't* include them in the systematic error budget. So, the only difference between C00 and C01 should be *the uncertainty* on the TDCFs, which doesn't change much at all, because the calibration lines which determine them are very loud. (ii) Separately, during the reference time, we don't need to include any TDCFs, or, said differently, they are *defined* to not modify the data; all scalar TDCFs, the "kappas" are set to unity, and the frequency dependent TDCFs (of which, there's only one -- f_cc -- because we're excluding f_{s} and Q_{s}). That's because the detector is already behaving exactly as the reference time model. It's OK if you're confused by this, we're continually confused by it too, due to the way we construct these budgets being quite convoluted. We're finding this out as we try to explain what we're doing in the O3 A systematic error paper. C'est la vie. It'll be better in O4. [/EDIT]This indicates two things to me: (i) That the time dependent correction factors at 1262998218 (Jan 14 2020 00:50:00 UTC) -- which is much after the reference measurement time, 1261415486 (Dec 26 2019 17:11:08 UTC) -- are actually not correcting for much at all -- which means the detector at these two times were quite stable and reproducible. GREAT! (ii) The difference between TDCFs calculated in C00 and C01 are also, no different from each other. This is also unsurprising, and encouraging -- the time I'm using 1262998218 (Jan 14 2020 00:50:00 UTC) is in the observing segment *right* after the 2020-01-03 model was installed on 2020-01-13. So this confirms that C00 is *just as good* as C01 *after* 2020-01-13. GREAT!However -- we're still *at* the drawing board with why these don't agree. We've got a few ideas from here: (1) Aaron had mentioned some (at the time) recently discovered issue with the TST driver, which he eluded to the day prior to my 2020-02-20 aLOG. The plot he showed us was this one, the description of which I repeat here for convenience: That plot shows the response function with (in blue) and without (in maroon) the systematic error present. You could convince yourself that the discrepancy between the GDS measurement (which would have this TST flaw) and the uncertainty envelope is *about* what you see as the blue curve in that preliminary plot. But, stay tuned for the analysis to be done right -- applying the correction in "the right direction" and putting these things on the same plot. (2) We've really only compared this exact same measurement, every time, to the uncertainty budget. It's a small hope, but still a hope, that if we compare several times of BB injections -- as well as swept sine measurements -- like was done in LHO aLOG 52145, then we might find that this measurement is just a weird, outlying measurement. but I'm not super excited that we'll understand either quicky. I'm *really* hoping that it's the PCAL / GDS measurement that's been processed in correctly, and/or it's an outlier. Soooo -- yet again -- stay tuned.
Non-image files attached to this report
Comments related to this report
Here're the highlights of how I've produced these uncertainty budgets. All details are recorded in T2000006-v5_notes_CalModelParameterSetGeneration_20190103.txt. The .hdf5 files that contain posterior distributions of all model parameters determined by MCMC live here: For A /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/ O3B_H1_A_MCMC_20200103Model_REF.hdf5 For C /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/ O3_H1_C_MCMC_2019-12-26_forreference_fmin20Hz.hdf5 These were produced by the following scripts, respectively, /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Scripts/FullIFOActuationTFs/ process_actuation_MCMC_model20200103_meas20190403_forreference.py /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Scripts/FullIFOSensingTFs/ process_sensingmeas_MCMC_model20200103_meas20191226_forreference.py For the unknown frequency dependent error, the Gaussian process regression posterior distribution .hdf5 files are here: For A /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/ 2020-02-24_GPR_Run_H1_A_O3B_meas20191204-20200224_collection_model20200103_NOPCALSYSERR_ALL_UIM_f7-250Hz_L0p1_PUM_f7-500Hz_L0p2_TST_f10-1000Hz_L0p5_posteriors.hdf5 For C /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/ 2020-02-24_GPR_Run_H1_C_O3B_meas20191120-20200224_collection_model20200103_NOPCALSYSERR_nofsQcorr_MCMCfmin20Hz_GPRfmin20_lengthscale0p35_withPCALXHFdata_model20200103_posteriors.hdf5 These were produced by the following scripts, respectively, /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Scripts/Uncertainty/ process_allmeas_writeGPRHDF5_20200224_A_meas20191204-20200224_model20200103_NOPCALSYSERR.py /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Scripts/Uncertainty/ process_allmeas_writeGPRHDF5_20200224_C_meas20191120-20200224_model20200103_NOPCALSYSERR_withHF.py (with a discussion of the RBF length scales I used for the GPR kernels are in LHO aLOG 55012) Using the above mentioned .hdf5 files, the exact calls I used to create the uncertainty budgets are therefore: (a) python /ligo/svncommon/CalSVN/aligocalibration/trunk/Common/pyDARM/RRNom.py --IFO=LHO --HDF5_A_MCMCresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/O3B_H1_A_MCMC_20200103Model_REF.hdf5 --HDF5_A_GPRresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/2020-02-24_GPR_Run_H1_A_O3B_meas20191204-20200224_collection_model20200103_NOPCALSYSERR_ALL_UIM_f7-250Hz_L0p1_PUM_f7-500Hz_L0p2_TST_f10-1000Hz_L0p5_posteriors.hdf5 --HDF5_C_MCMCresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/O3_H1_C_MCMC_2019-12-26_forreference_fmin20Hz.hdf5 --HDF5_C_GPRresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/2020-02-24_GPR_Run_H1_C_O3B_meas20191120-20200224_collection_model20200103_NOPCALSYSERR_nofsQcorr_MCMCfmin20Hz_GPRfmin20_lengthscale0p35_withPCALXHFdata_model20200103_posteriors.hdf5 --modelPath=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/params/ --modelFilename=modelparams_H1_20200103 --IFOmodel=modelPars --sampleNumber=1000 --seed=1111 --version=ref --outDir=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/ --plot1SigmaUncs --saveSummaries --gpsTime=1261415486 (b) python /ligo/svncommon/CalSVN/aligocalibration/trunk/Common/pyDARM/RRNom.py --IFO=LHO --HDF5_A_MCMCresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/O3B_H1_A_MCMC_20200103Model_REF.hdf5 --HDF5_A_GPRresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/2020-02-24_GPR_Run_H1_A_O3B_meas20191204-20200224_collection_model20200103_NOPCALSYSERR_ALL_UIM_f7-250Hz_L0p1_PUM_f7-500Hz_L0p2_TST_f10-1000Hz_L0p5_posteriors.hdf5 --HDF5_C_MCMCresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/O3_H1_C_MCMC_2019-12-26_forreference_fmin20Hz.hdf5 --HDF5_C_GPRresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/2020-02-24_GPR_Run_H1_C_O3B_meas20191120-20200224_collection_model20200103_NOPCALSYSERR_nofsQcorr_MCMCfmin20Hz_GPRfmin20_lengthscale0p35_withPCALXHFdata_model20200103_posteriors.hdf5 --modelPath=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/params/ --modelFilename=modelparams_H1_20200103 --IFOmodel=modelPars --sampleNumber=1000 --seed=1111 --version=C00 --outDir=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/ --plot1SigmaUncs --saveSummaries --gpsTime=1262998218 --GetData=GWPY (c) python /ligo/svncommon/CalSVN/aligocalibration/trunk/Common/pyDARM/RRNom.py --IFO=LHO --HDF5_A_MCMCresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/O3B_H1_A_MCMC_20200103Model_REF.hdf5 --HDF5_A_GPRresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/2020-02-24_GPR_Run_H1_A_O3B_meas20191204-20200224_collection_model20200103_NOPCALSYSERR_ALL_UIM_f7-250Hz_L0p1_PUM_f7-500Hz_L0p2_TST_f10-1000Hz_L0p5_posteriors.hdf5 --HDF5_C_MCMCresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/O3_H1_C_MCMC_2019-12-26_forreference_fmin20Hz.hdf5 --HDF5_C_GPRresults=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/2020-02-24_GPR_Run_H1_C_O3B_meas20191120-20200224_collection_model20200103_NOPCALSYSERR_nofsQcorr_MCMCfmin20Hz_GPRfmin20_lengthscale0p35_withPCALXHFdata_model20200103_posteriors.hdf5 --modelPath=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/params/ --modelFilename=modelparams_H1_20200103 --IFOmodel=modelPars --sampleNumber=1000 --seed=1111 --version=C01 --outDir=/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Results/Uncertainty/ --plot1SigmaUncs --saveSummaries --gpsTime=1262998218 --GetData=GWPY where for C00 and C01 data I've chosen 1262998218 (Jan 14 2020 00:50:00 UTC), because this is just after the start of the DMT-ANALYSIS_READY segment which resumed after we updated the model on 2020-01-13 between 21:50:08 UTC and 22:47:32 UTC.