craig.cahillane@LIGO.ORG - posted 10:22, Thursday 05 November 2015 - last comment - 13:47, Thursday 05 November 2015(23137)
LHO O1 Calibration Uncertainty
I have posted the latest LHO calibration uncertainty plots. I have reduced our kappas' uncertainty to 1% in mag and 0.5 degrees in phase. We are now limited by our measurements. I believe that our "statistical uncertainty only" plots are underestimating error. This is because the systematic error that remains in our measurements is ignored when we consider only the uncertainty bars. One way to combat this underestimation is to not fit our systematic to the final weighted mean of all our measurements, but instead fit to each of our measurements and take the std of our remaining systematic errors as our total systematic error. This ought to fix the fact that I was ignoring remaining systematic errors in our "statistical uncertainty only" plots. Plots 1-4 show nominal O1 model mag and phase uncertainty (the statistical uncertainty and systematic error summed in quadrature). Plots 5-8 show the systematic corrections model mag and phase uncertainty (statistical uncertainty only). I believe these are currently underestimated. Plot 9 is the comparison of the nominal O1 calibration response function to the systematic corrections model I make. (The red line is the systematic correction model, the dashed lines are the associated uncertainty)
Non-image files attached to this report
Comments related to this report
Several follow up comments:
- On describing I would say it differently, plots 1-4,
(1) 05-Nov-2015_Strain_Uncertainty_Magnitude.pdf
(2) 05-Nov-2015_Strain_Uncertainty_Phase.pdf
(3) 05-Nov-2015_Strain_Uncertainty_Squared_Magnitude_Components.pdf
(4) 05-Nov-2015_Strain_Uncertainty_Squared_Phase_Components.pdf
are not the "nominal" uncertainty. These are the uncertainty if we incorrectly add systematic errors with statistical uncertainty in quadrature (i.e. implying we don't know the sign of systematic error and/or that they don't affect the mean of the Gaussian distribution, which we do and we know does). The reason we show these is to show how the uncertainty has traditionally been quoted, given the level of sophistication search algorithms had been able to handle. It's also much each to make a "single number" statement from this curve, which is what most people want so they can discuss the uncertainty colloquially.
Now that parameter estimation groups are interested in greater detail (i.e. have asked questions like "what do you *really* mean by '10%'??"), and we have solid handles on some of our systematic errors, we can offer an alternative display of the statistical uncertainty ONLY, namely plots 5-8,
(5) 05-Nov-2015_Strain_Uncertainty_Magnitude_Systematics.pdf
(6) 05-Nov-2015_Strain_Uncertainty_Phase_Systematics.pdf
(7) 05-Nov-2015_Strain_Uncertainty_Squared_Magnitude_Components_Systematics.pdf
(8) 05-Nov-2015_Strain_Uncertainty_Squared_Phase_Components_Systematics.pdf
and then display the resulting affect on the mean in plot 9,
(9) 05-Nov-2015_Strain_Uncertainty_Systematics_vs_Nominal_Residuals.pdf
Black solid shows a zero mean error on the response function, with statistical uncertainty and systematic error incorrectly added in quadrature, shown in black dashed. Red solid shows the mean, corrected by the systematic errors, with the correct statistical uncertainty surrounding it.
- The f > 1 [kHz] statistical uncertainty is more ill-defined than reported. We simply don't have the measurements up there to confirm such small precision, so Craig, for the time being, has merely cut off the uncertainty at the last frequency sweep's data point (~900 [Hz]) and used that as the default value out to the limit of the frequency range. As such, the uncertainty appears to be limited by the 1% / 0.5 [deg] statistical uncertainty of the calibration lines, translated from lower-frequency (~330 [Hz]) because that's we scale of the overall optical gain in the sensing function. While we don't expect the uncertainty to be much larger at high frequency, we simply don't have any quantitative upper bound. Nor do we have any idea what kind of systematics dragons tharr be. As such, I suggest we continue take the f > 1 [kHz] uncertainty with a grain of salt.