19:09 back to Observing.

J. Kissel

I repost the table, without typos in the (new analysis) columns in actuation:
                                                                           MAP     (68% C.I.)  MAP       (68% C.I.)

 Date                                                  2016-11-12          2016-11-12          2017-01-03
                                                   (former analysis)       (new analysis)       (new analysis)
 Optical Gain                K_C         [ct/m]        1.15e6              1.164e6 (0.09%)     1.086e6   (0.07%)
 Couple Cav. Pole Freq.      f_c         [Hz]          346.7               346.7   (0.4%)      360.4     (0.4%)
 Residual Sensing Delay      tau_C       [us]          2.3                 2.3     (50%)       0.65      (200%)
 SRC Detuning Spring Freq.   f_s         [Hz]          7.4                 7.4     (1.1%)      6.9       (0.5%)
 Inv. Spring Qual. Factor    1/Q_s       [ ]           0.05                0.05    (8.7%)      0.03      (12%)

 UIM/L1 Actuation Strength   K_UIM       [N/ct]        8.164e-8            8.104e-8  (0.2%)    8.091e-8  (0.2%)
 PUM/L2 Actuation Strength   K_PUM       [N/ct]        6.841e-10           6.773e-10 (0.03%)   6.768e-10 (0.02%)
 UIM/L3 Actuation Strength   K_TST       [N/ct]        4.389e-12           4.347e-12 (0.04%)   4.357e-12 (0.02%)
Note that (new analysis) sensing function parameter fits are MCMC results, not nlinfit results, which are what is used for updating the DARM loop model. However, MCMC results are consistent with the uncertainty of the nlinfit results, so we consider the results interchangeable. Rather, we use the MCMC results for response function uncertainty, where we use nlinfit results to update the DARM loop model -- we should and will use MCMC results for updating the DARM loop model in the future such that we have a self-consistent pipeline from reference measurement analysis to response function uncertainty estimation. 

I attach an update to the sensing function plot for each reference data set which improves the MCMC fit results vs. measurement plot to show residuals and parameter values with uncertainties, such that it can now be directly compared to the same plot produced by the nlinfit code.

As promised, a complete description of how these results were generated:

            :::: How I generated 2016-11-12 (new analysis) ::::
---------
- Updated measurement base-level processing scripts and DARM model, which fixes the AA/AI filter gain analysis bug identified in LHO aLOG 32907),
${CalSVN}/aligocalibration/trunk/Runs/O2/DARMmodel/src/
    DARMmodel.m        (r4093, lc: r3511)
    computeActuation.m (r4093, lc: r4003)
    computeSensing.m   (r4093, lc: r4025)

---------
- Generated a new fit for actuation parameters using MCMC analysis:
pre-process actuation measurements using
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Scripts/FullIFOActuatorTFs/
        actuatorCoefficients_Npct.m (r4065, lc: r4065)
which spits out text files of the actuation function for each stage with portions known to negligible uncertainty removed (e.g. frequency dependence of SUS dynamics, AI filter shape, etc.),
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Measurements/FullIFOActuatorTFs/2016-11-12/
        2016-11-12_H1SUSETMY_L1_actuationStrength_Npct.txt (r4111, lc: r4111)
        2016-11-12_H1SUSETMY_L2_actuationStrength_Npct.txt (r4111, lc: r4111)
        2016-11-12_H1SUSETMY_L3_actuationStrength_Npct.txt (r4111, lc: r4111)
which are then loaded into
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Scripts/FullIFOActuatorTFs/
        fitActCoefs_Npct.m (r3977, lc: r3977)
where each stage is fit independently, so you have to switch the variable "stage" between 'L1', 'L2', or 'L3'.
Running on ONLY the 2016-11-12 reference measurement time (instead of two days of measurements, as Darkhan originally did)
   dateListRef = {'2016-11-12'};
   gpsListRef = [1162969230];

yields the above results. 

There's a typo in the above table: the new analysis of L3/TST actuator coefficient is 4.347e-12 [N/ct], as shown in the above attachement https://alog.ligo-wa.caltech.edu/aLOG/uploads/32989_20170105090730_2016-11-12_H1_SUSETMY_ACT_COEF_REF_fitCorner.pdf

---------
- Generated a new fit for the sensing function parameters using nlinfit and MCMC analysis:
pre-process sensing function measurements with
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Scripts/PCAL/
        fitDataToC_20161116.m (r4040, lc: r4028)
which produces an nlinfit list of parameters on the IFO optical plant (i.e. with stuff known to negligible uncertainty removed, like frequency dependence of OMC DCPD high frequency electronics), and also spits out a text file of that IFO optical plant for MCMC fitting,
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Results/Sensing
        2016-11-12_H1_meas_sensingTF_withoutDCPDTFs_ctpm.txt (r4079, lc: 4072)
This is loaded into 
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Scripts/PCAL/
        fitCTF_mcmc.m (r4112, lc: r4112)
to produce the MCMC maximum a posteriori (MAP).

Even though all MCMC results are repeated using python code, the sensing function parameter's posterior distribution from the above matlab MCMC code are exported to text:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/ER10/H1/Results/Sensing/
2016-11-12_H1_sensing_burn80p_nWalk100_output.txt      (r4079, lc: r4077)
2016-11-12_H1_sensing_burn80p_nWalk100_posteriors.txt  (r4079, lc: r4079)
Actuator parameter posterior distributions are not yet exported from the matlab code.

            :::: How I generated 2017-01-03 (new analysis) ::::
--------
- Used the updated versions of same the base-level processing scripts described above for 2016-11-12 (new analysis)

--------
- Generated a fit for actuation parameters using MCMC analysis:
Above mentioned actuation function pre-processing script copied from ER10 directory to O2 directory (prior to making aesthetic changes to the script in rev 4065),
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Scripts/FullIFOActuatorTFs/
        actuatorCoefficients_Npct.m (r3811, lc: r3811)
became
    ${CalSVN}/aligocalibration/trunk/Runs/O2/H1/Scripts/FullIFOActuatorTFs/
        fitDataToA_20170103.m (r4095, ls: r4061)                                (Note the intentional name change to match sensing function scripts)
which produces text files of actuation functions with frequency dependence known to negligible uncertainty removed,
    ${CalSVN}/aligocalibration/trunk/Runs/O2/H1/Results/FullIFOActuatorTFs/    (Note change in location because the text files are a result,
        2017-01-03_H1SUSETMY_L1_actuationStrength_Npct.txt (r4095, lc: r4056)    not a part of the measurements)
        2017-01-03_H1SUSETMY_L2_actuationStrength_Npct.txt (r4095, lc: r4056)
        2017-01-03_H1SUSETMY_L3_actuationStrength_Npct.txt (r4095, lc: r4056)
which are analysized by the MCMC code copied over from ER10 to O2,
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Scripts/FullIFOActuatorTFs/
        fitActCoefs_Npct.m (r3977, lc: r3977)
became
    ${CalSVN}/aligocalibration/trunk/Runs/O2/H1/Scripts/FullActuatorTFs/
        fitATF_mcmc_20170103.m                                                  (Note the intentional name change to match sensing function scripts)
and again, run individually for each stage on 2017-01-03 reference measurement.

--------
- Generated a fit for sensing function parameters using nlinfit and MCMC analysis,
pre-process sensing function measurements with a new copy of
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Scripts/PCAL/
        fitDataToC_20161116.m (r4040, lc: r4028)
now named
    ${CalSVN}/aligocalibration/trunk/Runs/O2/H1/Scripts/SensingFunctionTFs/     (Note folder change to match actuator scripts and the rest
        fitDataToC_20170103.m (r4095, ls: r4090)                              of O2 directory structure)
which produces an nlinfit list of parameters on the IFO optical plant (i.e. with stuff known to negligible uncertainty removed, like frequency dependence of OMC DCPD high frequency electronics), and also spits out a text file of that IFO optical plant for MCMC fitting,
    ${CalSVN}/aligocalibration/trunk/Runs/O2/H1/Results/Sensing
        2017-01-03_H1_meas_sensingTF_withoutDCPDTFs_ctpm.txt (r4095, lc: 4056)
This is loaded into a copy of
    ${CalSVN}/aligocalibration/trunk/Runs/ER10/H1/Scripts/PCAL/
        fitCTF_mcmc.m (r4041, lc: r4041)
    (copied prior to aesthetic changes to fit vs. measurement plot which now shows residuals in r4112 as described above)
which is now
    ${CalSVN}/aligocalibration/trunk/Runs/O2/H1/Scripts/PCAL/
        fitCTF_mcmc_20170103.m (r4095, lc: r4091)
to produce the MCMC maximum a posteriori (MAP) (and also has the same aesthetic changes as the ER10 version, r4112).

Even though all MCMC results are repeated using python code, the sensing function parameter's posterior distribution from the above matlab MCMC code are exported to text:
${CalSVN}/aligocalibration/trunk/Runs/O2/H1/Results/SensingFunctionTFs/
2017-01-03_H1_sensing_burn80p_nWalk100_output.txt       (r4095, lc: 4058)
2017-01-03_H1_sensing_burn80p_nWalk100_posteriors.txt   (r4095, lc: 4058)
Actuator parameter posterior distributions are not yet exported from the matlab code.
     
----------
A super-duper thanks to Darkhan, Kiwamu, and Evan for writing the brunt of these functions. It took a while to find everything, how the analysis flow was to go, and how to correctly into file names, but it otherwise works like a charm with very reproducible results. I look forward to making the functions even more streamlined, and potentially merging functionality so there's not so many functions of which to keep track. 
----------

Incidentally, running the a2l script from th MEDM launch screen seems to crash all of the MEDM screens. I thought, at first that it was only the new Ops workstation but I now see it happening on the other workstation as well.

back to Observing @ 05:23UTC

Re: running a2l from sitemap->sus->a2l screen crashes medm screens

There was a typo on the a2l button that I must have added before the break. There was an apostrophe at the end of the command for running the a2l script, this obviously was doing something bad. It was actually my punctuation all out of order.  Needed to be ' &  not  & '  .  It's fixed now, a2l should run normally from this screen.

Pressure reading from PT180 is once again rising after being valved into main volume. We suspect the water load is causing this. End stations show no sign but also get 2.5x more pumping speed from cyropumps due to smaller volume.

PT140 pressure reading on diagonal volume (no crypopump) started dropping this week after a three month incline. These changes are due to LVEA temperature fluctuations.

Correction:  water pumping speed is more like 1.8x more at end stations. 

Corner volume = 445,000 liters

End volume = 88,000 liters

Corner cyropumping = [100,000 - 3,000] x 2 (L/s)

End cryopuming = 100,000 - 30,000 (L/s)

More plots from beginning of O2 (Nov 30) to show that these filters still have the right model and EPICS.

Same set of plots one more time, this time in early ER10 (Nov 16). Note that kappas were not applied in the GDS pipeline this time, leading to a notable difference in the spectra.

These filters have been updated to account for corrections made to the DARM loop parameters since the AA/AI filter bug fixes. For information on the model changes, see:
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=33153

The updated filters were produced using all the same files (updated versions) in SVN revision #4133. The only exception is that the EPICS file and the parameters file to produce it were:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/ER10/H1/Scripts/CAL_EPICS/DCS20161112_H1_CAL_EPICS_VALUES.m
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/ER10/H1/Scripts/CAL_EPICS/callineParams_20161118.m

Note from the plots the slight discrepancy between GDS and DCS, presumably due to the corrections to the model. Also note that DCS and CALCS do not agree on the kappas. This is likely not cause for concern, as the model used to compute them was different. The EPICS and pcal correction factors were produced using the same parameter files as the filters, so they should be correct.

Apologies. We've been at NLN for about that long. In Observation for only about 1 hour.

Seems like H1:DMT-CALIBRATED is 0 (zero) not 1. Is this related to the calibration task performed today?

Is this why GWI stat thinks that H1 is not OK?

Sent a message to Jeff Kissel, Aaron Viets and Alex Urban.

I tried a few things to see if I could figure out why the calibration flag wasn't set. 

1) restarted the redundant calibration pipeline, This probably caused some of the backup frames to be lost but the primary and low latency frames would not be affected. The Science_RSegs_H1 process 

 https://marble.ligo-wa.caltech.edu/dmt/monitor_reports/Science_RSegs_H1/Segment_List.html

is generating segments from the output of the (restarted) redundant pipeline, but it is getting the same results.

2) Checked for dataValid errors in the channels in the broadcaster frames. dataValid would probably cause the pipeline to flush the h(t) data. No such errors were found

3) checked for subnormal/Nan data in the broadcaster frames. Another potential proble,m tha tmight cause the pipeline to flush the data. No problems of this type were found either.

4) checked pipeline log file - nothing unusual

5) Checked for frame errors or broadcaster restarts flagged by the broadcast receiver. Last restart was Dec 5!

So, I can see no reason for the ht pipeline to not be running smoothly.

Alex U. on behalf of the GDS h(t) pipeline team

I've looked into why the H1:DMT-CALIBRATED flag is not being set, and TL;DR: it's because of the kappa_TST and kappa_PU factors.

Some detail: the H1:DMT-CALIBRATED flag can only be active if we are OBSERVATION_READY, h(t) is being produced, the filters have settled in, and, since we're tracking time-dependent corrections at LHO, the kappa factors (except f_CC) must each be within range -- outside of 10% their nominal value, the DMT-CALIBRATED flag will fail to be set. (See the documentation for this on our wiki page: https://wiki.ligo.org/viewauth/Calibration/TDCalibReviewO1#CALIB_STATE_VECTOR_definitions_during_ER10_47O2)

I attach below a timeseries plot of the real and imaginary parts of each kappa factor. (What's actually plotted is 1 + the imaginary part, to make them fit on the same axes.) As you can see, around half an hour or so in, the kappa_TST and kappa_PU factors go off the rails, straying 20-30% outside their nominal values. (kappa_C, which is a time-dependent gain on the sensing function, and f_CC both stay within range during this time period.)

Earlier today, Jeff reported on some work done with the L2/L3 actuation stages (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=32933) which may in principle affect kappa_TST and kappa_PU. It's possible we will need a new set of time domain filters to absorb these changes into the GDS pipeline. (I also tried a test job from the DMT machine, but the problems with kappas were still present, meaning a simple restart won't solve the problem.)

GWIstat (also the similar display gwsnap) was reporting that H1 was down because of the h(t) production problem; it did not distinguish between that and a down state.  I have now modified GWIstat (and gwsnap) to indicate if there is no good h(t) being produced but otherwise the detector is running.

The attached pdf shows that CALCS and GDS agree on the calculation of kappa_tst. I suspect we may need to calculate new EPICS. Jeff (or perhaps Evan or Darkhan) will need to confirm this based on the recent L2/L3 crossover changes that Alex pointed out.

Here is a comparison between h(t) computed in C00 frames (with kappas applied) and the "correct"-ish calibration, with no kappas applied. The first plot shows the spectra of the two from GPS time 1167559872 to 1167559936. The red line is C00, and the blue line has no kappas applied. The second plot is an ASD ratio (C00 / no-kappas-applied) during the same time period. 
The cache file that has the no-kappas-applied frames can be found in two locations:
ldas-pcdev1.ligo-wa.caltech.edu:/home/aaron.viets/H1_hoft_GDS_frames.cache
ldas-pcdev1.ligo-wa.caltech.edu:/home/aaron.viets/calibration/H1/gstreamer10_test/H1_hoft_GDS_frames.cache

Also, the file
ldas-pcdev1.ligo-wa.caltech.edu:/home/aaron.viets/H1_hoft_test_1167559680-320.txt
is a text file that has only h(t) from GPS time 1167559680 to 1167600000.