10:22 UTC Observing Accepted the unknown attached SDF differences.
09:25 UTC No immediately obvious cause.
07:59 UTC Observing Accepted the unknown attached SDF differences.
TITLE: 04/02 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Cheryl
CURRENT ENVIRONMENT:
Wind: 8mph Gusts, 7mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.18 μm/s
QUICK SUMMARY:
Finished initial alignment. Going through relocking steps.
TITLE: 04/02 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Patrick
SHIFT SUMMARY: locked for most of the shift, then a dip in range, and it seemed to be recovering, then lockloss.
LOG:
About 23:56 UTC on 1 April 2019, the CALCS CPU MAX increased from 58 to 63, turning the timing error readback from green to red on the CDS overview. It has since dropped to zero in trend data since, but remains at 63 on the medm. Plots showing the change, and a 10 day trend.
CPU_METER_MAX is a latched value which gets reset when diag_reset is pressed. The attached ndscope trend shows that the cpu_meter of calcs has been fluctuating for some time, and was increased slightly around 16:30 today (PDT). This has increased the probability of it exceeding the maximum processing time of 60uS and triggering the timing bit on STATE_WORD.
cleared the MAX CPU with DIAG RESET, and it dropped to 28, and is currently 35.
Jenne, Sheila, Jeff, Lili
We made a few changes to the pyDARM function actuation.py today.
1) In the testDCgain calculation, there were two minus signs that canceled each other out, as Jenne pointed out. there was a -1 for the linearization sign, which was always applied even though we don't use the linearization. This was canceled by the "attractive" sign, which was due to confusion about the ESD direction. We've established that when the bias sign is positive, the ESD has the same sign as the osems, so there isn't any need to track another minus sign in addition to the bias sign. We've removed these two signs which were canceling each other out, and edited the comments.
2)In process_actuation_measurements, there is some code that takes the measurements and divides out the drivealign filter if the excitation point is calexc, or does nothing if the excitation point is test (after drivealign). This code was also removing the sign of the gain in the drivealign filter, but there was a typo that meant that for the test mass, instead of dividing the gain of the L3 drivealign L2L filter, it was removing the sign of the L2 LOCK filter. Removing this sign isn't necessary, and if it were done as intended there would be an incorrect sign in the plots. This bug wasn't noticed since the gain of the L2 LOCK filter happened to be positive. We have removed the removing of the sign, and simplified the code in this section.
3) In the plots produced by this code, there are some places where the model/measurement is plotted, and some places where 1-|measurement/model| is plotted, which can cause some confusion about the sign. Jeff has been changing these to measurement/model slowly, so I have continued to make that change in the actuator fitting plots.
S. Dwyer, J. Driggers, J. Kissel, L. Sun,
While Sheila spent the day perusing and debugging
^/trunk/Common/pyDARM/src/
actuation.py
in hopes to find the remaining problems with the time-independent path of DELTAL EXTERNAL that continues to plague us (see LHO aLOG 48102 for the latest on that),
Lilli created a DARM loop model that replicates what works in CAL-CS,
^/trunk/Runs/O3/H1/params/
modelparams_H1_20190401_ER14_095A.py
-- see LHO aLOG 48115 for more details.
Once Jenne was convinced that the replica SUS oscillators were a major problem with the computation of the time-dependent correction factors (see LHO aLOG 48129),
I figured that we should update the Reference Time Model Parameters at Calibration Line Frequencies a.k.a "EPICS records" in front-end such, regardless of whether things are perfect, the time-dependent correction factors are at least functional and their parameter values are close.
I generated the EPICs records with
^/trunk/Runs/O3/H1/Scripts/CALCS_FE/
createEPICS_for_20190401.py
which is calling the above mentioned model, and a few functions inside
^/trunk/Common/pyDARM/src/
computeDARM.py
which I've updated to write with 16 significant figures. ***
Attached are the results and a screenshot of the new EPICs records.
Further, I attach a time series of the time-dependent correction factors. One can see
- The values are
kappa_C = 0.99 virtually spot on with the reference model
f_cc = 404 Hz virtually spot on with the reference model
f_s = 13 Hz # bogus because calculation is for anti-spring
Q_s = 0.5 Hz # bogus because calculation is for anti-spring
kappa_T = 0.591 wrong, don't yet know why
kappa_P = 1.054 5%... coincident with what's needed to fix the PCAL2DELTAL TF? Don't understand why, could be coincidence
kappa_U = 1.051 5%... coincident with what's needed to fix the PCAL2DELTAL TF? Don't understand why, could be coincidence
- the time-series of the past 3000 seconds indicate what we've see that these TDCFs are just as stable as the IFO, confirmed by broad band injections (see LHO aLOG 48098)
These time-dependent correction factors are still not actually correcting anything, so the systematic error we claim still remains -- 1% / 5 deg. Again, see LHO aLOG 48102.
The EPICs records have been accepted into both the safe.snap and OBSERVE.snap files, and have been committed to
^/trunk/Runs/O3/H1/Results/CALCS_FE/
epicsrecords_model-H1_20190401_created-20190401.txt
*** We tried to reduce the number of significant figures of the EPICs records such that we wouldn't constantly have to battle the floating point precision comparison with SDF, this we are now writing the records to EPICS with the lines
subprocess.call(['caput',IFO+':'+chan+'_REAL','{:0.16e}'.format(chan_real)])
subprocess.call(['caput',IFO+':'+chan+'_IMAG','{:0.16e}'.format(chan_imag)])
to limit the write to EPICs to 16 significant figures, i.e. the limit of floating point precision. However, differences still have shown up when the .snap file is compared against the running values. We *can* limit the precision of these numbers further, but we want to make sure that limiting the precision on these numbers won't impact the precision of the finally computed time-dependent correction factors.
Quick correction to Jeff's alog:
In the values of kappas that he quotes, kappa_TST is near 1.05 (good), and kappa_UIM is the one that is pretty far off at 0.591.
JoeB reminded us to check on the phase between the actual suspension oscillators at the end stations, and the replicas of those oscillators used for demodulation at the corner station. Similar to the problem that he reported back in February (LLO alog 43375), our oscillators had a fixed phase between them.
However, the phase delays here did not correspond to integer seconds of delay, which is concerning. At LLO (and we will copy in a week or two) they are going to move the CAL-CS model back to a computer than has an I/O chassis so that the timing will be right, and they hopefully won't get these timing slips. That should solve the hypothesized problem of the slips coming from the timing not being right. However, our phase delays not corresponding to integer seconds indicates that there may be yet another problem. Unclear.
The good news is that the trick of forcing a re-sync by zeroing the frequency of the oscillator and then resetting it to its nominal value has removed the phase delay. In the attached figure, the blue reference is the phase delay that we had before re-syncing the oscillators, and current red shows that all of the delays are zero, as they should be.
Also, we pushed new EPICS records today, corresponding to the modelparams file that Lilly made earlier today (alog 48115). Jeff will comment further on those.
With these 2 changes, the kappa_TST and kappa_PUM are reporting sensible numbers, about 1.05 each. However, the UIM is still wonky. As has been suggested elsewhere, perhaps since this is our lowest frequency actuator stage we are getting this one wrong due to imperfect modeling of the sensing function (which goes into the 'reference values' that are part of the kappa calculation)? The intrepid alog reader will note that this kappa value for the lower 2 stages of 1.05 sounds quite suspiciously related to the values of 0.95 that we have put into the H1:CAL-CS_DARM_ANALOG_ETMX_L[1, 2, 3]_GAINs. However, since those affect CAL-DELTAL_EXTERNAL, but not the DARM_ERR that is used for the kappa demodulation, setting the actuator gains to 1 doesn't change the kappas. So, there's still some 5% business that requires more thought. And, we need to figure out what's up with the UIM kappa.
Changed the nominal and tolerance for the PEM-C_MSR_RACK[12] channels to 22C and 10C, respectively.
Changed the ALS-X_REFL_SERV_COMOFS from +0.25 to +0.60 to get the ALS VCO in End X away from its rail, when the green is shuttered.
TITLE: 04/01 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 109Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
Wind: 10mph Gusts, 8mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.15 μm/s
QUICK SUMMARY: H1 has been locked almost 20 hours, winds are decreasing.
Out of Observe for Calibration record changes and tweaks to OMC.
Back in Observe
TITLE: 04/01 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 109Mpc
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY: 19hr lock with nothing much to report. ALSX polarization is reaching 20% and may need to be adjusted on the next lock loss.
LOG:
I've written a python EPICS IOC (h1_wap_ioc.py) which reports the status of the CDS WAPs. The IOC ping's the WAPs once a minute. After each ping cycle, a timestamp is updated to show data validity.
A WAP MEDM screen is available from the sitemap under the CDS pull down. The status of corner and end station WAPs are summarized on the CDS overview in the bottom right corner, along with a button to open the WAP MEDM.
This system uses O3 colours, that is to say GREEN is nominal (the WAP is OFF), RED is non-nominal (the WAP is ON).
The attached image shows the WAP MEDM and the WAP section on the CDS Overview MEDM.
BTW: Keita has said that we can leave the WAPs in the midstations ON.
Jenne and I saw this glitch on the X arm around 11 am. We think it is interesting to see it in high frequency. Here are the screen shots. They are all of the same data, but they are zoomed in on different times so that you can see some of the details more clearly. It is cool to see that it's definitely a glitch in length, and that the pitch follows shortly afterward. We were looking at these channels for other reasons, but since they are not saved to frames it seemed interesting to capture them.
Both PSL chiller filters look good. The Crystal Chiller filter appears a bit more yellow than the Diode Chiller. I believe this to be OK, as this is a different type of filter than the Diode Chiller filter. I will check the filters in the PSL enclosure at next access to see if there is any discoloration in the filter under the PSL table. No water was added to either chiller. Closing FAMIS #8611