1315 -1400 hrs. local -> To and from Y-mid (then X-mid, unrelated) Opened exhaust check-valve bypass-valve. Opened LLCV bypass-valve 1/2 turn -> LN2 @ exhaust in 55 seconds -> Restored valves to as found configuration. Next CP3 overfill to be Monday, July 11th.
The IFO top node appears to be not monitoring some SEI BS and ALS nodes. The IFO top node is reporting OK, even though multiple nodes are not reporting OK:
This is an indication of two issue:
The USERAPPS/h1/guardian/IFO_NODE_LIST.py that is in the SVN shows that those nodes should be managed, so my guess is that what's currently running has not been committed to the SVN.
My advice would be to not focus too much on a potential O2 configuration, but focus instead right now on the ER9 configuration. There's no problem in updating the NOMINAL states as needed. They should be set to whatever is appropriate for ER9, and make sure they're all monitored by the top node. We can then adjust things as needed for O2.
In other words, IFO should always be monitoring all nodes, and we should just adjust the NOMINAL state for each node to reflect whatever is the ideal IFO configuration at the moment.
I confirmed that the USERAPPS/sys/h1/guardian/IFO_NODE_LIST.py file has indeed been modified locally to remove monitoring of the SEI BS and ALS nodes, and that those changes have not been committed to the SVN.
Many guardian files have local modifications that have not been committed to the SVN:
jameson.rollins@operator1:~ 0$ guardlog list | xargs -l guardutil files 2>/dev/null | sort | uniq | xargs -l svn status
M /opt/rtcds/userapps/release/als/common/guardian/ALS_COMM.py
M /opt/rtcds/userapps/release/als/common/guardian/ALS_DIFF.py
M /opt/rtcds/userapps/release/als/common/guardian/ALS_YARM.py
M /opt/rtcds/userapps/release/ioo/common/guardian/IMC_LOCK.py
M /opt/rtcds/userapps/release/isc/common/guardian/VIOLIN_DAMPER.py
M /opt/rtcds/userapps/release/isc/h1/guardian/BOUNCE_ROLL.py
M /opt/rtcds/userapps/release/isc/h1/guardian/ISC_DRMI.py
M /opt/rtcds/userapps/release/isc/h1/guardian/ISC_library.py
M /opt/rtcds/userapps/release/isc/h1/guardian/ISC_LOCK.py
M /opt/rtcds/userapps/release/isc/h1/guardian/lscparams.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_BS_ST1_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_BS_ST1.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_BS_ST2.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ETMX_ST1_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ETMX_ST1.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ETMX_ST2.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ETMY_ST1_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ETMY_ST1.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ETMY_ST2.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_HAM2_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_HAM3_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_HAM4_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_HAM5_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_HAM6_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ITMX_ST1_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ITMX_ST1.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ITMX_ST2.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ITMY_ST1_CONF.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ITMY_ST1.py
M /opt/rtcds/userapps/release/isi/h1/guardian/ISI_ITMY_ST2.py
M /opt/rtcds/userapps/release/isi/h1/guardian/SEI_CONFIG/const.py
M /opt/rtcds/userapps/release/isi/h1/guardian/SEI_CONFIG/CS_states.py
M /opt/rtcds/userapps/release/isi/h1/guardian/SEI_CONFIG/HAM_states.py
M /opt/rtcds/userapps/release/isi/h1/guardian/SEI_CONFIG/manager.py
M /opt/rtcds/userapps/release/omc/common/guardian/OMC_LOCK.py
M /opt/rtcds/userapps/release/sus/common/guardian/SUS_PI.py
M /opt/rtcds/userapps/release/sys/common/guardian/IFO.py
M /opt/rtcds/userapps/release/sys/h1/guardian/DIAG_MAIN.py
M /opt/rtcds/userapps/release/sys/h1/guardian/IFO_NODE_LIST.py
jameson.rollins@operator1:~ 0$
The lack of monitoring some nodes was at my request. These are nodes that currently are requested to be different from their true nominal state so that we can get diagnostic information (ALS nodes), or to avoid unneccessary potential locklosses (BS ISI node). For O2, these will be placed back in their low noise nominal modes. We removed them from the monitoring so that we would be able to set the Intent bit to Observe for the ER run. Once ER9 is complete, we will uncomment the nodes so that they are back to being monitored.
We should indeed be better about checking things into the svn.
I comitted these except for the isi ones
We have operated for over 10 hours in "nominal low noise, observation, intent bit set" condition. This was a temporary configuration to allow locking, after the laser diode box failure, for the purposes of testing ER9 pipelines. We are returning to 40W operation.
(Cheryl, Sheila, Kiwamu, Haocun, Ross, Keita)
ISC_LOCK.py and lscparams.py were reverted so the target power is 40W and SOFT offsets are set correctly.
ISS offset was remeasured at 40W and set to -0.932693 (instead of the original -0.9826934814453125).
IMC_LOCK.py was changed to set this offset, and also the final ISS 2nd loop gain was set back to 20dB for 40W (instead of 24dB for 25W).
No change was made to TCS as nobody touched the TCS CO2 guardian yesterday.
IFO got back to the correct 40W nominal low noise state after the above change.
After this, SUS_PI was also changed (yesterday it was accidentally changed such that QPD and OMC-based damping were enabled for the same mode at the same time).
H1 is in Observe at 25W.
State of H1: locked at 25W in ENGAGE_ISS_2ND_LOOP
Activities / Events:
Lockloss plots:
Upon my arrival (at 7:45 am PT) in the control room I found the IFO locked and in "Observation" mode with the intent bit set. The Lock Clock showed 8:50 hours and counting. The input power was 24.3 W. Camera images looked stable and as expected. (Because the video5 computer had frozen! argh). DARM ASD looked terrible, but not unreasonable for the operating condition of the IFO. ISC_Lock Guardian was in Nominal Low Noise.
This is a better start of the day then we have had for awhile. This should allow the ER9 testing of the hwinj, cal, and data pipelines and, with the intent bit set, allow the analysis streams to test their readiness.
15:13 UTC (8:13 PT) - just lost lock. Morning activities?
Patrick, Sheila, Carl and Ross damping PIs
After Keita's ISS fixes we had a few hours of bad weather and earthquakes that contributed to dificulty locking. Once things calmed down we locked and sat at several stages to see if the lock was stable, DC readout, 25 Watts, ISS 2nd loop engaged, and now we have made it to some kind of "nominal low noise" state. Patrick hit the intent bit.
There are several things wrong with this spectrum, some of which could be solved by going back to 40 Watts in the morning:
The last attached screenshot is just to show how many locking attempts everyone made in the last 14 hours. The range displayed by the DMT viewer in the control room seems to have a problem, it should be stable at some low range but on the DMT viewer it looks like we lost lock in the last half hour.
Sheila's comment
There were several things which had to be done today because of the change to run ER9 at 25 Watts.
We should remember to undo these things as soon as we go back to 40 Watts. According to Ross and Carl, there isn't a reason to think our PI situation has gotten worse than it was for the last 2 weeks where we had stable locks more than 4 hours long, but the PI damping would need to be babysat at 40 Watts. Maybe an engineering run is a good time for detector engineers and operators to learn how to do this.
There are several things that should be done if we want to continue the ER at 25 Watts with a reasonable sensitivity:
none of these are all that hard to do, but to me it seems like it would be more productive to try continuing the ER at 40 Watts, to see what we can learn about the IFO in a state that is more similar to the configuration we would like to run at for O2.
DARM-ISS 2nd loop coherence doesn't change by increasing the ISS 2nd loop gain by 6dB. Jitter or whatever, ISS is imprinting noise onto intensity.
Keita, this is something I think we saw in the past too:
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=20394
CO2 related comment:
The phase of the 18040Hz mode flipped during this lock stretch. The damping has been switched to the arm transmission QPDs gain 10,000 phase + 60 deg.
The 18056Hz mode also became unstable and was succesffully damped with settings in SUS_PI guardian.
The ETMX 15541.8Hz mode and ETMY 15542.6Hz modes that have been difficult to damp were damped well tonight with the arm tranmssion QPD signals. The rational was that the arm transmission QPD's differentiate the arm the mode is in to some extent reducing the beat effect. These settings are in the SUS_PI guardian. The revert comment out the ETM QPD gain settings and uncomment the OMC gain settings for these modes.
Carl - I'm wondering if the width and apparent double nature of each mode could be due to the same mode being excited in the coating-bearing mass and in the reaction mass - could the reaction mass body modes be getting rung up as well? Just a thought. Might be nonsense.
There has been something wrong with the ETMY oplev for quite a while now, it says that the optic is moving a lot more than the other test masses, swinging around by more than half a micro radian. This must be false, and needs some investigation.
Looking at the SUM out of the oplev at 2 different times, it seems like there may be an issue with the laser. The first attachment shows the ETMy SUM signal from a period in April while the second shows the same signal at the start of July (y axis is counts, and the scale is the same between the two pictures). As can be seen the SUM signal is noisier now that it was back in April. This could be what is causing the issue Sheila reports above. We can swap the laser at the next available opportunity to see if this fixes the problem.
The lockloss from the state REFL_POP_WFS which happened at 23:56:29 UTC July 07 2016 is very similar to what is described in alog 26840 and comments. I've attached the guardian logs for both ISC_DRMI and ISC_LOCK, and a plot of the lockloss.
The first thing that happens is that the ISC_LOCK gaurdian starts to transition our front end triggering for the LSC from POP18 to POPLF, by lowering the thresholds, sleeping 0.1 seconds, and then setting the trigger matrix elements to 0.
Yes, the expected behavior is that the target state of a jump transition of a managed node will be executed normally after the transition. The stall just prevents the system from following any standard transtions after the state returns true. Nothing prevents jump transitions, though.
If you don't want the system to do anything after a jump I would suggest inserting a do-nothing state in between.
With the measured sensing function (28123) in hand, we did an MCMC-based numerical fitting for the sensing function (see E. Hall, T1500553).
The fitting results are
[New sensing function form]
As reported by Evan (27675), it seems that the extra roll off in the DARM response at low frequencies are due to an SRC detuning (or something equivalent). While such detuning should be suppressed by some control loop ideally, we decided to include the detuning-induced functional form in addition to the ordinary single-pole response. We use the following approximated form for the DARM response
S(f) = H / (1 + i f / fc ) * exp( -2 * pi * f * tau) * f^2/(f^2 + fs^2)
where H, fc, tau and fs are the optical gain, DARM cavity pole, time delay and spring frequency. Some details of the derivation will be reported later. Apparently, we now have an additional quantity (i.e., fs) to fit.
Note that since H1 seems to be in (unintentionally) an anti-spring detuning, fs should be a real number whereas it should be an imaginary number for a pro-spring case. Obviously, Q is set to infinity for simplicity.
[MCMC-based numerical fitting]
Following the work by Evan (T1500553), we adapted his code to include the spring frequency as well. In short, it is a Bayesian analysis to obtain posteriors for the parameters that we want to estimate. We gave a simple set of priors as follows for this particular analysis.
The quad plot below shows the fitting result. The estimated parameters are obtained by taking the mean values of the resutling probability distribution. As usual, the two plots on the left hand side show a bode plot of the measured and fitted data. The two plots on the right hand side show the residual of the fit. As shown in the residuals, there are several points which are as big as 20% in magnitude and 6 deg in phase below 10 Hz. Otherwise, the residual data points seem to be within 10-ish % and 4 deg. The code is attached in pdf format.
EDIT: I am attaching the actual code as well.
A detailed derivation of the new functional form can be found at https://dcc.ligo.org/LIGO-T1600278
EvanG noticed that we have unintentionally included high frequency poles in the previous analysis (28157). So we made the same fitting for the data with all the high frequency poles removed.
Here are the fitting results for the latest data:
Since the bode plot looks very similar to the one posted above, I skip showing it here. Observe that the time delay is now smaller than it was because we now don't have the high frequency poles.
C. Cahillane, K. Izumi I have added in a new term to our sensing function fit, an optical spring Q factor to gain back phase information. From the functional form f^2/(f^2 + f_s^2) we only get a magnitude correction from detuning, but we also expect detuning to slightly affect phase in the sensing function response. This can be clearly seen in lower right subplot Figure 1 of this comment, where Kiwamu originally plotted the full RSE sensing function. Here we see that when we have 1 degree of detuning we also have +1.5 degree phase difference at 10 Hz when compared to the sensing function without detuning. In the phase residual plot in the original post you can see this phase loss in the actual ER9 sensing measurement. In order to try and gain back some of this phase information, we have added in an additional term to the detuning function:f^2 f^2 ----------- ===> ----------------------- f^2 + f_s^2 f^2 + f_s^2 - i*f*f_s/QThis adds another parameter to our fit, but lets us get back phase information lost to detuning. ********** Figure 2 shows Kiwamu's original fit parameters posted above in red alongside my new results in green. I argue that including the optical spring Q factor has improved the phase fit significantly. Quantitatively, here are the phase Χ-Square values:With Optical Spring Q Χ-Square = 85.104 Without Optical Spring Q Χ-Square = 819.28********** I have modified Kiwamu's SensingFunction.ipynb into a SensingFunctionSimulation.ipynb which fits to both phase and magnitude of the sensing function. SensingFunctionSimulation.ipynb is attached as a zipped file in this aLOG and is also in a Git repo called RadiationPressureDARM owned by Kiwamu. Fit parameters:Optical gain = 9.124805e+05 +/- 8.152381e+02 [cnts/m] Cavity pole = 3.234361e+02 +/- 5.545748e-01 [Hz] Time delay = 5.460838e+00 +/- 3.475198e-01 [usec] Spring frequency = 9.975837e+00 +/- 5.477828e-02 [Hz] Spring Inverse Q = 1.369124e-01 +/- 3.522990e-03(I choose to parametrize using inverse Q = Q^{-1} because Q^{-1} can be zero.)
