Keita pointed out that the ISS 3rd loop 560 was overloaded, and was in low noise when it should have been in high dynamic range.  I went to check it out and indeed, it saturates even without any input or with a terminator. 

Fil didn't have a spare working 560, but loaned me an SR650.  We are using an AC coupled low pass at 30 Hz, gain of 0 dB, positive polarity, to imitate the settings Keita used in 27895

We don't know how long the 560 has been overloaded, but this probably is the reason why we have had so much difficulty with the CSOFT instability in the last week.

Experiment on CP4 today:

Started at 90% full
Filled to well over 100% at 45% open on LLCV
At 12:10pm local, CP reached 100% full (took ~3 hr 40 min)
I sat at MY for about two hours with nothing happening (at this flow rate it takes a lot longer than 35 min. to overfill till LN2 comes out)
Finally I began to ramp the LLCV 5%, and then 10%, every 20 min. or so
At 72% open the signal from the flow meter was very noisy
After a few min. at 72% the exhaust pressure started to rapidly rise so I set to PI-mode with min. allowance set to 39% (love the new code!)
The pressure was still high and unstable and because I didn't want LN2 to spew onto the flow meter I lowered the PI-code min. to 37%
The exhaust pressure has slowly ramped back down to nominal
Leaving in PI-mode (at 37% min) over the weekend and will periodically monitor pump level and pressure

It appears that something goes wrong when zooming in that creates random lines across the plots.

3pm local

18 sec to overfill CP3 with 1/2 turn open on LLCV bypass valve. Lowered CP3 LLCV from 20% to 19%.

Attached are trends of the last month+ of relative humidity and temperature data for the 2 desiccant cabinets in the VPW.  One is used for 3IFO storage.  The last trends are posted at alog:  28526 .  The last month of data show normal trends with some bumps when the cabinet was opened for some time and the RH raised breifly.  Temp of the cabinets is the usual ~daily fluctuation of ~10deg F.

During this morning's PSL ISS code change, I tested if restarting h1pslpmc model with the FLOW IIR filter replaced with an EPICS part still closes the shutter, and indeed it did.

Looking at full data DAQ trends at the time of model startup shows what is going on. The FLOW ADC data is downsampled from the 64kHz IOP model to the 32kHz h1psliss model. The downsampling filter starts the data at zero and takes several cycles to settle to the digitized value. As an example the plot below shows an LSC channel at the time we restarted h1lsc yesterday. In the PSL PMC case, since any instantaneous FLOW value outside of the acceptible range causes a shutter close trigger, this will always close the shutter on model startup.

One solution we can think of is to add code in the FLOW_ERR C-file which triggers FLOW_OK to become False only if the flow values are out of range for several consecutive cycles. For a decision on whether the issue is of enough concern to warrent this code change we'll defer to the PSL team.

This morning the RMS watchdogs for all of the test masses have been tripping, usually during locklosses at higher ISC_LOCK states. Resetting these requires opening the tripped Quad overview, and setting H1:SUS-*TM*_BIO_L2_UL_RMSRESET to zero, then  back to 1. Since I've had all 4 trip at once, and Verbal_Alarms yells each one at you multiple times, I made an alias in my .bashrc profile to do this. Resetting all of the RMS WDs doesn't seem to effect anything, or disrupt lock, so this alias can be used any time one of these WDs trips. Entering rms_wd in a terminal now sets the WD bit to zero, sleeps for a second, then sets it to 1.

Operators can easily add this to their .bashrc file from a terminal by copy/pasting :

echo 'alias rms_wd="caput H1:SUS-ITMX_BIO_L2_UL_RMSRESET 0 && caput H1:SUS-ITMY_BIO_L2_UL_RMSRESET 0 && caput H1:SUS-ETMY_BIO_L2_UL_RMSRESET 0 && caput H1:SUS-ETMX_BIO_L2_UL_RMSRESET 0 && sleep 1 && caput H1:SUS-ITMX_BIO_L2_UL_RMSRESET 1 && caput H1:SUS-ITMY_BIO_L2_UL_RMSRESET 1 && caput H1:SUS-ETMY_BIO_L2_UL_RMSRESET 1 && caput H1:SUS-ETMX_BIO_L2_UL_RMSRESET 1"' >> .bashrc

You then need to open a new terminal to get the new alias.

Terra, Sheila

After recovery tonight we did a few things:

• Reduced the gain of the PRC2 loops, by a factor of 5 each. (PRC2 P gain from -5000 to -1000; PRC2 Y gain from -10000 to -2000) because we saw an instability at around 1.5 Hz that could be reduced with the lower gain.  These loops have had higher gain than normal the last few weeks (the story is here)
• Reverted the TMS matrix to the one that is insensitive to transmon motion.  The thinking was that TMS drift is the reason we are not fining repeatable offsets, and that by reverting at least things would be more repeatable from day to day.  The old offsets worked OK, giving us a recycling gain around 34 at 2 Watts and 24 at 50 Watts, although at 50 Watts we saw the CSOFT instability slowly growing.  We also see that at 40 Watts we can recover a recycling gain of 32 by using our old pop A offsets (0.4 pit, 0.2 yaw)
• We modified the a2l script so that we can run it while the SRM dither loop is on, and ran it.  (for now I saved the modified scripts with _LHO at the end of the name.)
• We have continued to have trouble with the CSOFT instability and tried to improve things by turning up the gain of the ISS 3rd loop by a factor of 10.  However, the third loop seems to have no effect on the instability tonight, although in the past it has clearly damped it. We weren't able to measure the open loop gain because each time we tried to measure we broke the lock, even with a fairly small excitation.
• We powered up with the dither lines on, and saw no evidence for spot motion on the test masses (consistent with what was seen in the past). 
• In our last two locks we powered up with dither lines on the PR optics and beamsplitter, and can see clear changes in the spot positions as we increase the power (power ups starting at 8:56:52 and 9:50:30 UTC), and we can see that that when we adjust offsets to improve the power recycling gain we are partially resotring the spots to their orgiginal positions before the power up.  We will work on calibrating the data, the attached screenshot shows uncalibrated spot positions for PRM in maroon, PR2 in orange, PR3 in red, and BS in taupe.

We have noticed that since we are going to higher powers, things are misalined after a lockloss and become realigned as they cool off.  We might want to account for this someday to save time.

WP 6122 h1psliss model change (and subsequent DAQ restart)

Keita, Peter, Patrick, Dave:

We installed Keita's PSL ISS model change on h1psl0. I also restarted the PMC model to see if it closed the shutter (it did) so now I have DAQ data on this problem.

We restarted the DAQ for the new ISS INI file. This also applied the FMCS changes (addition of Relative Humidity channels).

This will generate an alarm on pump level. 

Filling at 45% open on LLCV, past 100% full until I see a change in flow meter curve or until ~20 minutes go by (takes around 30 min. for LN2 to spew out the exhaust).

J. Kissel, J. Warner

As Jim brought the instrument up this morning, we were alarmed to see a forest of lines around 20 to 25 Hz. As they did not disappear or change height as we went through the lock acquisition sequence, as ASC loops turned on, noticed them in MICH and PRCL, found a whole bunch of ASC-ADS SDF differences, and I heard Shiela suggest that she might try an LLO scheme for dithering the alignment, I began to suspect that these lines were intentional.

Indeed, after poking around, I found the dither alignment overview screen and found several oscillators pushing out excitations at 
19.1, 19.7. 20.1. and 20.8 Hz in Pitch and 21.3. 21.9, 22.3, and 23.0 Hz in yaw, being sent to PR2, PR3, and the BS.

I confirmed that these lines actually make it to the SUS by gathering an ASD of the input control request at the bottom of PR2, PR3 and the BS indeed I see the same features in PR3 and BS (why not PR2?).

*phew!*

We'll leave these on, in case the plan was to just gather data with these lines present.

See attached ASD of input request to the SUS, a screen cap of the Dither Overview screen, and the corresponding DARM ASD.

We had a couple locklosses when the ISS saturated, both with the third loop on and off.  We just increased the ref signal to -1.52 V to move the diffracted power to around 4%.  Was some setting incorect after a reboot this morning?  SDF indicated that everythind is OK. 

I swapped the oplev laser for the ETMy optical lever in order to address the problem noted by Sheila here.  The laser will need 4-6 hours to come to thermal equillibrium.

The SN of the new laser is 130-1.  This completes WP #6013.

the Verbal Alarms code was logging to the ops home directory. Prior to the move of this home directory (WP5658) I have modified the code to log to a new directory:

/ligo/logs/VerbalAlarms

We restarted the program at 14:04 and verified the log files are logging correctly.