Previously posted as comment to 17504.

Note that ISI Stage2 was a State3 trip meaning that the Damping Loops were still functional.  It would be nice to know when/why the stage tripped.  But I'd guess that the Guardian was set to fully isolated before the ~0700utc short lock and was untripped during the short lock, likely triggering during acquisition and then triggering again at the trip.  That is what I first thought.

See attached--It looks like the stage2 when isolated survived the IFO lock loss but not the MICH reacquisition.  This means, we can use the Guardian to transition the ISI between Isolated Damped and Fully Isolated with tripping nor requiring operator intervention.

This is a 50 minute stretch so the IFO lock Loss is only a couple minutes before the St2 WD trigger but there may be enough time to deisolate stage2.  Looking at a zoomed in full data, looks like a full 70+ seconds after lock loss based on this CAL_DELTAL channel.  The second lock loss had even more time so I don't think it is tied to the lock loss but more to the reacquisition.

Of course, we also need to show that turning Stage2 Isolation loops on is a positive thing.

J. Kissel, J. Warner, J. Romie, H. Radkins, N. Kijbunchoo, K. Izumi, G. Moreno

SUMMARY: We just finished ~7 HOUR lock stretch at our best sensitivity ever, between 32 and 34 [Mpc]!

We've all been pleasantly surprised this morning to see that the lock stretch that Sheila Evan, Dan and Lisa started last night lasted the entire night. Unfortunately, as of ~10 minutes ago (~8:40a PDT, ~15:40 UTC), there are GIANT glitches and non-stationarity that keep popping up, spoiling the sensitivity. There's no one in the LVEA, so we're not at all sure what's caused the sudden change in behaviour. Wind seems fine at ~5 [mph], ground motion is still pretty low, the 1-3 [Hz] hasn't even come up yet.

Anxious to explore things, Jim installed some low-pass filters in the HAM5 and HAM6 ISIs at around 8:45 to try to reduce scattering / acoustic coupling to the ISI, and Kiwamu began exploring MICH coupling to DARM. 

But, as I write this log, we lost lock. However, for DetChar purposes, one can assume the detector was undisturbed from March 27 9:00 UTC to March 27 ~15:00 UTC.

model restarts logged for Thu 26/Mar/2015
2015_03_26 00:32 h1fw1
2015_03_26 08:55 h1calcs
2015_03_26 08:57 h1omc
2015_03_26 09:03 h1broadcast0
2015_03_26 09:03 h1dc0
2015_03_26 09:03 h1fw0
2015_03_26 09:03 h1fw1
2015_03_26 09:03 h1nds0
2015_03_26 09:03 h1nds1
2015_03_26 14:44 h1fw0

two unexpected restarts. CAL and OMC model work, with associated DAQ restart.

Scott L. Ed P. Chris S.

My apologizes for this late report from yesterday. I had to leave early to get to town and pick up oil filters for the new generator, already time for P Ms.
The cleaning crew was able to clean 76 meters of tube yesterday towards X-1-7 double doors. Good day.

The whistles in DARM are still quite prominent in this lock. As briefly summarized in this alog, they happen whenever the PSL VCO frequency crosses 79.2 MHz. This points to the likely cause. This is a reliable enough indicator that we can get some statistics. In the first hour of the current lock (Mar 27 9 UTC to 10 UTC), the whistles happen at a rate of 4 per minute.

Below are four spectrograms. The first two show some whistles identified this way. A new feature is that there seems to be a second oscillator very close by, or maybe some harmonic of the beat note. The next two spectrograms show a minute with several whistles, and the next minute where none occurred. Even when none go through zero frequency, you can still see the beat note hovering up near Nyquist.

Evan, Sheila, Dan, Lisa

We made it to 34 Mpc by managing to do at the same time all of the things we have tried before (low noise ESD on ETMY, low noise coil drivers for all the optics, POPAIR for vertex length DOF, more aggressive SRCL cut-off), plus some more ASC cut-offs. Implementing all of these things at the same time happened to be more challenging that expected because of a few problems that slowed us down today:

• trip of EY ESD driver at every unlock happening from the final low noise state (requiring a trip to the end station for resetting the driver);
• trip EX PUM whatchdog (not immediately recognized from the control room, also requiring a trip to the end station);
• some new which required some effort to get damped;

Some more details

• The improvement below 20 Hz in the sensitivity is entirely due to the cut-offs we added to the CHARD loops;
• Even after transitioning the BS M2 coil driver to low noise, and moving to POPAIR, we still have some coherence with MICH below 100 Hz. We tried to engage the MICH subtraction path, but we unlocked in doing so. Since unlocking from low noise costs us a trip to the end station, we didn't try harder..
• We closed the OMC REFL beam diverted;

Final IFO & Guardian state

• We promoted the SRCL 20 Hz UGF + cut-off in the Guardian, as it seems to work well;
• The transition to ETMY ESD low noise + DARM boost is still done by hand (it is written in the Guardian; we are just not brave to try it)
• The transition to BS low noise is also done by hand, and since it has failed at least once in the past, we decided not to engage it in this last lock
• Stage 2 ISI on BS is tripped..
• Sheila prepared some more cut-offs for the ASC DHARD loops, but we haven't tried those yet.

We leave the interferometer locked at 34 Mpc, starting at March 27 9:00 UTC

Evan & Sheila, Dan & Lisa

Tonight some new failure modes of the violin damping manifested themselves, and we found new ways to damp them.

On ITMY, the following modes and settings were identified:

Note that in the earlier catalog the 504.875 line was mis-identified as an ETMX mode, and the 503.119 line was not paired with an optic.

Earlier in the evening we tripped the RMS watchdog on ETMX, and had to drive out the end station to power cycle the coil driver.  

This wasn't obvious from the control room.  Three things that would help are to 

1) add this to the OPS overview screen

2) add it to the SYS_DIAG gaurdian

3) add it to the SUS guardians

Here is a plot of coherences with some PEM accelerometers.  We have been thinking that the noise around 200-250 Hz was due to the PSL periscope PZT, based on coherences like the one in the lower left plot.  However, there is suscpicously similar coherence between the ISCT6 accelerometer and DARM.  Indeed, the coherence between these two accelerometers is high in this frequency range, suggesting that some cross talk could be the dominant signal in these accelerometers.

The right two panels show that the accelerometers which have coherence with DARM around 13 Hz and 16 Hz are also coherent with each other, but not nearly as much.  

Rich A., Lisa, Sheila, Jeff, Evan

The EY ESD driver has tripped twice today, apparently in the same failure mode as described previously. Each time we have had to drive down to the end station to reactivate it.

The supplies are set at ±430 V, each with a 75 mA current limit. When the driver is active, it draws 23 mA dc from the positive supply and 20 mA dc from the negative supply. When tripped, it draws 3 mA dc from each supply.

Sheila, Hugh

IFO lost lock ~1426utc, Richard suspects it was his testing of the ESD.  Almost 45 minutes later, the ISCINF_LONG stepped to its rail, LIMIT set to 250000nm.  And I mean stepped, ~1 second or less.  This of course put a bit much demand on the HEPI loop.  It spiked its output and pretty much instantly tripped the watchdog on the Actuator followed by the ISI shortly thereafter.  The HEPI and ISI reisolated normally.

The first attached plot shows the lock loss.  This 30 minute trend has the DARM OUT with the ISCMON_X_INMON quickly going to zero and the HEPI Outputs moving in response.  No problems here, HEPI does fine.  You also see the arm or something grab again as you can see the Tidal drive to the HEPI start again but this only lasted for a couple minutes.

The second plot attached zooms out to 2 hours, with the IFO lock loss seen on the ISCMON on the left.  ~45 minutes later the DARM steps, this time stepping the ISCMON and tripping HEPI and about 8 seconds later, the ISI (not shown.)

In the third attachment, Sheila dug around in LSC/ALS space and put together some trends showing the spike into HEPI originating when the ALS-X_LOCK_STATE steps from level 2 to 6 (PLL Lock to Red Lock.)  This is unrealistic as the Lock state dropped from Red Lock just ~90 seconds earlier.  Notice the TIDAL_CTRL_INMON steadily growing (DARM not Cleared.)

Sheila is looking deeper into the Guardian feeling it may have failed to do something it possibly should have to prevent the DARM OUT from going to the rail and the unreasonable rapid transitions of the ALS LOCK STATE.