J. Kissel, E. Goetz, K. Izumi, S. Ballmer, P. Thomas, T. Shaffer, D. Tuyenbayev, C. Cahillane, P. King

We had a pretty rough day today as far as robust interferometry. Lots of problems resulted in no low-noise locks today (well, none that lasted more than a few minutes. Also "low noise" still only ~40 Mpc). Some problems were related to freshly commissioned things, some due to infrequently seen not-yet-fixed problems, some due to maintenance day recovery, and finally some due to honestly challenging things that are just not yet well-commissioned. The message: we were not able to perform any of the measurements needed for Calibration Day (i.e. measurements of the IFO's optical plant and of the ETMY Isolation Stage Actuation Strengths).

We'll discuss tomorrow if and when to reschedule. 

Details:
----------------

Why do you care? It's only ER9...
Most importantly, it doesn't matter if the run is 3 days or 3 months, we need to measure the interferometer so that we can *quantify* how well or how poor the calibration uncertainty is. Otherwise, when told "It's just ER9 ... you don't have to do better that ___% and ___ [deg]" (where you can pick your favorite large numbers for "___") we do not have any way to claim whether we are meeting that accuracy other than gut feel. Yes, we could go through the laundry list of things that have changed, try to model them, and make some sort of "how has the response function changed" model and infate the uncertainty accordingly, but that takes just as much, if not more, effort than measuring the IFO as is because those changes have not been quantitatively well-tracked over these past few months (which is totally understandable for a commissioning period).

There's been very little change between O1 and ER9, right? What's the big deal?
Here's a list of things that have changed between O1 and ER9 that all affect the most sensitive region of the interferometer at the several percent level:
- ETMY ESD Actuation Strength has changed due to charge accumulation / evolution. This hasn't been quantified in Longitudinal since the PostO1 calibration period, and we've since reduce the charge (as measured in angle) by manipulating the bias voltage sign. We need to confirm its current value. Especially since we've either not had calibration lines on for the past few months or we've not had a sensitive enough interferometer to resolve them.
- We want to identify the PUM / UIM Actuation Strength better. The Calibration Lines during O1 revealed a 2% systematic error on either the PUM or UIM. Because we only ran with a limited number of lines, we don't know to which stage we should associate this error. 
- Given all of the new alignment work over the past few months, and the recent woes on recycling gain, we want to get a new reference for the DARM coupled cavity pole frequency to make sure that 341 Hz is still a good reference.
- Now that we're running at 40 W, and it has recently been shown that signal recycling cavity detuning is affecting us as low as 10-20 W, we need to pay close attention to this, as the detuning might be high enough to start affecting the DARM actuation and sensing crossover at ~45 Hz.
- We've updated all of the coil driver and OMC sensing chain compensation to match the analog electronics "perfectly." All of these upgrade filters have a frequency dependent improvement on the few percent level. We need to make sure that all of those filters are indeed making the measurements and model comparison better.
- The new digital AA and AI (or IOP downsampling) filter has phase significantly different from the O1 filter, even down as low as a few hundred Hz. 

Other things that have changed that will affect high and low frequencies:
- A New OMC DCPD split whitening chassis has been installed to support PI tracking. Though the Gravitational Wave DCPD path is *nominally* the same, i.e. it has the same circuit design, its components are real and different, so they need to be remeasured. These will likely be differences in the high-frequency poles, so there affect will be likely be small and above ~1 kHz.
- Evan and Rana have changed the local QUAD damping loops. So we need to create a new SUS dynamical model and update everywhere its needed. This should primarily affect frequencies and dynamics below 10 Hz.

Why can't you get what you need next Tuesday, the day before the run?
*If* we get all the measurements we need, we need time to analyze and process those measurements, update all of the appropriate calibration filters (both in the front-end and in the GDS pipeline), make sure the calibrated pipeline agrees with our measurements and is producing sane results, run for a day or two's worth of lock stretches and confirm that it all works . Again, if we want to be quantitatively sure of our calibration, one really has to take all of the appropriate steps to well-calibrated data, regardless of the length of the run. Further, we've made drastic software changes to the GDS calibration pipeline, in order to make incremental progress toward O2. These need to be tested, and they'll likely only be ready by the end of the week.

Today's Lock Loss Break Down; Why we had so much trouble
There're 5 major lock stretches shown in the attached inspiral range time series. The lock losses and in between problems are described below:
Lock 1: Patrick and I'd gotten all the way to a 40 W (nice work commissioning team!), and was in the middle of switching coil drivers to their low-noise configuration (ISC_LOCK state "COIL DRIVERS") but got hit by some Izumi-Ballmer dP/dtheta instabilities. I found out later that this was because the ASC SOFT degrees of freedom had had their offsets commissioned last night, but they were left out of the Guardian because they weren't yet confident in their robustness. Apparently, with out them, the IFO is aligned to a place with reasonably large dP/dtheta, so I lost lock.

Between locks 1 and 2:
- We were having the same problems with the PSL FSS vs.ISS that Shiela found last night. We spent some time with Peter King trying to diagnose and debug.
- The OMC SUS had tripped (which is apparently a reasonably common occurance after lock-loss because of the suddenly bogus OMC input alignment signals)
- We'd lost lock in the middle SRM's bottom-stage matrix dance that facilitates coil driver state switching. However, the DOWN states did not restore these matrix values. Stefan spent some time fixing this. See LHO aLOG 28053.

Mini-lock 2: Intentionally broke the lock because PSL FSS / ISS / Noiseeater was oscillating, and because we were past DARM ASC signals being turned on we were blasting the PUM stage of test masses with junk, and we were worried about violin mode ring-ups and PUM stage coil driver RMS watchdog trips.

Big Lock 2: Lost lock after 40W power up because of ISS 2nd Loop Offset transient.

Between locks 2 and 3: 
- Continuing to have problems with PSL servos, Kiwmau and Peter found that settings had not been restored properly after maintenance day. These issues were completely resolved after we reverted some PMC locking servo gains using the correct SAFE.snap SDF file. See LHO aLOG 28058.

Lock 3: Unknown lock loss. These things happen! ¯\_(ツ)_/¯

Between locks 3 and 4
- OMC LSC Input Matrix ramping failed to scale the DARM gain correctly. Solution found in LHO aLOG 28074.
- Violin Modes on ITMY and ETMX rung up for unknown reasons, some PUM stage RMS watchdog trips, took a while to damp back down with some manual gain reduction and gradual re-increase.
- Needed to come up with no stages of OMC whitening to prevent saturations

Lock 4: Lost lock after 40W power up due to ISS 2nd Loop Offset transient.

Between locks 4 and 5
- TJ takes over, runs initial alignment.

Lock 5: Lost lock after 40W power up due to ISS 2nd Loop Offset transient.

As of this entry we're on Lock #6. And if we lose lock due to 2nd ISS offset problems again, Stefan's going to spend some time commissioning it. 

Note, that the 2nd loop closure problems (recall that the 3rd loop is ON during power up, and then once complete, we have to transition back to turning on the 2nd loop) have been newly difficult because there is just more intensity noise coming out of the IMC than there was during O1. Both the DC alignment into the mode cleaner has changed for the worse, increasing the natural jitter-to-intensity conversion of the IMC, and also the HPO just produces more intensity/jitter noise now that it's in use. Check out LHO aLOG 28076 for Stefan's report on today's attempts to improve it.