Vern asked for some of the pie charts I made for ER7 for the first weeks of ER8.  Here they are, based on data from 15:00 UTC August 17th to 15:00 UTC on the 28th, with about 40 minutes of missing data.

A few notes:

• We are suceeding in the CARM offset reduction more often. Durring ER7 only 1 in 5 DRMI locks reached the nominal operating state.  Now 1 in 3 DRMI locks reach the low noise state, and 75% of them make it through the CARM offset reduction.
• We have locklosses due to the beamsplitter coil driver switching (there is a transient due to analog switching), locking since this guardian state uses blocking calls these locklosses are listed in the pie chart as Low noise ESD ETMY, and probably account for most of the 11 locklosses from this state.  This is a lockloss we did not have durring ER7 since we did not switch the BS coil drivers.
• We also had a series of locklosses from the state DRMI on POP.  Durring this state we switch the state of the PRM coild driver.  While this coil driver was causing huge glitches we took one coil out, in this condition the switching became unreliable and caused locklosses.  This problem has been solved by replacing the chasis. 
• In the first few days of the run we had difficulty with SRC ASC, which caused many locklosses from the states DC readout, Increase power, and other states neaar this part of the sequence.  This is a problem that has been partially addressed, but will likely return.  We hope to spend some commisioning time to better understand the phasing and sensing matrix for these AS WFS.  It would be good to do this now, while the ASC is working.
• Although we have many locklosses from the earliest steps of the locking sequence, Locking arms green and Locking ALS, these locklosses are not much of a concern since they do not cost us much time.
• DRMI locking times are similar to ER7, the median time to lock is 4 and a half minutes (It was 5 minutes durring ER7).  We still have some extremely long locking attempts (up to 2 hours!)  I'm hoping most of these were commisioning. 

Our duty cycle is a bit worse than it was durring ER7, although there are several reasons we would expect it to have improved.  Some of the downtime can probably be explained by commsioning and calibration activities, althoug I haven't yet tried to take that into account.  It will be easier to get a clear picture of things once we start running undisturbed more. 

Windy for the first 2/3 of the shift, therefore not much success locking.  As the wind was dying down, the locking became quicker, but typically (but not limited to) losing it at SWITCH_TO_QPDS.   I have added instructions to the OPS wiki regarding turning on OMC DCPD whitening in NOMINAL_LOW_NOISE.

Activity log:

2:39 locked on DC readout for Sheila/Evan commissioning

3:24 lockloss at ENGAGE_ISS_2ND_LOOP

4:00 Dan shows up on site, starts looking into PRM and PR3 noise noted by Evan and Betsy

5:02 locked DC readout

5:11 locked NOMINAL_LOW_NOISE, Dan doing some OMC work

6:07 set to Undisturbed

Currently the OMC alignment dither lines are between 1675 and 1750Hz.  We'd like to move these lines above 2kHz to avoid polluting the analysis band of the CW searches.  Previously, when we moved the dither line frequencies from ~600Hz to ~1700Hz, we noticed there was a sign flip in the dither sensing matrix.  We expect the tip-tilt response to be flat at these high frequencies, so before moving the lines again I thought it was prudent to check what kind of resonance structure we were dealing with.

The attached plot shows transfer functions from OM1 pitch and yaw to OMC-DCPD_NORM while in full lock.  Overall the reponse of the tip-tilts is what we expect, but there are resonance features around 1900Hz and above 2.2kHz.  Also the phase changes smoothly by 180degrees between 500Hz and 1600Hz, so that explains the sign flip.  Note the dropout in coherence around the violin harmonics.  The range of the current dither frequencies is marked by the black dashed lines.

It looks like we can move the dither lines to just above 2kHz.  The phase will be a few tens of degrees different than the current frequencies, so the demod rotation phases will need to be changed.  The magnitude of the tip-tilt response is roughly the same, and anyways the current dither amplitudes are only a hundred counts, so we're well below saturation.  Should be a simple fix.

Betsy, Sheila, Travis, Evan

Something about the following coils appears to be unhealthy: PRM M1 RT&SD, PR3 M1 T1&T2. See attached OSEM sensor spectra.

According to Betsy, this high-frequency junk started around 2015-08-29 15:00:00 Z. On the control side, this junk dominates the rms of the PRM M1 LF drive (it is about 10000 ct).

Probably this is related to the problem that Kiwamu saw last night with PRM M1 DAMP L.

Nairwita Mazumder, Rich Abbott

A few days back Jim noticed  (alog ) that the "Bumbling line" which varies over a large frequency range is again back on ETMX seismic channels . This was first noticed on March and disappeared before ER7 and again was seen from 4th August. One can see the lines at all the horizontal and vertical sensors on ETMX. 

I have attached a pdf containing some follow up work done during Rich's recent visit to LHO. The first plot in the pdf is the spectrogram of ETMX GS13 on 26th August. It can be seen that there are multiple wandering lines having a fixed offset.  We were suspecting that some magnetometers at the End X might be the culprit (as we could not find any correlation between temperature fluctuation  with the line ). 

The second and third plots are the spectrum of H1:PEM-EX_MAG_EBAY_SEIRACK_Z_DQ and H1:ISI-ETMX_ST2_BLND_Z_GS13_CUR_IN1_DQ for 2nd August and 26th August respectively. The red one is for 2nd August when the bumbling line could not be found and the blue one is the recent data (26th August). It is clear that the peaks appearing on  ISI-ETMX_ST2_BLND_Z_GS13 after 3rd August are correlated with the peaks of the spectrum (which also appeared around the same time) of SEIRACK magnetometer .

The plots on the second page shows the coherence between GS13 and the magnetometers in the VEA and SEIRACK. It looks like  the magnetometer on the SEI rack has stronger coherence with GS13 sensors than the magnetometer located at VEA .  I have marked two points (blue and red cross)  in the coherence plots to highlight two of the many peaks.

(All time in UTC)

15:00 Take over from TJ. IFO not locked.

15:32 DRMI wouldn't lock after 10 mins. I was about to do the PRMI when I started seeing AS90 flashing. I gave it a few more minutes and it managed to lock.

          Lost lock at REDUCE_CARM_OFFSET soon after. Looks like ETMX was saturating. Wind speed ~20 mph.

15:45 Lockloss again at REDUCE_CARM_OFFSET. Turning ISI BSC WIndy blend filters on for BSC1,2,3.

18:23 Lockloss. Power outage. Guardian couldn't turn the ETMX ESD back on. We called Richard for help. See alog21030.

21:22 Switched the Windy blends off.

22:50 Begins the initial alignment after trouble relocking.

23:47 We're running into an issue with MICH_DARK_LOCKED. The beam would diverge away. I've tried setting the ALIGN_IFO to DOWN and waited a little bit before trying again. Didn't work. Sheila is working on it.

There were no indication of anything was going bad right before the lockloss. No glitches in the spectrum, no earthquake, nothing. The saturation alarm yelled ITMX and DRMI was unlocked. The SUS OMC SW watch dog tripped but seems to have uptripped itself. After that the Guardian wasn't able to turn ETMX ESD driver on and the reset switch kept flickling. Sheila is on her way to the site so I'm waiting for her. 

I noticed the light in the control room flickered at the exact time of the lockloss. I suspect a power glitch may have caused this lockloss.

I set the Operating Mode to UNKNOWN.

After several locklosses at REDUCE_CARM_OFFSET, I took it very slowly. I stayed at DRMI_LOCKED until the saturation alarm complained less about ETMX. Even though the wind speed is only 15-20 mph but PRM seems to wiggle a lot so I turned on the 90 mHz windy blend filters for all the corner station BSCs. The intent bit has been set to observing.

ER8 days 12 and 13. No restarts reported.

Times in UTC

925 - Took the IFO to DOWN for Kiwamu to work on more calibration measurements

1016 - H1SUSETMX Timing error cleared

1208 - NOMINAL_LOW_NOISE, but BS bounce was rung up and Kiwamu tried to damp it to no avail

1219 - Itbit set to Undisturbed

1416 - Lockloss (Wind is picking up, 20mph, but that shouldn't be a problem)

1431 - Lockloss @ DC_READOUT (everything looked good, not sure what happened unless it really is the wind)

1438 - Gardeners here for tumbleweed cleanup

1444 - Lockloss @ DARM_WFS

1454 - Lockloss @ CARM_ON_TR

1500 - Handing if off to Nutsinee

Over the last ~5 days, nearly 100 SDF channels have gone into alarm.  We should probably link these to the INTENT bit, since many of these are actually real configuration control changes of the IFO, yet currently do not dictate any GO/NO GO of observation mode.  When connected to the intent bit, it will force commissioners and operators to address these changes in realtime, rather than let them stack up for me to reinvent every 5 days.

Since super-winds (and calibrations) have the IFO down, I've taken the opportunity to work on these - so far I've cleaned up:

- ETMX and ETMY L1 and L2 damping loop outputs enabled last Tuesday around the maintenance period - maybe the BURT pushed these buttons.  I  verified that the SAFE file sets the gain of the loop to be zero and the IN and OUTPUTS are ON.  Guardian then controls the gain.

- Accepted a few pages of offset changes, and the extra AS_B_RF45_AWHITEN_SET2 due to Evan/Stefan running the dark offsets script/commissioning last Thur night (alog 20976).  I'll wait for them to comment that the 1st attached SDF diffs are 1) their intended gain changes as it's vague in their alog, and 2) if they are ready to keep/accept them.

- Since SDF couldn't write the very small ~ 10e-19 CALCS epics values to the SAFE.snap file (commissioned AUG 11 alog 20452), I wrote them in by hand and then hit LOAD TABLE on the CALCS SDF.  All of the values cleared, since now they match the very small values loaded in their respective medms.  I've committed the CALCS safe.snap to svn.

- ALS X and Y - I need someone to point me to an alog regarding why the ALS_WFS_DOF_3 gains changed on both X and Y in the middle of the night on Thur night (2am Fri - see second SDF diff attachment.)

Jeff, Evan

There is now a new DMT viewer template for viewing the BLRMS of the corner and end-station STSs. It is userapps/isc/h1/scripts/Seismic_FOM_STS.xml.

This is meant to replace the Guralp DMT viewer template.

Right now it only displays the lowest two BLRMS (0.03 to 0.1 Hz and 0.1 to 0.3 Hz).

[As an aside, the frequency bands implied by the channel names in DMT viewer don't seem to match up with the front-end channels. For example, the front-end EX-X channels are named as follows:

H1:ISI-GND_STS_ETMX_X_BLRMS_100M_300M
H1:ISI-GND_STS_ETMX_X_BLRMS_10_30
H1:ISI-GND_STS_ETMX_X_BLRMS_1_3
H1:ISI-GND_STS_ETMX_X_BLRMS_300M_1
H1:ISI-GND_STS_ETMX_X_BLRMS_30_100
H1:ISI-GND_STS_ETMX_X_BLRMS_30M_100M
H1:ISI-GND_STS_ETMX_X_BLRMS_3_10

while the DMT channels are named as follows:

H1:ISI-GND_STS_ETMX_X_DQ_0p03-0p1Hz_48h
H1:ISI-GND_STS_ETMX_X_DQ_0p1-0p2Hz_48h
H1:ISI-GND_STS_ETMX_X_DQ_0p2-0p35Hz_48h
H1:ISI-GND_STS_ETMX_X_DQ_0p35-1Hz_48h
H1:ISI-GND_STS_ETMX_X_DQ_1-3Hz_48h

Are these BLRMS distinct from what's being computed in the frontend?]