In 22947, I said "...the coil driver sent a bad status..."  This is too broad and points fingers at the Coil Driver.  While the zero bit may have originated at the coil driver, as we believe that occurrance will latch the state it is very possible the bit dropping to zero may have originated down stream.  There is the binary interface chassis bundling all the 9pin channels into db37s and then the Bin I/O card in the I/O chassis.  Trouble shooting these doesn't appear easy, especially in O1 time.

It makes sense that we tailor the model side to not trip the ISI when these less than a second changes occur.  Otherwise we should just wait to see if it is a problem.  Looking back a month, the first time it occurred was during the power outage but that was legit.  Otherwise, it occurred Tuesday but Richard says that might have been him exploring, experimenting or some other exing.  And then yesterday.  So other than the model change which we'll discuss that what should be a simple code adjustment, we'll leave it til it breaks for now.

TITLE: "10/30 [DAY Shift]: 15:00-23:00UTC (08:00-16:00 PDT), all times posted in UTC"

STATE Of H1: Lock Acquisition

OUTGOING OPERATOR: TJ

QUICK SUMMARY: Wind ~30 mph. Switched to 90mHz blend. TJ reported he had trouble with DRMI when so he had to switched it back to 45 mHz. Just had a lockloss at DRMI so I'm gonna try switching the blend back to 45 before doing DRMI. Things don't look very hopeful.

• Title: 10/30 OWL Shift: 7:00-15:00UTC (00:00-8:00PDT), all times posted in UTC
  

• State of H1: Unlocked since 12:42 UTC (5:42 PST)
  

• Shift Summary: The wind started picking up halfway through my shift, I'm pretty sure a large gust or two took us out of lock. I have been trying different things to get it back up and running with no success. Finished Initial Alignment and leaving it for Nutsinee.

• Incoming Operator: Nutsinee

• Activity Log:

• 12:42 Lockloss
• 13:02 switch blends X and Y dof from the 45mHz to the Quite_90's. I had a handful of locklosses before it would aquire DRMI, but it would aquire ALS nicely. With this and the still high useism, I thought I would give the 90's a try.
• 13:08 Switch back to 45's. The switch to the 90's immediately looked bad, I couldn't advance to DRMI.
• 14:00 Loss at SWITCH _TO_QPDS
• 14:11 Loss at SWITCH _TO_QPDS
• 14:20 Started Initial Alignment. In the past, if it fails at SWITCH_TO_QPDS repetitively, this could be the sign of bad alignment. Thought it was worth a try.

Here is some info that Sheila wanted.

13:50 started to try PRMI

13:52 PRMI locked

13:56 Transition start and fail

(A few trends attached one just zoomed in more than the other)

Lockloss at 12:42 UTC, seemed to be a few large gusts right before the we lost it. EX trace hit 50mph.

Winds have picked up to into the high 20's with some gusts hitting 40mph. Looks like the IFO is really struggling to hold on, but has so far. Let's hope the wind dies down a bit.

Jordan Palamos, Robert Schofield

Report attached: copying background and summary below

--------------------------------------------------------------------------------------------------------

Background

In early September as part of the PEM injection set we injected RF at specific frequencies where coupling to the interferometer was thought possible. The injections were done using dipole antennas and RF amplifier located by the water tank on site. For those injections the 9 MHz and 45 MHz injections were picked up by the PEM radio channels in the corner station electronics bay ~1000x louder than the background. Of the 10 frequencies chosen only the 45MHz injection showed up in the gravitational wave channel, barely visible over the noise. Those results have shown that RF coupling to the interferometer is small at these frequencies where coupling might have been expected and additionally for the coupling at 45Mhz any RF signal would have to show up very strongly in our auxiliary channels before it shows up in the gravitational wave channel. In order to check for strong and unexpected couplings outside of these frequencies we injected a swept sine spanning a wide range of frequencies.

Summary

We injected an rf sweep from 9 kHz to 100 MHz to check for coupling from environmental rf to the interferometer. We found no evidence for strong coupling at any frequency within the band. An initial upper limit for coupling is that a signal will need to show up with an SNR of 100 on the RF scanner to have an SNR of 1 in DARM, but further investigations may be able to lower that.

• Title: 10/30 OWL Shift: 7:00-15:00UTC (00:00-8:00PDT), all times posted in UTC
  

• State of H1: Observation Mode at 75Mpc for the last 8hrs
  

• Outgoing Operator: Patrick

• Quick Summary: Patrick seemed to have a quiet shift, the wind is low, useism is trending down, CW inj are ON.

TITLE: 10/29 [EVE Shift]: 23:00-07:00 UTC (16:00-00:00 PDT), all times posted in UTC
STATE Of H1: Observing @ ~77 MPc.
SHIFT SUMMARY: Remained in observing remainder of shift. Winds and seismic are mostly unchanged.
INCOMING OPERATOR: TJ

Quiet shift, not much to report.

Five SUS ETMY saturations since start of observing.

Intention Bit: Undisturbed (Oct 29 23:20:02 UTC)
Current time: Oct 30 00:00:00 UTC
SUS E_T_M_Y saturating (Oct 30 00:37:48 UTC)
SUS E_T_M_Y saturating (Oct 30 00:39:18 UTC)
Current time: Oct 30 01:00:00 UTC
Current time: Oct 30 02:00:00 UTC
Current time: Oct 30 03:00:00 UTC
SUS E_T_M_Y saturating (Oct 30 03:27:37 UTC)
SUS E_T_M_Y saturating (Oct 30 03:36:40 UTC)
Current time: Oct 30 04:00:00 UTC
SUS E_T_M_Y saturating (Oct 30 04:25:33 UTC)

Have remained in observing. No changes to report.

O1 Day 41

model restarts logged for Wed 28/Oct/2015
2015_10_28 13:36 h1sysecaty1plc3sdf

restart of Beckhoff SDF node as part of testing (done when H1 out of observation mode)

TITLE: 10/29 [EVE Shift]: 23:00-07:00 UTC (16:00-00:00 PDT), all times posted in UTC
STATE Of H1: Observing @ ~79 MPc
OUTGOING OPERATOR: Nutsinee
QUICK SUMMARY:
Lights appear off in the LVEA, PSL enclosure, end X, end Y and mid X. I can not tell from the camera if they are off at mid Y.
Winds are less than 15 mph.
ISI blends are at 45 mHz.
Earthquake seismic band is between 0.01 and 0.1 um/s. Microseism is between 0.1 and 0.3 um/s.

Just reached NLN and went to Observing.
The ISS second loop engaged on its own just after I got setup to engage it manually.
Attached are the SDF differences I reverted (ASC DHARD loop gains and EMTY violin damping filter gains).

TITLE: "10/29 [DAY Shift]: 15:00-23:00UTC (08:00-16:00 PDT), all times posted in UTC"

STATE Of H1: Lock Acquisition

SUPPORT: Evan

SHIFT SUMMARY: The ifo was locked for the most part. Few ETMY saturation alerts that seems to associate with DHARD Y amplitude increase during high wind. Kyle done with HAM3 ion pump work. Now relocking. Wind ~10mph. useism ~0.2um/s. Jeff also have asked operator to change the blend from 45 mHz to 90 mHz next time the ifo lose lock.

INCOMING OPERATOR: Patrick

ACTIVITY LOG:

15:24 Joe driving down the x arm to work on the beam tube

17:08 Joe was back, and going out again.

18:46 Joe back

20:08 Kyle to HAM3

          Joe to X arm again

20:18 Kyle out. He said he did climb on HAM3. The ifo didn't lose lock.

          Evan driving ETMY to find 3rd harmonics violin modes

21:00 Evan done. Back to Observing.

22:14 Joe back for the day, but Chris is still out there.

          Kyle to LVEA. Switched to Commissioning.

22:38 Kyle out.

           Evan wanted to do more measurement.

22:39 Lockloss.

23:00 The ifo is half way to NLN. Handing off to Patrick.

I had energized the ion pump before lunch (time?) such that the ion pump had been pumping in parallel with the pump cart for a few hours -> New ion pump is OK.  

(Evan Hall, Borja Sorazu)

This is  a preliminary entry on the identification of 3rd violin mode harmonics with test masses (figures and further analysis will be entered later on). This information will be used to estimate mechanical losses.

Dates and times of injections: 2015-10-28 from 20:42:30 to 20:45:00 --> Injection on ITMX

                                                     2015-10-29 from 20:20:00 to 20:32:00 --> Injection on ITMY

                                                     2015-10-29 from 20:37:30 to 20:45:00 --> Injection on ETMX

                                                     2015-10-29 from 20:53:00 to 21:00:00 --> Injection on ETMY

Injection description: Fed back, in phase, DARM to penultimate mass coil drivers in pitch after bandpassing it between 1450 to 1500Hz.

Frequency modes identification (preliminary):

ITMX:

     1463.09Hz, increased by factor 4

     1467.96Hz, increased by factor 4

     1462.31Hz, decreased by factor 1.3

ITMY:

     1461.41Hz, increased by factor 4.6

     1462.03Hz, decreased by factor 4

    1463.09Hz, increased slightly

    1470.38Hz, decreased slightly

    1470. 83Hz, increased by factor 2

    1472.5Hz, decreased by factor 9

ETMX:

     1463.09Hz, decreased slightly by factor 1.077

     1471.93Hz, increased by factor 6.6

    1474.08Hz, increased by factor 4

    1475.25Hz, increased by factor 9

    1478. 17Hz, increased by factor 4

ETMY:

     1482.6Hz, increased by factor 1.75

     1484.67Hz, increased by factor of PI

Future measurement: We will do a similar injection with DARM fedback 90 degrees out of phase.

This trip appears to have occured twice with the lock dropping on the first one.  The second was about 20 minutes later.  The watchdog registers tconvert 1130165885 Oct 29 2015 14:57:48 UTC.  The guardian first detected a problem at:

2015-10-29T14:38:52.53819 ISI_ITMY_ST2 [HIGH_ISOLATED.enter]
2015-10-29T14:57:49.03752 ISI_ITMY_ST2 [HIGH_ISOLATED.run] USERMSG: SETPOINT CHANGES. see SPM DIFFS for differences
 

After the WD detected a problem so it doesn't look like the guardian should be suspected.

Looking through the Coil Driver BIO channels, the Stage2 V3 CD status is the only element that registers a change.   The first attached 1 hour plot shows the BIO signals, the lock loss, and the stage3 coil driver outputs.  I don't see the V3 response any different than the V1 or V2 so nothing clear there.  The shift to zero is expected as the DC position is reset to zero offset at each Isolation.  The second 10 day plot shows this drive reset to zero each time and then slowly drift away from zero during the Isolated period.

Next I'll look through the model for meaning and why.

Kiwamu reported in 22781 about a couple lock losses due to the drive to HEPI hitting the 250um limit.  HEPI can handle more than this but should not need to if the PSL (likely Ref Cav) temperature does not trend off somewhere.

Attached are several PSL Temp channels for 14 days with the limit hit time noted on the run away Tidal Channel.  Certainly many of the PSL Temp channels show a trend that correlates pretty well with the run away.  In iLIGO we had tight temp control of the Ref Cav, do we have a temp of the Ref Cav now or is there another signal that would best represent that?