At 21:31:21utc, H1 was dropped out of OBSERVING.  I did a quick scan and did not see anything obvious.  There have been several Guardian Nodes which are NOT green:  ISI (for a TRamp, Violins (from violin activities), Inj Trans (it's waiting to go back to Success state), etc, but these remained as they have been.  A few notes:

• The Intent Bit was able to go back to OBSERVING shortly after dropping out,
• Was not able to catch this drop in time to bring up the list of "Not OK" Guardian Nodes--so no light shd there.
• Trended Diff Count for dozens of SDF nodes, but saw nothing change from 0 for all of the ones I trended.
  • Previous examples I remember of quick changes which dropped us out of Observing were related to TCS & Slow Controls/Beckhoff.  Then there is also Squeezer, but that is usually for when the Squeezer drops out of lock.  None of these were smoking guns on this drop.

Will keep the Guardian and SDF medm windows open in case there's another drop---so I can hopefully catch what flashes and sheds ligth as to the culprit here.

21:33:04utc Went back to OBSERVING

• H1 lock approaching 45hrs.
• Winds have slowly trended up in the last 5hrs, and are currently hovering above 10mph (with gusts above 20mph).

This morning (17:08utc) we had an EQ alert for an earthquake on the small side (according to the SEISMON EQ Response plot).  On the other operator work station, I run with Seismon, SEI_CONF, and SEI channels up on ndscope to monitor EQs; I generally only transition to the EARTHQUAKE state when I see H1 begin to show effects from the EQ (via IMC, ASC, and seismometers).

The earthqake turned out to in fact be a small one with seismicity being elevated from 17:30-18:04utc as seen on the STSs.  So I ignored this EQ.

H1 Transitioned to EARTHQUAKE, But By Whom?

At about 17:40utc received call from Eyal S. at LLO via TeamSpeak letting us know that we were still in the EARTHQUAKE state (Thanks for catching this, Eyal!).  I checked the SEI_CONF log and it transitioned from WINDY to EARTHQUAKE at 17:12utc.  I'm certain I did not transition it, and Cheryl was here, but she did not tell me she transitioned, so I'm going to guess an operator didn't transition.  There are plans to automate how we handle earthquakes (see Jim's alog from last Thursday), but I don't think this is in effect yet.

18:09:  Since this EQ was negligible & the small seismic waves were long gone, I transitioned H1 back to the WINDY state.

Timeline Summary:

• 17:08 EQ alert received
• 17:17 SEI_CONF mysteriously transitioned to EARTHQUAKE state
• 17:30-18:04 Small seismic bump observed on site.
• ~17:50 LLO notifed me we were still in EARTHQUAKE state
• 18:09 I transitioned SEI_CONF back to the WINDY state

Addressed TCS Chillers (16:48-16:56utc & with Cheryl as my Buddy since she was also here while I did this):

• TCSx Chiller:  Had level of 29.1 cm.  Added 75mL for 30.0.
• TCSy Chiller:  Had level of 9.7 cm.  Added 40mL for 10.2cm.

HAM & BSC CPS:  All CPS' look good for high-freq range from attached plots.

TITLE: 12/22 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
OUTGOING OPERATOR: Cheryl
CURRENT ENVIRONMENT:
    SEI_CONF state: WINDY
    Wind: 9mph Gusts, 6mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.37 μm/s

Microseism continues downward trend for last 24hrs (was squarely on 90th percentile @ ~0.5um/s, and is now in between 90th & 50th percentile @ ~0.3um/s); Yay.  Other than that very quiet sitewide with only the one EQ Ed rode through during the EVE shift.
QUICK SUMMARY:

H1's been locked for over 41hrs with a nice range just under 120 Mpc.

TITLE: 12/22 Owl Shift: 08:00-16:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 116Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY: H1 in Observe all shift
LOG:

• 08:51 UTC - Kyle on his way to the site, to look at the instrument air compressor
• ~ 0930 UTC - Kyle completes adjustments to the instrument air compressor
• 13:00 UTC - I listened for unusual noise from the chiller yard, and hear only typical sounds, no instrument air motor sounds (though they may be present for a short time every few hours, as expected)

Violin Damping, non-nominal values:

• ITMX mode 8, nominal = -4, currently at -20
• ETMY mode 1, nominal = +0.1, currently at +0.5
• ETMY mode 6, nominal = 0, currently at -10 (I might reduce this before leaving, but currently this value)

The change of guardian code to allow violin mode damping gain changes during Observe has paved the way to damp previously undamped modes, reducing the peaks in DARM.   To capture what I've learned, changes I've made, planned changes, I'm putting in a summary.

Overall, I've found a number of filters that need to be adjusted, both damping and monitor filters, and typically the damping filters need to be more narrow, and monitor filters need to be biased to reduce the overlap for two close peaks.

Status, text in red indicates critical information:

• ETMY modes 12, 18, and 20 Damping: 
  • modes 18 and 20 are damping well, mode 12 is damping slowly, and the current gains have worked well during relocking at least 3 times - testing phase is over, gains are stable, mode 12 could use some attention, but it's not the highest priority
• ETMY modes 12, 18, and 20 Monitor Filters: 
  • currently reading each peak separately, in the stated where all three are small
• ETMY modes 1 and 6 Monitor Filters:
  • adjusted, improved, and would benefit from another look to see if there could be more separation
• ITMX modes 7 and 8 Damping:  
  • mode 7 peak is abscent from DARM, mode 8 peak is small but still visible, mode 7 gain is tested and stable at 10, but mode 8 gain last relocked at -4, and is now set to relock at -20.  Guardian will return this gain to -4 at relocking.  Please continue testing, by manually setting the gain to -20 gain at relocking H1, and then keep an eye on both modes 7 and 8
• ITMX modes 7 and 8 Monitor Filters: 
  • I've updated these, but they are not loaded, and until loaded, readback for mode 8 is only somewhat reliable, will have to see how much it improves
• ETMX mode 6 Damping: 
  • I've updated this damping filter, but it is not yet loaded, and until it is, it's not possible to damp both ETMY and ETMX mode 6 peaks.   ETMX mode 6 frequency is correct, as shown by the fact that I damped this peak until it was not visible above the background, with the filters that are currently in ETMX mode 6, but only with with ETMY mode 6 damping disabled

Filtering I've changed, as of 22 Dec 2019:

• ETMY modes 18 and 20 Damping: filters on mode 18 and 20 adjusted
• ETMY modes 18 and 20 Monitor Filters: modes 18 and 20 filters adjusted
• ETMY modes 1 and 6 Monitor Filters: filters on mode 1 and 6 adjusted
• ITMX mode 8 Damping:  filter on mode 8 adjusted
• ITMX modes 7 and 8 Monitor Filters:  filters on mode 7 and 8 adjusted (not yet loaded)
• ETMX mode 6 Damping: filter on mode 6 adjusted (not yet loaded)

Planned Changes on ITMX modes 2 and 3 Damping Filters:

• Current Filters:
  • mode 3: butter("BandPass",6,504.689,504.749)gain(120,"dB")
  • mode 2: butter("BandPass",4,504.841,504.941)gain(120,"dB")
• Change To:
  • mode 3: butter("BandPass",6,504.709,504.729)gain(120,"dB")
  • mode 2: butter("BandPass",6,504.881,504.901)gain(120,"dB")
• Bode plots attached:
  • LEFT = mode 3, RIGHT = mode 2
  • black = current, red = changed
  • vertical cursor at 504.8Hz on each plot shows changes at about half way between the two peaks

Current Mode "Pairs:"

• ETMX mode 6, ETMY mode 6 - this may change after the new ETMY mode 6 filter is loaded
• ETMY modes 1 and 6
• ETMY modes 11, 12, 18, and 20 - I've included mode 11, which hasn't consistantly participated with modes 12/18/20, but has come into play more than once, so is something to watch if driving any of the modes 12/18/20 with a higher than nominal gain
• ITMX modes 7 and 8
• ITMX modes 2 and 3
• ITMX modes 13 and 18
• ITMY and ETMX have pairs, to be added at a future date, when identified/confirmed

STATE of H1: Observing at 114Mpc, Observe clock a 7.5 hours
CURRENT ENVIRONMENT:
    SEI_CONF state: WINDY
    Wind: 4mph Gusts, 3mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.42 μm/s

CHECKUP ON CORNER STATION PUMP:  when listening from the kitched door, all is well, normal background noise

TITLE: 12/22 Owl Shift: 08:00-16:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 116Mpc
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
    SEI_CONF state: WINDY
    Wind: 5mph Gusts, 4mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.37 μm/s
QUICK SUMMARY: H1 in Observe, and I noticed the chiller yard at the Corner Station making a loud clicking sound

When I arrived for my shift I noticed a loud clicking sound coming from the Corner Station chiller yard.  Ed and I walked out there, and there is a "Switching Failure Alarm," on the pannel near the motor that's making the noise.    There's also a periodic release of air.  This is not normal for the chiller yard.  There's no alarms in the CR, however, this is new since this morning.  Calling Richard.

TITLE: 12/22 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY:

All of the fun stuff is in the previous logs from this shift

It has been drizzling rain off and on and Danny Sellers contaced me from LLO earlier to inform me of their lockloss following a "rung-up" BRS. They are back up and running

Cheryl has informed me of a mechanical sounding, clicking noise eminating from the corner station.
LOG:

04:56 The squezer fell out of lock again

• re-locked itself after 2 re-tries

04:57 H1 back to Observing

03:38 Verbal Alert and Telephone call

• Verbal initially reported it as a GRB-Short but GraceDb doesn't include it in the list of "Events", only "Superevents"

03:39 Conferrred with Joe Hanson at Livingston and confirmed the Superevent.

• Set INJ_TRANS to INJECT_KILL

03:40 Conference in EM Follow-up channel on TS is concurring that it looks like a clear BBH and that both observatories were in nominally good/non-glitchy condition at the time of the event.

01:14UTC The new coefficients were loaded as advised by Cheryl's alog , pursuant to the squeezer lockloss and the opportunity afforded during temporary loss of the Oberving Intent Bit.

After watching the manager try and fail to re-lock a number of times, I took the very un-scientific approach while I looked through alogs.

• I put the SQZ_MANAGER in "time out":  INIT⇒ DOWN
• a minute later I selected SQZ_READY_IFO - for no particular reason except it was an intermediate step. I left it there for a couple of minutes
• I then moved it to SQZ_ASC - my reasoning was the same as above, for a couple of minutes.;
• Then I went for it and welected the nominal state - SQUEEZING and it stuck.

01:23 H1 back to Observing

Perhaps the timing is a bit to quick for the system to stabilize for a solid locking sequence? anyone?

TITLE: 12/22 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 116Mpc
OUTGOING OPERATOR: Camilla
CURRENT ENVIRONMENT:
    SEI_CONF state: WINDY
    Wind: 17mph Gusts, 12mph 5min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.43 μm/s
QUICK SUMMARY:

• Camilla reminded me of the ITMX violin mode coeffeicients that need to be loaded at the next moment of opportunity
• Dr Schofield is in the control room investigating the uSeism noise that seems to be up-converting into DARM