I set the OPS_OBSERVATORY-MODE to Earthquake, but I'm not entirely certain that was the culprit.  We were at the peak of the motion from a 5.3M EQ in Chile, but the BLRMS top out at only ~0.1 um/s, so well below the typical lockloss threshold.  Jim's StripTool shows some wiggles in the BS CPS traces, but not what I'd call glitches.

For the SEI team, I attach screenshots of the USGS, Terramon, and SEISMON 5-EQ screens that show that SEISMON is not reporting the latest EQs (there have been several EQs over 4.0M with dates of 2017-07-14 that don't show up on SEISMON). 

Called LLO control room to verify they received the alert.  Jeremy at LLO verified they did, and logged back in to Teamspeak since their Teamspeak machine had rebooted.

All appears to be well.

TITLE: 07/14 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 53Mpc
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
    Wind: 5mph Gusts, 3mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.06 μm/s
QUICK SUMMARY:  We are finally back to Observing even with the range still a bit lower than we'd like.  I'll keep an eye on the BS ISI CPS for glitches if we have any locklosses.

TITLE: 07/13 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 54Mpc
INCOMING OPERATOR: Travis
SHIFT SUMMARY:

I spent the beginning of the shift trying to help Jeff and Hugh damp violin modes. I worked on adjusting the gains for the first fundamental modes 2, 4, 6 and 8 of ETMY. Increasing the gain on modes 4 and 6 did not seem to have any effect on the ~508.219 Hz mode. Increasing the gain on modes 2 and 8 may have helped damp the ~507.992 Hz and ~508.66 Hz modes respectively, or they could have just been coming down regardless, it was unclear. Thomas, Pep and Jenne worked on moving the beam spot positions but reverted their changes before they left. We had another lock loss attributed to the BS ISI CP glitching. Hugh made another change to try and address it (see his alog). We can now make it to NLN without intervening in the violin mode damping and without skipping any states. After consulting with Vern we set the intent bit to observing despite the range of ~54 MPc.

LOG:

23:55 UTC GRB verbal alarm. LLO not on teamspeak. Ignoring.
00:42 UTC Lock loss. BS ISI CP glitch. Thomas and Pep reverting beam spot moving ASC changes.
00:44 UTC Hugh to CER to powercycle BS interface chassis.
00:48 UTC Untripped BS ISI. Hugh back to CER to tighten screws.
01:00 UTC Thomas and Pep done. Relocking.
01:36 UTC NLN.
02:01 UTC Observing after consulting with Vern.

Damping of the baffle has been shown to have successfully reduced noise in DARM from local vibration injections around the input beam tube (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=36979), but an improvement for site-wide vibration has not been shown until now.

While we have been down for some time following the Montana quake, there was enough data before the quake that I could look into indicators of site-wide coupling. The evidence here suggests that the baffle damping also reduced coupling of global 10-30 Hz vibrations and thus supports the hypothesis that the baffle was the dominant coupling location on site for this band.

The rush hour range drop was thought to be produced by scattering from the Swiss cheese baffle (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=35735), and it was thought that damping might reduce this nearly daily drop in range.  Each of the panels in Figure 1 show the 10-30 Hz seismic band and the inspiral range. The rush hour traffic features are the broad ~gaussian peaks (from many cars) at the beginning and end of the work days. The panels from April and last December show that the range dropped during each rush hour (unless the range was already low), and that this has been happening for the entire run. The recent plot does not show similar range drops, suggesting that site-wide coupling in this band has been substantially reduced (as an aside, we do still often loose lock during the morning rush hour, even though the range doesn’t drop, and this bears further investigation).

To test for a global effect when the traffic noise was not elevated, I used Jenne’s jitter subtraction code and compared non-rush hour times before and after the vent. I took 1024 second stretches that were matched in original inspiral range and applied Jenne’s code. Figure 2 shows that the range with subtraction was, for each pair, higher after the vent. The difference was only a few percent but is significant (at p<0.05 ). Because the scattering noise is highly non-stationary, damping may also have improved stationarity (compare minute-scale range variation in Figure 1).

After consulting with Vern, we (Jeff, Hugh and I) set the intent bit to observing at 02:01 UTC. The range is around 53 MPc.

Pep C. Thomas V. Jenne D.

After heroic efforts to successfully dampen the violin modes and get to NLN, the BNS range sat around 55 MPC so Jenne thought it would be a good to optimize the beam position in the arm cavities to match the pre-Montana-Earthquake.

This is done by first changing the A2L gain coefficients to what they were on June 17th 02:00:00 when Sheila was able to achieve better BNS range, and then turning on the dither scripts that show the length to darm coupling found in userapps/isc/common/scripts/decoup/setA2lines.py.  We then adjust the CSOFT and DSOFT loop offsets to minimize the dither resonances, which effectively moves the spot position back to what Sheila's settings were in mid-June.  After doing this, there seems still be the same amount of jitter noise at low freq ~10-80Hz.

However, it seems that the input pointing from the IMC has also shifted since the earthquake according to Cheryl's ALOG and we were messing with the IMC loop DOFs to adjust the WIT sensors back to their pre-EQ values, this proved to hurt our sensitivity a bit so we might need to spend more time fine-tuning this approach.

Because we haven't really improved the sensitivity, we changed the A2L gains back to their values on July 12th 01:06:22 and going to a more stable lock state.

Still drilling into the fine print but this still holds although there may be more details to uncover.

After this glitch, I power cycled the sensor chassis in the CER--this did not include the T240s.  After doing so I noticed the power cable to the Corner3 chassis was not screwed in and the cable was somewhat tight and the connector did not seem square/fully seated.  I pulled a bit of slack in, seated and screwed down the connector--we'll see...

C. Gray, J. Kissel, R. McCarthy, H. Radkins, P. Thomas

We now believe we've confirmed the functionality of all fundamental violin mode damping for all 32 modes of the 4 test masses. As such, I've reconciled the VIOLIN_MODE_DAMPING guardian state, and updated the Violin Mode Table such that all agree with each other and what has been confirmed successful. I couldn't keep track of every mode that had changed, but I'd say it was about 5 modes out of 32 that have changed its settings (and not restricted to any particular test mass, the ITMX has been relatively trouble free). Our latest theory is that the fiber mode shapes and/or their coupling have changed as a result of the earthquake, and thus they require new actuation magnitude / phase.
 
While we only have statistics of 2 lock acquisition sequences, we've been able to successfully turn on automated turn-on of these damping loops as per normal.

We're still having trouble completely squashing all fundamentals (namely EX MODE4 at 505.805, and EY modes 3-6 around 508.2 are still interesting), but they're at least stable.

In our spare time, we're slowly crushing the 2nd harmonics (stuff that's not coded into guardian because it's not typically problematic, but still rung up from the EQ), and updating the reference table accordingly. This is limited by the number of filter banks and our patience, but we're otherwise successful. After O2, we should definitely increase the number of filter banks on each suspension.

Notes:
- this further work on violin modes has not improved the sensitivity. 
- this work has not been impacted by the IFO or IMC beam spot restoration / moves / tests that occurred concurrently.
- just about all lock losses today are from the Beam Splitter ISI's CPS glitching.

TITLE: 07/13 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    Wind: 17mph Gusts, 13mph 5min avg
    Primary useism: 0.06 μm/s
    Secondary useism: 0.06 μm/s 
QUICK SUMMARY:

Sitting at NLN. Jenne and Thomas are moving spot positions. Jeff and Hugh are damping violin modes. I will attempt to help damp violin modes.

TITLE: 07/13 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake
INCOMING OPERATOR: Patrick
SHIFT SUMMARY:

Today we had multiple electronics glitches on the BS ISI CPS's which tripped the ISI.  Jim went out to power cycle in the morning & then reseated cards in the afternoon.  Jim has a striptool running on his computer looking at the CPS for the BS ISI.  So, if we get another BS ISI trip, one should go look at his striptool to see if there is a spike on the StripTool.

Majority of today was spent continuing to damp fundamental (~500Hz) violin modes, updating Guardian settings needed to damp them, and also update the violin mode table wiki page.  We have damped all of these modes to a point where we are now happy & can now NOT have to skip the VIOLIN_MODE_DAMPING_1 Guardian step.  So after 1-attempt, all the violin modes have been happy for us---so now we should be able to proceed all the way to NOMINAL_LOW_NOISE.  

Jeff, Hugh, & Patrick will continue damping violin modes.  

Jenne & Thomas are currently running a reverse A2L to get us to a good alignment state for the Test Masses.  

We are currently at about 50-55Mpc.

Terra & Anamaria mentioned being available if we need additional help.

One other operational note is that we don't have a viewable AS90 (purple) trace on the wall StripTools.  This is related to some work Kiwamu performed.  We should get used to it since this channel will not be around after O2.
LOG:

• 15:46 Elizabeth working on All Sky camera on roof
• 15:49:  Jim heading to BSC2 to power cycle BS ISI CPS Satellite rack (out at chamber)---this is due to another BS ISI Watch Dog Trip.
• 16:39:  @LOW_NOISE_ASC (skipping VIOLIN_MODE_DAMPING_1) & addressing violin modes now.
  • Jeff K, Hugh, & Corey working on ITMx, ETMx, & ETMy violin modes
• 16:55:  Communications Message from Kendall at Hanford site.  Seeking LHO Communications contact.  Forwarded her to Bailey.
• 17:04:  Kiwamu heading to CER to check out some RF cable options & also went out to elec rack at HAM6.
  • 18:00-19:16 WP7075:  RF signal cables for HAM6 work
• 20:50 Chandra to mid-stations for tools
• 21:08 LOCKLOSS:  BS ISI had another CPS Watchdog trip (Jim heading out to address)
• 21:30 Liz brought in another tour

Corey, Jeff, Jenne, Kiwamu,

Apart from the violin issues, we experienced two issues in locking the interferometer yesterday.

• The ASC SRC2 loops (a.k.a. the SRC pointing loops using ASC-AS_C) ran away multiple times.
  • This seemed to be due to saturation in the AS_C QPD caused by the violin modes.
  • Once Jeff and Corey damped them to the level where the AS_C QPD didn't saturate, Jenne and I were able to close the SRC2 loop without any issues.
• The DC readout transition failed several times.
  • This was also observed early in this week (37467). We still don't know why it kept failing.
  • More confusingly, it never failed today even though we didn't change any settings. This might be somehow due to the high violin modes that are now much lower today.
  • Jenne took a look at the spectra of AS_RF45_Q and DC readout signals from a past successful transition. They turned out to be almost identical (to our eyes) to those we currently see. So there are no issues in the sensing side.
  • We once went through the matrix ramping manually, by ramping them to a couple of intermediate matrix values. This worked fine although we are not sure what this means.

As was alluded to earlier by Corey, it looks like the CPSs on BSC2 started glitching this morning and caused the ISI to trip, breaking several locks. Attached plot shows several hours around the locklosses this morning. The windows where the ST1 WD was at state 4 are when the WD tripped. The long fuzzy period on the left of the CPS plots are the earthquake this morning, only ST1 H3 & V3. It looks like the CPS started glitching some time after the earthquake. I went out, pulled and reconnected the power on the satellite racks on the chamber. I've had a striptool running of the CPS since power-cycling and it looks like the glitching has stopped. Given the glitches start before getting bad enough to trip the ISI, maybe we can come up with some of diagnosing before this becomes a problem, so we at least know what we have to fix. I don't know what that would be though. We could at least get longer trends (an hour or two) of CPS if a "mysterious" ISI trip is suspected of causing a lock loss.

WP 7075

To further proceed with the 72 MHz WFS (Wave Front Sensor)s (37042), today I made the hardware changes (mostly cabling), as summarized below, while the interferometer was in a violin-mode-damping state.

I am going to leave the hardware configuration as it is. If this new setup doesn't cause extra noise in the interferometer, they will stay semi-permanently.

Here is a summary of the new configuration:

• We have the 118 MHz RF modulation signal applied to the EOM with the full RF power (+33dBm@PSL rack).
• And we have the 72 MHz RF local oscillator signal sent to the HAM6 area, providing the 72 MHz WFS signals.

[The modifications]

• I plugged the new 72 MHz RF source to the unused RF Heliax cable so that the signal is sent from the CER to the ISC R3 rack (by HAM6).
  • Additionally, I newly installed a balun at the HAM6 side which was missing for some reason.
• At the ISC R3 rack, I routed the 72 MHz to the distribution amplifier which had been used for the 90 MHz demodulators.
  • In this configuration, the distribution amplifier feeds the WFS demodulators which had been also used for the 90 MHz but now with the 72 MHz signal.
• At the ISC R3 rack, the 90 MHz RF cable was rerouted so that it directly feeds the demodulator for ASAIR_B_RF90.
• I adjusted the RF power level by inserting appropriate RF attenuaters while looking at the EPICS readouts from the distribution amplifier and WFS demodulators.
• In the LSC, I changed the FM8s for LSC_ASAIR_B_RF90_(I|Q) from +3 dB to -1 dB in order to keep the same level of signals. See the attached screenshots for the changes that I made.
  • Also, I have changed the digital demodulation phase of ASAIR_B_RF90 such that the signal is maximized in I when the interferometer was locked at 30 W.

LLO has had this for a while, but with help from DaveB, Arnaud, KeithT, SebB and MichaelC, we now how a seismon screen that shows the last five earthquakes. With the old single event screen it was conceivable that a smaller earthquake could happen after a larger, problematic earthquake arrived, making it difficult to identify the source in the case of a eq induced lockloss. There are still a number of issues with the new code (some inconsistencies in channel names, new channel names not in the DAQ, the Location name doesn't display on the new overview, some of the old code is still running to keep EDCUs happy...), but at the moment it mostly seems to work. This probably will break the verbal notifications until TJ and I have a chance to talk about how to update the test.

I'm also not sure how stable the new configuration is, so I'd like to ask operators to keep the an eye on the screen and let me know if seismon's list continues to agree with the USGS list. Unfortunately, this change also breaks the old seismon screen, though I'll leave the link on the sitemap up, is case we go back to the old code. The new screen can be found by going to SEI --> SEISMON-5_event on the sitemap.