TITLE: 07/26 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY: Slightly rough time getting back from a lock loss (see alog50840). Since then we have been observing for 4.5 hours.
LOG:

• 1525 GRB Fermi - Long
• 1536 Lock Loss
• 1704 Kyle to MY then MX
• 1818 Observing
• 1900 Kyle back
• 2017 GRB Fermi - Long

Derek Davis, Adrian Helmling-Cornell, Brennan Hughey, Jenne Driggers

Summary: Any increase in frequency noise at Hanford is worse than the current impact of whistles

Whistles currently are one of the most problematic noise sources for burst searches, and hence are a high priority to mitigate. Approximately 1% of data is corrupted by whistle glitches, although they can be easily identified in general. 

At Livingston, the rate of whistles dramatically decreased after swapping out the VCO. We investigated whether the same swap would be beneficial here at Hanford. The main trade off is a significant increase in frequency noise. 

The overall impact of a factor of 10x increase in frequency noise can be seen in attachment 1. Such an increase in frequency noise would drop the range down to ~65 Mpc. Even a more modest increase of 3x would lower the range to ~105 Mpc. As this decrease is much more significant than the current impact of the whistles, we conclude that any whistle fix must not significantly increase frequency noise for it to be beneficial. 
 

Anne Baer, Derek Davis

Summary: Extremely large earthquakes create short periods of significant periods of scattering that can produce significant background triggers for the searches.

Through mid-June of O3 there have been eight earthquakes that resulted in ground motion above 1000 nm/s. For half of these times, the interferometer was able to stay in lock throughout the entire period of the earthquake.

Half of these high earthquake times have significant periods of loud scattering arches that impact the searches. It is not clear what causes these times to result in a larger number of glitches. When this scattering does occur, the glitches tend to produce loud outliers in the PyCBC background, especially for templates with durations of 0.5-4.0 seconds. Earthquake-related glitching is one of the limiting sources of noise for this part of the parameter space.

Spectrograms showing the duration of the high earthquake periods show significant noise at low frequency (see attachments 1 and 2), corresponding to scattering arches in the zoomed in omega scans (see attachments 3 and 4).

The excess noise produced by these large earthquakes is also visible in hourly omicron glitch plots, with excess glitching at 20-50 Hz lining up with the start of the earthquake periods (see attachment 5, 6, 7, and 8).

In all, extremely large earthquakes are related to significant periods of excess noise and should be flagged by the searches. However, as noted in alog 50844, this glitching is not significant enough to warrant leaving observing mode and sacrificing the increase in observing time.

John Zweizig, Dripta, Nicolas Arnaud, Brennan Hughey

In FRS ticket 13123 it was asked whether it was okay to leave H1 in observation ready/intent while transitioning to earthquake mode.  The short answer is "yes", it appears okay to leave the IFO in observation ready mode when making this transition.

We examined glitch rate based on omicron triggers, BNS range, and other metrics during the ~minute-long period when the transition into earthquake mode was taking place for 55 of these transitions during observing time.  We compared the values to periods within a few minutes before and after the transition into earthquake mode to see whether the actual transition was having a negative impact on data quality.  The average glitch rate for SNR>6 omicron triggers does appear elevated by a factor of ~1.6 relative to surrounding times during the transition periods, but this is not nearly a substantial enough increase to warrant taking the IFO out of observing. This increased glitch rate wasn't traceable to a specific type of glitch, as the increase in glitch rate is very broad band.  Average BNS range is about 105 Mpc during the transitions into earthquake mode, which is not substantively different than the average BNS range in the minutes immediately surrounding the transition.  A direct check of the coherent WaveBurst online burst analysis shows that loud background triggers (as characterized by a significance of about one event per year of background) occur ~50% more often during the earthquake mode transition times than one would expect at a random time during the run, which is a pretty minor deviation from average background.

When assessing the data quality of candidate triggers it will be worth noting if they occur during one of these transition periods, so it will be useful for John Zweizig to continue generating the segments he produced to enable this study.  However, the impact of the transition into earthquake mode on data quality appears pretty modest.  It isn't an issue the transient searches will need to automatically veto, and it certainly isn't worth taking the IFO out of observing during the transition period.

    

1st attempt:

• Green locked all on its own!
• DRMI was very close and would frequently catch but on a TEM10. When the try PRMI timer was up, it never transitioned. I'm not sure why it didn't do this yet. So I took it there manually and it locked up quickly, then locked DRMI right after.
• Lost lock at ENGAGE_ASC_FULL_IFO. Many sus saturations for a minute or two and ASC didn't look good. Not sure why there were problems at this point, it seemed well aligned.

2nd attempt:

• Green locked on its own again!
• Find IR COMM was trying to fine tune basically nothing. I brought it to manual and then did a fast search, this worked.
• DRMI did not lock, went to PRMI automatically (the DRMI flashes did not look as good as last lock attempt). PRMI locked up quickly and didn't need much ASC tuning.
• DRMI just would not catch. I tried tweaking SRM and though I had numerous "mini-locks" and good flashes, I could not get it to lock. Giving up and running and IA.

Initial Alignment:

• All automated except SRC align, I wanted to try Jenne's new SR2 states, but I could not select them and the automation ended up fixing whatever was wrong anyway.

3rd attempt:

• Green auto locked
• DRMI quickly locked
• Made it the rest of the way, no issues

SDF:

• Only one diff, and it was a high precision with MC3. This is the first time in a long time that I have not had the usual ones.

Anne Baer, Adrian Helmling-Cornell, Siddharth Soni, Derek Davis

Summary: Blip glitch behavior at Hanford has changed slightly between O2 and O3, but generally appears with the same features as before. Blip glitches do not appear related to the high-SNR glitches seen in O3.


We investigated blip glitches at Hanford and how they have, or have not, changed from O2 to O3 using the blip data from GravitySpy.

Comparing the displayed quality factor, and the distributions of SNR and peak frequency of the blips, we found that the peak frequency seems to have shifted lower for O3 (see attachment 1, O2 in red, O3 in blue). The distributions of SNR show only a minimal difference between O2 and O3. Notably, this suggests that blip glitches do not account for the large number of high-SNR glitches observed in O3 (see attachment 2).

The minimum frequency also appears to have slightly dropped in O3, but we are unsure whether this is a statistical noise or not. However, this does match the expected change from a constant amplitude source.

The ASD curves of the blips from O2 and O3 do not show any significant differences in glitch spectra, nor does the spectra seem to indicate any significant features, looking mostly like white noise after 70 Hz (see attachment 3 for example ASDs)

The distance of the blips from the start or end of a lock segment also seems to be very similar, indicating that the blips are not related to lock loss or acquisition. O3 blips seem to be slightly more likely to be towards the beginning of lock and O2 more towards the end, but neither significantly so. The time duration between individual glitches also appears to follow that expected from a random process.

Comparing the rates of blip glitches in April of O2 and O3 at Hanford, there does appear to be a small increase in the number of glitches (see attachment 4). However, this rate was highly variable in O2, and higher by an order of magnitude during winter months and periods of low internal humidity. 

Seems very sudden, nothing obvious from the lock loss plots. 1248190591

TITLE: 07/26 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 45Mpc
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
    Wind: 7mph Gusts, 5mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.06 μm/s
QUICK SUMMARY: Almost a 4 hour lock, calm environment. ITMX violin mode 13 is still a bit high.

TITLE: 07/26 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 118Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY:

Everything was stable after the last re-lock. there were a couple of normally expected saturations. Handing off to TJ.
LOG:

08:54 Lockloss _HTTS saturation and....

09:28 after much manipulation of both arms' ETMs and TMS, I was able t move on.

09:32 weak PRMI lock and PRMI_ASC caused lockloss.

09:43 IA - Arms not copperating at all. Before this gets too time consuming, I'm doing an alignment.

10:09 IA complete - begin locking

10:24 After reaching a rathr good looking PRMI, Guardian moved to DRMI 1F and failed to catch. Upon moving back to PRMI the mode cleaner be3came unlocked and the ISC_LOCK went back to down.

11:10 NLN

SDF diffs accepted (images below)

11:11 H1 back to Observing

TITLE: 07/26 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 116Mpc
OUTGOING OPERATOR: Travis
CURRENT ENVIRONMENT:
    Wind: 5mph Gusts, 4mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.06 μm/s
QUICK SUMMARY:

Pretty beautiful clear night. Calm winds. ITMX Mode 13 rung up over refernce zbout 9 e-17 but trending downward as per ndscope. Range seemingly not affected.

TITLE: 07/26 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: Ed
SHIFT SUMMARY:  Locked for my entire shift and in Observing for most of it after PEM measurements completed.  No issues to report.
LOG:

23:36 Kyle back from MY

0:46 Switched SEI Config to EARTHQUAKE for 5.1 EQ in California.  

1:14 SEI Config back to WINDY

M. Ball

I looked at the electrical voltages measured at the two end stations and the corner station during the lockloss associated with the power glitch during the thunderstorm on 7/24 at 3:56 UTC.

I have attached an image of the voltage signals at these stations and DARM during the glitch showing little to no power dip during this time. Unfortunately, the sample rate of the voltage monitors is too low to see full oscillations, so I can only say with confidence there was less than 10% dip in power.

I have also attached microphone spectrograms from 10 minutes on either side of the lock loss, and there are no signs of thunder around this time.

I have created a new set of states for the ALIGN_IFO guardian to do the alignment of SR2 with SRM misaligned.  This is something that operators often do by hand when the SRC won't lock.  I have not yet tested these states, but hope to on Tuesday when we're recovering from maintenance, or next time I'm around that we need to do an alignment. Once the states are tested we can add some logic to TJ's INIT_ALIGN guardian to give up on SRC locking after some seconds of trying, do SR2 alignment, then go back to SRC alignment.

Finally!!

TITLE: 07/25 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Commissioning
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    Wind: 14mph Gusts, 11mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.05 μm/s
QUICK SUMMARY:  H1 is finally back to locking.  Currently in Commissioning mode for PEM work.

TITLE: 07/25 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Travis
SHIFT SUMMARY: Currently commissioning in low noise. Lock reaqusition was a bit tricky, ALSX WFS would pull away whenever it would lock. I got around this by turning off the DOF1 P filter and then turning it on after the rest of the WFS had settled down a bit. Then DRMI looked well aligned and had good triggers, but it just wouldn't hold. An automatic PRMI, then a bit of SRM Y finally let us aquire DRMI. Violin mode 13 on ITMX is a bit elevated, and I have yet to find a good gain/phase setting. Hopefully Travis can crush this mode.
LOG:

• 1559 Richard to turn on and off HVAC in CER. Should not impact the instrument.
• 1644 Vanessa to MX
• 1654 Richard done
• 1745 Kyle to MY
• 1821 Kyle back
• 2023 Lock Loss
• 2054 Kyle to MY
• 2158 Robert, Matthew to LVEA to set up for commissioning time.
• 2202 Reached Nominal Low Noise, but we will begin commissioning

J. Kissel

Having lost lock in the same "within a few seconds" timescale from doing nothing but rotate the rotation stage and adjust the "POWER SCALE" -- i.e. take the LASER_PWR -- guardian from POWER_2W to POWER_3W, I'm suspecting PDs falling off of useable area. We not the that IMC global position has moved quite a bit (at the ~10 urad level) from last week -- and since then we've had two major crashes (dolphin crash on Monday, and ) -- Here's a trend of the ISS QPDs. There's a major change on 2019-07-14 ~11:53 UTC (Sunday Jul 14 2019 04:53:00 PDT), but nothing recently of note.

Attached is a trend from June 19 2019 to now (prior to June 19th, we're stable at the Jun 19th value since the start of O3).