Verbal alarms started alerting that EY was saturating at 02:04 UTC. It has continued since. Attached is a plot of the L3 ESD drive signals for the last 2 hours. They have been growing. Is this the violin mode at 4.7 kHz that Keita and Cheryl were investigating?

TITLE: 12/16 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 71.2515Mpc
OUTGOING OPERATOR: Cheryl
CURRENT ENVIRONMENT:
    Wind: 0mph Gusts, 0mph 5min avg
    Primary useism: 0.08 μm/s
    Secondary useism: 0.20 μm/s 
QUICK SUMMARY:

Locked for over 11 hours.

TITLE: 12/15 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 70.0775Mpc
INCOMING OPERATOR: Patrick
SHIFT SUMMARY:

Activities:

• earthquake: event time: 22:23:29UTC: alog 32611
• earthquake: event time 22:53:03UTC: no alog, 0.05um/s on DMT
• EY gltich at 23:23:09 and 23:23:17UTC
• range drop 23:24UTC, L1 lost lock
• DUST ALARM: 23:25:11UTC: PSL: 0.3um, 220PCF, cleared: 23:36:11UTC

Starting time of event on BLRMS: 22:48UTC

Lock status, did H1:

• no effect: 22:48UTC
• glitch: 23:10UTC

EQ reported by Terramon before it actually arrived: no

• Overall Duty Factor 75.2%
• Range generally around 70 Mpc, with some fluctuations down to 60 Mpc (often around 12-14 utc).
• ~1080 Hz glitches/peak continues throughout shift
• Dataquality degraded in the minutes prior to lockloss on Monday, Dec 12.
• Some "crash" at EY on Tuesday caused loudest pycbc-live trigger for the day (visible in many channels).
• BBH injection on Wednesday, Dec 14 01:05:44

A scan of coherences can be found at the following link:

https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_lho_1165834817/

Here's my summary

• low frequency (<20 Hz) is coherent with CHARD_Y (fig. 1)
• MICH/SRCL/PRCL are low (MICH very low, the other two somewhat higher but still far from the noise spectrum) (fig. 2,3,4)
• not much below 100 Hz
• IMC WFS signals between 100 and 800 Hz, especially at the bumps in th 300-400 Hz region, where the coherence is very high (fig. 5). The same coherence pattern is visible in ISS signals (fig. 6) and IMC REFL (fig. 7)
• Not sure what this is, but there is large coherence with SUS-ETMY_L2_FASTIMON_UL_OUT_DQ and similar signals (fig. 8), broadband between 100 and 1000 Hz and very high above 600 Hz. The coherence seems to be the largest for UL, the smallest for UR and intermediate for the other two coils
• around 1kHz there is coherence with OMC-PI_DCPD_64KHZ_AHF_DQ, but I'm not sure if this is important, since I don't know what that channel is (fig. 9)

There has to be some bonus with this weather.

TITLE: 12/15 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 71.2751Mpc
OUTGOING OPERATOR: Corey, Jim
CURRENT ENVIRONMENT:
    Wind: 1mph Gusts, 0mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.21 μm/s
QUICK SUMMARY:

• locked for 5 hours
• one drop from Observe for violins, Jim's alog 32600

Terra noticed that one of the 1000 hz violins was pretty high. Because LLO is down, we decided to quickly try to address it. At 17:29, I ramped off the gain for the 4.7khz mode for ETMY mode 10, switch the BP filter to FM1 for the 1009.44 hz mode and left the 100db filter on. While watching the filter output and a live spectra, I slowly worked the gain up, it's sitting at 100 right now and both 1009hz modes seem to be coming down. We'll watch for a little while longer, then move back into Observe.

TITLE: 12/15 Owl Shift: 08:00-16:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 73.335Mpc
INCOMING OPERATOR: Cheryl later if the winter weather/roads cooperate
SHIFT SUMMARY:

Have been in locked for all but 1hr of the shift (down for two locklosses).  Weather is the big news of the shift thus far.  There have been delayed work advisories issued and the latest is a Work Cancellation for the Hanford site---this is all due to Adverse Weather Conditions.  

I have talked with the LLO operator and have notified them of our situation & we should talk to them as soon as we get coverage back here, so they can stand down with covering for us.

LOG:

• Dust Monitor Alarms for the LVEA up to 30k+
• EY Saturations:  11:49, 11:57
• 8:09 Topped of TCSy Chiller
• 9:32-10:05 Lockloss
• 10:28 Snow plow passing by the site on Rt10
• 11:25 Topped PSL Crystal Chiller
• 13:15-13:45  Lockloss
• 13:45 Snowblower near Control Room

Josh, Andy, David

After stumbling on the ringing phones in the LVEA in O2 (entry 32503) we sent this request, to find other times in h(t) that look like phone ringing glitches, to GravitySpy citizen scientists. Though it's dirty laundry airing this is a way to engage a network of many interested folks to find issues.  

The reply from user @EcceruElme pointed to two similar glitches in O1/ER10/O2.
One is in Livingston at 1132398529.14 (Nov 24th 11:08 UTC - link) and looks like an ER10 instance of engaging violin mode damping while in observation ready but before SDF checks were being enforced, so ok.
One is in Hanford at 1128269436.32 which is 2015-10-07 at 16:00 during a long nice science mode lock (summary page for that day). This sounds a little like a fax machine then some bangs and a human voice through a loudspeaker and some more bangs. In the hour around that time there are also some other loud bangs. We have no idea what it could be but it happened in otherwise unbroken excellent O1 data so perhaps folks on site know what it is and if it can be (or has been) turned off.

Attachments:

- Audio file of Input Optics Microphone with no filtering.
- Spectrogram of Input Optics Mic (it was also loud in BS Mic, others in LVEA but quiet in PSL room Mics) and Strain
- OmegaScan of Strain (from GravitySpy) 

Looking at a few tasks on the calibration to-do list, I ended up down the rabbit hole. Sadly, it's getting late and will have to return to this tomorrow.

Investigations:

1) Working on fixing the PCAL --> CAL-CS_DELTAL_EXTERNAL calibration (see LHO alog 31994). Kiwamu's original script computed a transfer function accounting for time delays caused by the DAQ and the CAL-CS whitening filter. However, this is already taken into account in the clock cycles delay in the CAL-CS model when adding the sensing and actuation paths. Since CAL-CS corrects for the phase delay caused by AA/AI filtering--analog and digital--using the delayed actuation path, then what one really wants is to correct CAL-CS for the magnitude changes of the AA/AI filtering and any CAL-CS whitening on this channel while also correcting the PCAL_RX_PD channel for the effect of AA and that we also have not applied a filter with two poles at 1 Hz (for the free-mass response):

DELTAL_EXT   W * [Derr/C_foton + A*Dctrl*delay]
---------- = ----------------------------------
PCAL_RX_PD         m * f^2 * AA_a * AA_d

 m     DELTAL_EXT [C_real/C_foton] / [W * abs(AA_a * AA_d * uncompensatedOMCpoles)]
--- = ------------------------------------------------------------------------------
 m                PCAL_RX_PD / [f^2 * AA_a * AA_d]

Note that the correction factors applied in the numerator of the final equation are really only to be applied to the sensing side. However, the corrections are only at high frequency (low-frequency corrections are simply unity), so the numerator term is dominated by the sensing function.

Thus the calibration to be applied is thus:

   [C_real/C_foton] * [f^2 * AA_a * AA_d]
----------------------------------------------
[W * abs(AA_a * AA_d * uncompensatedOMCpoles)]

Unfortunately, this results in a large deviation in phase at higher frequencies (~60 degrees at 1 kHz, see attached figure) while recent measurements using an incorrect calibration filter do not suffer from this large phase deviation. The phase deviation is due to application of the analog and digital AA on the Pcal. Why?

2) I constructed an O2 version of the front-end calibration time delay diagrams based on the model (see attached), but when I look at the GDS correction factors to be applied to the CAL-CS channels, I do not find matching delays. In addition, the values obtained to not mesh with the recent work done at LLO (see LLO alog 29899). This will have to be investigated further tomorrow also.

I've restarted the primary and redundant h(t) pipelines at Hanford at GPS second 1165664237. This restart picks up a software update to gstlal-calibration-1.1.2-v1, which contains a couple of bug fixes:

(1) The h(t) data product should now be identical between the primary and redundant pipelines when time-dependent corrections are applied (i.e., the previously observed numerical roundoff error has been addressed);

(2) The pipeline will now run properly in offline mode, which had become a serious problem after the first bug fix referred to above.

The filters file has also been changed; see this aLOG: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=32389

Note, time-dependent corrections to h(t) (the kappa factors) are still being applied in low latency at Hanford, with the exception of changes to the coupled cavity pole frequency.