TITLE: 03/29 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Nutsinee
SHIFT SUMMARY:

IFO has been down for Commissioning all day in search of some un-wantedness on an ITM surface.
LOG:

   Run BRS Drift Mon check: results are posted below. BRS-X looks good. BRS-Y has been in a downward trend since 02/15/2017. It crossed the lower warning line around 03/22/2017. With the exception of a couple of spikes up to zero, it has been below the warning line since. 

Laser Status:
SysStat is good
Front End Power is 34.09W (should be around 30 W)
HPO Output Power is 165.3W
Front End Watch is GREEN
HPO Watch is GREEN

PMC:
It has been locked 0 days, 6 hr 15 minutes (should be days/weeks)
Reflected power = 16.57Watts
Transmitted power = 60.79Watts
PowerSum = 77.36Watts.

FSS:
It has been locked for 0 days 4 hr and 17 min (should be days/weeks)
TPD[V] = 3.368V (min 0.9V)

ISS:
The diffracted power is around 3.7% (should be 3-5%)
Last saturation event was 0 days 4 hours and 45 minutes ago (should be days/weeks)

Possible Issues:

Xtal Chiller "light" is showing intermittent alarm on Statues Screen. 300ml of water was added

model restarts logged for Tue 28/Mar/2017
2017_03_28 11:26 h1omc
2017_03_28 11:27 h1calcs
2017_03_28 11:30 h1broadcast0
2017_03_28 11:30 h1dc0
2017_03_28 11:30 h1fw0
2017_03_28 11:30 h1fw1
2017_03_28 11:30 h1fw2
2017_03_28 11:30 h1nds0
2017_03_28 11:30 h1nds1
2017_03_28 11:30 h1tw1
2017_03_28 12:47 h1calcs
2017_03_28 12:49 h1broadcast0
2017_03_28 12:49 h1dc0
2017_03_28 12:49 h1fw0
2017_03_28 12:49 h1fw1
2017_03_28 12:49 h1fw2
2017_03_28 12:49 h1nds0
2017_03_28 12:49 h1nds1
2017_03_28 12:49 h1tw1

maintenance day, new calcs and omc code, associated DAQ restarts.

model restarts logged for Mon 27/Mar/2017 - Fri 24/Mar/2017 No restarts reported

Starting CP3 fill. LLCV enabled. LLCV set to manual control. LLCV set to 50% open. Fill completed in 201 seconds. LLCV set back to 17.0% open.
Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 1137 seconds. LLCV set back to 33.0% open.

Last night we had some decent winds so here is a comparison of the ITMY STS (STS2-B) and the HAM5 STS (STS2-C) with STS2-C at the Roam1 location--guess I'm going to have to make a drawing.

Spectra starting at 1845 UTC 28 March, winds pretty steadily above 15mph with gusts to 30, from 30 to 50 degrees (hitting the SW corner of the LVEA building.)  This wind state was pretty consistent for nearly four hours; more than enough for ten 1mHz averages.  See attachment the first.

The second attachment are X Y and Z spectra plots comparing the STS2 B & C during the windy period last night and a calm reference time of March 24--seen also in aLog 35075. The Reference traces are from the quiet time and the current traces are from last night. Notice the change in the microseisms especially the primaries (around 60mHz) and how the tilts impacting the X & Y traces pretty well obliterates this primary.  Is that the tertiary showing in the references around 0.18Hz?  Sorry about the cryptic title: CurrWind 20 Y-X means the current wind is 20mph from the +Y - X direction (45 dataviewer degrees or true South [for LHO]).

Not sure what to say about the differences in the Z signals between 1 & 7mHz during the windy time...  But, as for X & Y tilt susceptibility of the ITMY and the Roam1 positions, I'd say they are the same.  Time to move to Roam2.

J. Kissel, J. Betzwieser

Patrick had called out that line features the DARM sensitivity around 35 Hz were "broadening and contracting" in a recent lock stretch starting 2017-03-29 11:10 UTC, which impacted the inspiral range (see LHO aLOG 35173). These features are the 35.9, 36.7, 37.3 Hz calibration lines actuated by the ETMY SUS, PCAL, and overall DARM_CTRL, respectively. 

What Patrick has seen is classic alignment-related up-conversion, where low-frequency (between 0.05-10 Hz) angular motion mixes with the intentionally, and necessarily loud calibration lines and creates side-bands on the CAL lines that mirror the ASD of angular motion. We've been anecdotally/casually/qaulitatively seeing this several times in the recent past.

This time, however, it seems to have gone so far as to pollute the calculation of the time-dependent correction factors, which are applied to the astrophysical output -- namely the relative ESD/TST stage actuation strength change ("kappa_TST"), and the relative optical gain change ("kappa_C"). Since the data were so noisy around these lines (the SNR dropped from the typical ~500 to less than 10), this dropped the coherence between excitation and DARM, and increased the uncertainty beyond our pre-defined thresholds for good data.

As such I recommend this data stretch to be flagged as garbage by DetChar, from where bits 13 & 15 ("kappa_TST median" and "kappa_PU median") of the H1:GDS-CALIB_STATE_VECTOR are red during this observational stretch.

I attach good deal of plots to help support the interrelated case between
    - The DELTAL_EXT sensitivity of the detector, zoomed in around both the 2150 Hz OMC dither lines and CAL lines 
    - OMC Dither Alignment Control (whose error signals are informed by 4, evenly spaced lines between 2125 and 2200 Hz)
    - OMC SUS and OM3 alignment
    - DHARD Pitch Control
    - The noisy kappa_TST and kappa_C 
and then other plots that show it's NOT causally related to 
    - Earthquakes (the recent EQ at 4:30 UTC was done by then)
    - ETMY Optical Lever Excursions (ETMY's optical lever went off into the weeds, but what's shown in the BLRMS is just the ring-down of the low-pass filter)
    - 1-30 Hz GND Motion (the increase happens *after* this alignment excursion)
    - Large differential motion of the ETMs (ETMX X and ETMY Y show that same ISI ST1 performance at 11:30 UTC and 03:00 UTC which are bad and good times, repectively)

I don't have any speculation as to which of the inter-related symptoms are the actual cause of the problem, and/or I don't have a clue as to why the OMC dither line heights are so much louder during the bad time. I'll ask around for speculation from other members of the commissioning team.

TITLE: 03/29 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
    Wind: 11mph Gusts, 9mph 5min avg
    Primary useism: 0.21 μm/s
    Secondary useism: 0.38 μm/s
QUICK SUMMARY:

Site Activities:

• Richard and Fil are driving down X arm looking in the beam tube for equipment
• tumbleweed balers are working on X arm, started about 1/2 way between Mid-X and EX, and are working their way toward the corner station

H1 Activities:

• H1 was locked and heading to NLN when I arrived
• it made it to NLN, and then had a lock loss due to an earthquake
• Hugh is working on ETMY BRS - preliminary go ahead to use if needed, but he's also continuing to watch it's recovery
• current H1 work:
  • Kiwamu has H1 for commissioning
  • ETMs need to remain mis-aligned during his work

After yesterday's abuse, the BRSY was just not ready to contribute to the cause of science.  After having the thermal enclosure breached for an hour or something, a big jolt from something when the remote desktop session was terminated and at least one...or was it two restarts of the code, I thought things had finally settled down but it was not so.

See the attached plot the first.  Note the trend up and noisy steps in DRIFTMON.  This channel should be smooth and quiet as the low frequency beam position.  When the signal steps notice the RX (tilt) and VEL signals also going noisy and haywire. When I left the BRS last evening, the DRIFTMON was running smoothly (from about 0100 to 0200 utc.)  But then it started the step behavior again and remained unusable until restarting this morning.

Also attached. the second, is two months of the DRIFTMON and an internal temperature signal.  Obviously yesterday's incursion was a good cool down and the trend down indicates we better do another beam centering within a week or two at the longest.

The last plot is the most recent 2 hours and it appears that the periodic step and noise behavior has stopped.  The large step down in the REF signal maybe indicates that Krishna's suspicion was correct in that the reference image captured by the camera was bad.  The direction of the DRIFTMON suggests the BRS is still warming; I'll keep a close eye on this today to watch for errant tendencies.

Summary: Injections suggest that vibration of the input beam tube in the 8-18 Hz band strongly couples to DARM and is the dominant source of noise in the 70-200 Hz band of DARM for transient truck and fire pump signals, and likely also for the continuous signal from the HVAC. Identification of the coupling site is based on the observation that local shaking of the input beam tube produces noise levels in DARM similar to those produced by global corner station vibrations from the fire pump and other sources, for similar RMS at an accelerometer under the Swiss cheese baffle by HAM2. The local shaker injections were insignificant on nearby accelerometers or HEPI L4Cs, and HEPI excitations cannot account for the noise, supporting a local, off-table coupling site in the IMC beam tube.  In addition, local vibration injections occasionally produce a 12 Hz broad-band comb, which is also produced by trucks and the fire pump, possibly indicating a 12 Hz baffle resonance.  While the Swiss cheese baffle seems the most likely coupling site, we have not yet eliminated the eye baffle by HAM3.

Several recent observations have suggested that we are limited by noise in the 100 Hz region that is produced by vibrations in the 10-30 Hz region.  There was the observation that our range increased by a couple of Mpc when the HVAC was shut down, ( https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=32886 ), and, additionally, the observations of noise from the fire pump and from trucks.

I noticed that the strongest signals in DARM produced by the fire pump and trucks had peaks that were harmonics of about 12 Hz. I injected manually all around the LVEA and found that the input beam tube was the one place where I could produce a 12 Hz comb in the 100 Hz region (injections were sub-50 Hz). Figure 1 shows that a truck, the fire pump, and my manual injections at the input beam tube produced similar upconverted noise in DARM.

I also used an electromagnetic shaker on the input beam tube.  Figure 2 is a spectrogram showing a slow shaker sweep and the strong coupling in the 8-18 Hz band. I wasn’t able to reproduce the  broad 12 Hz comb with the shaker, possibly because, as mounted, it didn’t couple well to the 12 Hz mode. But the broad-band noise produced in DARM by the shaker is more typical of trucks and the fire pump: only occasionally does the 12 Hz comb appear.  One possibility is that the bounce mode of the baffle is about 12 Hz.

Figure 3 shows that, for equivalent noise in DARM, the RMS displacement from the shaker and the fire pump were about the same at an accelerometer mounted under the Swiss cheese baffle by HAM2. Figure 4 shows that the shaker vibration is local to the beam tube. While the shaker signal is large on the beam tube accelerometer, it is almost lost in the background at the HAM2 and 3 accelerometers and the HAM2, 3 L4Cs.  Finally, the failure to reproduce the noise with HEPI injections, both during PEM injections at the beginning of the run and a recent round by Sheila, further support the off-table source of the noise.

While everything is consistent with coupling at the Swiss cheese baffle near HAM2, we haven’t eliminated the eye baffle near HAM3. This might be done by comparing a second accelerometer under the eye baffle to the one under the Swiss cheese baffle, but I didn’t have another spare PEM channel.

If it is the Swiss cheese baffle, it might be worth laying it down during a vent. Two concerns are the blocking of any beams that are dumped on the baffle, and the shiny reducing flange at the end of the input beamt ube that would be exposed.

An immediate mitigation option is to try and move beams relative to the Swiss cheese baffle while monitoring the noise from an injection. Sheila and I started this but ran out of commissioning time and LLO was up for most of the weekend so I didn’t get back to it.  If someone else wants to try this, either turn on the fire pump or, for even more noise in DARM, the shaker by HAM3 (the cable goes across the floor to the driver by the wall, enter 17 Hz on the signal generator and turn on the amp, it should still be set).

Shaker injections have shown the input beamtube to be sensitive for some time ( https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=31016). During pre-run PEM injections, an 8 to 100 Hz broad-band shaker injection on the input beamtube showed strong coupling. However the broad-band injection was smaller in the sensitive 10-18 Hz band then in other sub-bands and so the magnitude of the up-converted coupling from this narrow sub-band was not evident. When we have detected upconversion during PEM injections in the past, we have narrowed down the sensitive frequency band with a shaker sweep, but, for the input beam tube, we didn’t get to this until last week.

Figure 5 shows fire pump, trucks and the HVAC on input beam tube accelerometers and DARM.

Sheila, Anamaria, Robert

It appears that some maintenance work on Tuesday March 14 led to degradation
with respect to narrow lines in H1 DARM. The attached inverse-noise-weight spectra 
compare 1210 hours of data from the start of O2 until early morning of March 14 with 
242 hours since that time.

Summary of substantial comb changes:

The comb with 0.9999-Hz spacing nearly aligned with N + 0.25 Hz (N = 15-55) is stronger.

There is a new comb with 0.999984-Hz spacing visible nearly aligned with N + 0.75 Hz (N = 41-71)

There is a new comb with 1.0000-Hz spacing visible at N - 0.0006 Hz (N = 104-125)

Much activity was reported in the alog for March 14, but what jumps out at me
are two references to HWS work: here and here. 

Are there HWS cameras running during observing mode? 

I had thought those things were verboten in observing mode, given their
propensity to make combs.

Fig 1: 20-50 Hz comparison (before and after March 14 maintenance)
Fig 2: 50-100 Hz (before and after March 14 maintenance)
Fig 3: 100-150 Hz  (before and after March 14 maintenance)

Attachment has full set of comparison sub-bands up to 2000 Hz with both A-B and B-A orderings to make clear which lines are truly new or louder.

In brief, I found some significant correlation between (a class of) blip glitches and the values of some suspension signals coming from the OSEM and local sensor readout.

I found that some of the blip glitches tend to cluster around particular values of the signals. In particular the signals listed below are the most significant ones:

• ETMX_M0_DAMP_L_INMON, ETMX_M0_DAMP_R_INMON, ETMX_M0_DAMP_V_INMON, ETMX_M0_DAMP_Y_INMON,
• ETMY_M0_DAMP_P_INMON, ETMY_M0_DAMP_T_INMON,
• ITMX_M0_DAMP_P_INMON, ITMX_M0_DAMP_R_INMON, ITMX_M0_DAMP_V_INMON,
• ITMY_M0_DAMP_P_INMON, ITMY_M0_DAMP_R_INMON, ITMY_M0_DAMP_V_INMON
• ETMX_L1_WIT_YMON,
• ETMY_L1_WIT_PMON, ETMY_L1_WIT_YMON,
• ITMX_L1_WIT_LMON,
• ITMY_L1_WIT_LMON, ITMY_L1_WIT_PMON
• ITMX_L2_WIT_PMON,
• ITMY_L2_WIT_YMON

Read below for more details on the methodology and the results.

Method

• First of all I downloaded a list of identified blip glitches from the GravitySpy page (thank you TJ for pointing me to this). In O2, at Hanford, there are in total 5548 glitches that are identified as blips. Most of them are identified with high confidence (>99%). The time spans between 1164564597 (Nov 30 2016 18:09:40 UTC) and 1171671859 (Feb 21 2017 00:24:01 UTC).
• I downloaded 60 seconds of data (using the 16 Hz monitoring channels) around each blip glitch (-30s to +30s) for all the test mass SUS-XXX_M0_DAMP_*_INMON, SUS-XXX_L1_WIT_*MON, SUS-XXX_L2_WIT_*MON, SUS-XXX_L3_OPLEV_*_OUT16 signals. I have 56 channels in total. 
• Even though I have 60 seconds of data, I just extracted the value of each signal at the time of each blip glitch. I also computed the first and second derivative of each signal at the same times.
• Then I randomly selected 5548 more times, uniformly distributed in the same range covered by the blip glitches. I ensured that the times I selected were in ANALYSIS_READY, not vetoed by any known flag (thanks again TJ for sharing the code to read the veto definer file) and at least 60 seconds from any known blip glitch. Those 5548 times can be considered as clean, glitch free times.
• For those clean times, I downloaded and sampled the same channels I used for the blip glitches.

At this point I have 5548 times identified as blip glitches, and the value of the suspension channels for each of those. And 5548 more times that are clean data, with the value of the suspension channels for each of those. 

• For each of the suspension signal, I computed the histogram of its values at the time of blip glitches, and another histogram at the clean times. By comparing the two histograms I can tell if blip glitches tend to happen for some particular values of the signals.

Results

Here's an example plot of the result. I picked one of the most interesting channel (SUS-ETMX_M0_DAMP_Y_INMON). The PDF files attached below contains the same kind of histograms for all channels, divided by test mass.

The first row shows results for the SUS-ETMX_M0_DAMP_Y_INMON signal. The first panel compares the histogram of the values of this signal for blip times (red) and clean times (blue). This histogram is normalized such that the value of the curve is the empirical probability distribution of having a particular value of the signal when a blip happens (o doesn't happen). The second panel in the first row is the cumulative probability distribution (the integral of the histogram): the value at an abscissa x gives the probability of having a value of the signal lower than x. It is a standard way to smooth out the histogram, and it is often used as a test for the equality of two empirical distributions (the Kolmogorov-Smirnov test). The third panel is the ratio of the histogram of glitchy times over the histogram of clean times: if the two distributions are equal, it should be one. The shaded region is the 95% confidence interval, computed assuming that the number of counts in each bin of the histogram is a naive Poisson distribution. This is probably not a good assumption, but the best I could come up with.

The second row is the same, but this time I'm considering the derivative of the signal. The third row is for the second derivative. I don't see much of interest in the derivative plots of any signal. Probably it's due to the high frequency content of those signals, I should try to apply a low pass filter. On my to-do list.

However, looking a the histogram comparison on the very first panel, it's clear that the two distributions are different: a subset of blip glitches happen when the signal SUS-ETMX_M0_DAMP_Y_INMON has a value of about 15.5. There are almost no counts of clean data for this value. I think this is a quite strong correlation.

You can look at all the signals in the PDF files. Here's a summary of the most relevant findings. I'm listing all the signals that show a significant peak of blip glitches, as above, and the corresponding value:

Some of the peaks listed above are quite narrow, some are wider. It looks like ITMY is the suspension with more peaks, and probably the most significant correlations. But that's not very conclusive.

To do next

• My guess is that all the peak above are coming from the same family of glitches. But this has to be checked. I can select a subset of blip glitches by setting some cuts in the signal values. And I can check if I get the same subset from all signals
• Once I have a family of glitches (for example those that correspond to SUS-ETMX_M0_DAMP_Y_INMON ~ 15.5 above), it's interesting to check if they are randomly distributed over the duration of the run, or if they cluster around a specific time
• Also, I have to try and see if that family of blip glitches can be characterized by some properties (like bandwidth, central frequency, duration, etc...)
• And there are many more channels to check. My initial motivation for looking at suspension signals was to check for crackling glitches: stress releases in the suspension blades, wires or fibers.
• Just for fun I trained a simple (shallow) neural network to try to distinguish glitches from clean times. Results were not good enough to be reported here, mainly because it's quite cleat than only a subset of blip glitches are correlated with suspension signals

J. Kissel, S. Aston, P. Fritschel

Peter was browsing through the list of frame channels and noticed that there are some differences between H1 and L1 on PR2 (an HSTS), even after we've both gone through and made the effort to revamp our channel list -- see Integration Issue 6463, ECR E1600316, LHO aLOG 30844, and LLO aLOG 29091.

What difference he found is the result of the LHO-ONLY ECR E1400369 to increase the drive strength of the lower states *some* of the HSTSs. This requires the two sites to have a different front-end model library part for different types of the same suspension because the BIO control each stage is different depending on the number of drivers that have been modified;
At LHO the configuration is
    Library Part        Driver Configuration            Optics
    HSTS_MASTER.mdl     No modified TACQ Drivers        MC1, MC3
    MC_MASTER.mdl       M2 modified, M3 not modified    MC2
    RC_MASTER.mdl       M2 and M3 modified              PRM, PR2, SRM, SR2

At LLO, the configuration is
    Library Part        Driver Configuration            Optics
    HSTS_MASTER.mdl     No modified TACQ Drivers        MC1, MC3, PR2
    MC_MASTER.mdl       M2 modified, M3 not modified    MC2, PRM, SR2, SRM
    RC_MASTER.mdl       M2 and M3 modified       none

This model's DAQ channel list for the MC and RC masters is the same. The HSTS master is different, and slower, because these SUS are used for angular control only: 
                           HSTS (Hz)        MC or RC (Hz)
    M3_ISCINF_L_IN1        2048               16384

    M3_MASTER_OUT_UL       2048               16384
    M3_MASTER_OUT_LL       2048               16384
    M3_MASTER_OUT_UR       2048               16384
    M3_MASTER_OUT_LR       2048               16384

    M3_DRIVEALIGN_L_OUT    2048               4096

Since LLO's PR2 does not have any modifications to its TACQ drivers, it uses the HSTS_MASTER model, which means that PR2 alone is going to show up as a difference in the channel list between the sites that seemed odd Peter -- that L1 had 6 more 2048 Hz channels than H1. Sadly, it *is* used for longitudinal control, so LLO suffers the lack of stored frame rate.

In order to "fix" this difference, we'd have to create a new library part for LLO's PR2 alone that has the DAC channel list of an MC or RC master, but have the BIO control logic of an HSTS master (i.e. to operate M2 and M3 stages with an unmodified TACQ driver). That seems excessive given that we already have 3 different models due to differing site preferences (and maybe range needs), so I propose we leave things as is, unless there's dire need to compare the high frequency drive signals to the M3 stage of PR2 at LLO.

I attach a screenshot that compares the DAQ channel lists for the three library parts, and the two types of control needs as defined by T1600432.