10:51 Bubba moving HEPI and forklift on trailer in through gate coming from LSB. HEPI is in back of pickup, forklift is on trailer. Heading to mid X.
10:55 Bubba is a little more than half way to mid X.
10:59 Bubba made turn into mid X
10:59:29 Lock loss
11:13 Bubba at mid X, starting opening door and moving HEPI in
11:49 Bubba driving back
12:12 Bubba done

Winds have all died down.

H1 has stabilized, but we had a 9Mpc step down starting at ~11:30utc & continue to stay at this range. 

As of 10:00utc, going on ~4hrs of double coincidence.  Winds have died down to under 10mph locally.

While going through the Ops Checksheet, had a few notes:

• For the Guardian check, I notice that we have a "BRS has rung up" message.  Since we are not using BRS for Sensor Correction ("it's off"), I'm not going to worry about it.
• For checking CDS Front Ends, we have lots of green up there, but I see that we have a purple H1ASC excitation...what's this?
• Dust Monitors:  LAB has (2) with Major alarms & no dust counts (think this is normal), EX has INVALID boxes on it.

It turns out that the problem we have been having today, which seems like a 1-5 Hz alignment problem, is actually caused by the verry low frequency offloading of PRM and SRM from M2 to M1.  The length feedbacks themselves are stable, and very slow, but there must be some interaction with the alignment that means that one or both of these make the IFO unstable.  

We're not going to try turning them on again.  We are turning the SR3 OpLev damping back on, since that seemed to solve the lockloss problems encountered on Travis' tuesday owl shift (alog 18777).  

First, some details, the injection is actually a sine-Gaussian with parameters:

SineGaussian t0 = 1116964989.435322284 q = 28.4183770634 f0 = 1134.57994534

The t0 can be safely ignored since this injection would have no counterpart in L1, but it should be noted that this injection *does* have all relevant antenna patterns and polarization factors applied to it (e.g. it is made to look like a "real" GW signal).

I attach the time and frequency domain plots of the waveform --- however they are relative to an O1 type spectrum and so may not be indicative of the actual performance of the instrument at this period of time. Given the most recent spectra and frequency content of the injection, this could be weaker up to a factor of ~2-3. The characteristic SNRs I calculated using the O1 type spectrum:

Waveform SineGaussian at 1116964989.435 has SNR in H1 of 7.673043
Waveform SineGaussian at 1116964989.435 has SNR in L1 of 20.470634
Network SNR for SineGaussian at 1116964989.435 is 21.861438

So, it's possible that this injection had an SNR as low as ~2, not accounting for variance from the noise.

The excitation channel (H1:CAL-INJ_TRANSIENT_EXCMON, trended) does show a non-zero value, and the "count" value is consistent with the amplitude of the strain, another monitor (H1:CAL-INJ_HARDWARE_OUT_DQ, at a higher sample rate) also shows the full injection, though it is not calibrated. So, the injection was successfully scheduled and looks to have been made. I also did an omega scan of the latter channel, and the signal is at the correct frequency (but has, notably, a very long duration).

I did a little poking around to see if this showed up in h(t) (using H1:GDS-CALIB_STRAIN). Unfortunately, it is not visible in the spectrogram of H1:GDS-CALIB_STRAIN (attached). It may be some peculiarity of the scheduling, but it's interesting to note that the non-zero excitation occurs about a second after the GPS time that Eric quotes. More interestingly, this does not seem to have fired off the proper bits in the state vector. H1:GDS-CALIB_STATE_VECTOR reports the value 456 for this period, which corresponds to the data being okay, gamma being okay, but no injection taking place. it also appears to mean that no calibration was taking place (bits 3 and 4 are off). I'm guessing I'm just misinterpreting the meaning of this.

I'd recommend, for future testing, a scale factor of 3 or 4, to make it *clearly* visible and give us a point of reference. We should also close the loop with the ODC / calibration folks to see if something was missed.

I can see that burst injections have been occurring on schedule, generally.  The schedule file (which you currently have to log into h1hwinj1 to view) reads, in part:
...
1117411713 1 1 burst_test_
1117419952 1 1 burst_test_
1117427152 1 1 burst_test_
...
Compare that to the bit transitions in CAL-INJ_ODC:
pshawhan@> ./FrBitmaskTransitions -c H1:CAL-INJ_ODC_CHANNEL_OUT_DQ /archive/frames/ER7/raw/H1/H-H1_R-11174/*.gwf -m fffffff
1117400000.000000  0x00003f9e  Data starts
1117400027.625000  0x00003fde  6 on
1117400028.621093  0x00003fff  0 on, 5 on
1117406895.000000  0x00003e7f  7 off, 8 off
1117411714.394531  0x0000347f  9 off, 11 off
1117411714.480468  0x00003e7f  9 on, 11 on
1117419953.394531  0x0000347f  9 off, 11 off
1117419953.480468  0x00003e7f  9 on, 11 on
1117427153.394531  0x0000347f  9 off, 11 off
1117427153.480468  0x00003e7f  9 on, 11 on
...
Bits 9 and 11 go ON when a burst injection begins, and off when it ends. The offset of start time is because the waveform file initially contains zeroes. To be specific, the first 22507 samples in the waveform file (=1.37372 sec) are all exactly zero; then the next several samples are vanishingly small, e.g. 1e-74 .  At t=1.39453 sec = 22848 samples into the waveform file, the strain amplitude of the waveform is about 1e-42.  At the end time of the injection (according to the bitmask transition), the strain amplitude has dropped to about 1e-53, but I believe the filtering extends the waveform some.  To give an idea of how much, the end time is about 130 samples, or ~8 msec, after the strain amplitude drops below 1e-42.

Earlier in the record, bit 6 went on to indicate that the transient filter gain was OK, bit 5 went on to indicate that the transient filter state was OK, and bit 0 went on at the same time to indicate a good summary.  Somewhat later, bit 8 went off when CW injections began, and bit 7 went off at the same time to indicate the presence of any hardware injection signal.

Note that tinj seems to have been stopped (or died) at about GPS 1117456410, according to the tinj.log file, and that's consistent with a lack of bit transitions in CAL-INJ_ODC after that time.

Checking the other bitmask channels, there's a curious pattern in how the hardware injection bits from CAL-INJ_ODC are getting summarized in ODC-MASTER:
pshawhan@> ./FrBitmaskTransitions -c H1:ODC-MASTER_CHANNEL_OUT_DQ -m 7000000 /archive/frames/ER7/raw/H1/H-H1_R-11174/H-H1_R-11174[12345]*.gwf
1117410048.000000  0x07000000  Data starts
1117411714.391540  0x05000000  25 off
1117411714.391723  0x07000000  25 on
1117411714.391845  0x05000000  25 off
1117411714.475463  0x07000000  25 on
1117419953.391540  0x05000000  25 off
1117419953.391723  0x07000000  25 on
1117419953.391845  0x05000000  25 off
1117419953.475463  0x07000000  25 on
1117427153.391540  0x05000000  25 off
1117427153.391723  0x07000000  25 on
1117427153.391845  0x05000000  25 off
1117427153.475463  0x07000000  25 on
...
The same pattern continues for all 7 of the injections.  We can see that there's a brief interval (0.000183 s = 3 samples at 16384 Hz) which is marked as a burst injection, then "no injection" for 2 samples, then back on for 0.083618 s = 1370 samples.  Knowing how the sine-Gaussian ramps up from vanishingly small amplitude, I think this is the real in the sense that the first whisper of a nonzero cycle returns to effectively zero for a couple of samples before it grows enough to be consistently "on". It is also interesting to see that the interval ends in ODC-MASTER slightly (5.0 ms) earlier than it does in CAL-INJ_ODC.  I suspect that this is OK and the model really runs at 16384 Hz, but CAL-INJ_ODC is a down-sampled record of the real bit activity.

I also confirmed that the injection bit got carried over to CALIB-STATE_VECTOR:
pshawhan@> ./FrBitmaskTransitions -c H1:GDS-CALIB_STATE_VECTOR -m 1ff /archive/frames/ER7/hoft/H1/H-H1_HOFT_C00-11174/H-H1_HOFT_C00-11174[012]*.gwf 
1117401088.000000  0x000001c8  Data starts
1117408113.375000  0x000001cc  2 on
1117408119.375000  0x000001dd  0 on, 4 on
1117408143.750000  0x000001df  1 on
1117408494.812500  0x000001dd  1 off
1117408494.937500  0x000001c8  0 off, 2 off, 4 off
1117411714.375000  0x00000148  7 off
1117411714.500000  0x000001c8  7 on
1117419953.375000  0x00000148  7 off
1117419953.500000  0x000001c8  7 on
1117420916.625000  0x000001cc  2 on
1117420922.625000  0x000001dd  0 on, 4 on
1117421174.000000  0x000001df  1 on
1117424707.062500  0x000001dd  1 off
1117424954.437500  0x000001c8  0 off, 2 off, 4 off
1117426693.000000  0x000001cc  2 on
1117426699.000000  0x000001dd  0 on, 4 on
1117426833.625000  0x000001df  1 on
1117427153.375000  0x0000015f  7 off
1117427153.500000  0x000001df  7 on
...
Bit 7 indicates burst injections.  It comes on for 2 16-Hz samples at the appropriate times.

As of 7:00am PDT on Friday, June 5, there have been 13 burst hardware injections at LHO over the last two days.  All of these are represented by segments (each 1 second long) in the DQ segment database, and can be retrieved using a command like:
pshawhan@> ligolw_segment_query_dqsegdb --segment-url https://dqsegdb5.phy.syr.edu --query-segments --include-segments H1:ODC-INJECTION_BURST --gps-start-time 1117300000 --gps-end-time 'lalapps_tconvert now' | ligolw_print -t segment -c start_time -c end_time -d ' '
These time intervals also agree with the bits in the H1:GDS-CALIB_STATE_VECTOR channel in the H1_HOFT_C00 frame data on the Caltech cluster, except that there is a gap in the h(t) frame data (due to filters being updated and the h(t) process being restarted, as noted in an email from Maddie).

Similar DB queries show no H1 CBC injection segments yet, but H1 CW injections are ongoing:
pshawhan@> ligolw_segment_query_dqsegdb --segment-url https://dqsegdb5.phy.syr.edu --query-segments --include-segments H1:ODC-INJECTION_CW --gps-start-time 1117400000 --gps-end-time 'lalapps_tconvert now' | ligolw_print -t segment -c start_time -c end_time -d ' '
1117406895 1117552800
By the way, by repeating that query I observed that the CW segment span was extended by 304 seconds about 130 segments after the segment ended.  Therefore, the latency of obtaining this information from the segment database ranges from 130 to 434 seconds, depending on when you query it.  (At least under current conditions.)

I also did similar checks at LLO, which revealed a bug in tinj -- see https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=18517 .

I figured out what was going wrong with the demod measurement - not enough low-passing before the decimation step, the violin modes at ~510Hz were beating against the 256Hz sample rate.  With another layer of anti-aliasing the demod results are in very good agreement with the RMS channel:

RMS channel: 594,400

Demodulated DARM_ERR: 593,800

To see what we might expect, I took the current GWINC model of suspension thermal noise and did the following.
1) Removed the horizontal thermal noise so I was only plotting vertical.
2) Updated the maraging steel phi to reflect recent  measurement (LLO alog 16740) of Q of UIM blade internal mode of 4 x 10^4. (It is phi of 10^-4, Q 10^4 in the current GWINC). I did this to give better estimate of the vertical noise from higher up the chain.
3) Plotted only around the thermal noise peak and used 1 million points to be sure I resolved it.

Resulting curve is attached. Q looks approx 100K, which is less than what was reported in this log. That is encouraging to me. I know the GWINC model is not quite right - it doesn't reflect tapered shape and FEA results.  However to see a Q in excess of what we predicted in that model is definitely in the right direction.

Here we take the Mathematica model with the parameter set 20150211TMproduction and we look at varying some of the loss parameters to see how the model compares with these measurements. The thermal noise amplitude in the vertical for the vertical bounce mode is tabularised around the resonance and we take the full width at 1/√2 height to calculate the Q (equivalent to ½ height for power spectrum). With the recently measured mechanical loss value for maranging steel blade springs of 2.4 e-5, the Mathematica model predicts a Q of 430,000. This is a little bit lower Q than the measurement here, but at this level the loss of the wires and the silica is starting to have an effect, and so small differences between the model and reality could show up. Turning off the loss in the blade springs altogether only takes the Q to 550,000, so other losses are sharing equally in this regime. The attached Matlab figures shows mechanical loss factor of maraging steel versus predicted bounce mode Q and against total loss plus the resonance as a function of loss.
Angus Giles Ken & Borja

Since there has been some modeling afoot, I wanted to post the statistical error from the fits above, to give a sense of the [statistical] precision on these measurements.  The best-fit Q value and the 67% confidence interval on the two measurements for the bounce mode are:

RMS channel: 594,410  +/-  26

Demodulated DARM_ERR: 594,375  +/-  1590

The data for the measurements are attached.  Note that this is just the statistical error of the fit -- I am not sure what systematics are present that could bias the measurement in one direction or another.  For example, we did not disable the top-stage local damping on ITMX during this measurement, only the DARM_CTRL --> M0 damping that is bandpassed around the bounce mode.  There is also optical lever feedback to L2 in pitch, and ASC feedback to L2 in pitch and yaw from the TRX QPDs (although this is very low bandwidth).  In principle this feedback could act to increase or decrease the observed Q of the mode, although the drive at the bounce mode frequency is probably very small.

I'm pretty sure these come from PRM, but one stage higher than you looked, M2. Attached is a lineup of PRM M2 UR and the glitches in CAL DELTAL. Looks like 2^16 = 65536 counts DAC glitches to me. I'll check some other times and channels, but wanted to report it now. 

After a lot of followup by many detchar people (TJ, Laura, Duncan, and Josh from a plane over the Atlantic), we haven't really been able to make DAC glitches work as an explanation for these glitches. A number of channels (especially PRM M2 and M3) start crossing +/- 2^16 around 7 UTC, when the glitches begin. Some of these glitches line up with the crossings, but there are plenty of glitches that go unexplained, and plenty of crossings that don't correspond to a glitch. It's possible that DAC glitches are part but not all of the explanation. We'll be following this up further since one of these glitches corresponds to an outlier in the burst search.

Duncan does report that no software saturations (channels hitting their software limit, as with the tidal drive yesterday) were found during this time, so we can rule those out.

Andy, Laura, TJ, Duncan, Josh: To add to what Andy said, here are a few more plots on the subject, 

1) The glitching being bad does coincide with SRM and PRM channels being close to 2^16 (plotted only positive values here, negative values are similar). Of course this is pretty weak evidence as lots of things drift.

2) A histogram of the number of glitches versus the DAC value of the PRM M2 UR channel has a small line at 2^16. Almost not significant. Statistically, with the hveto algorithm, we find only a weak correlation with +/-2^16 crossings in the PRM M2 suspensions. Again, all very weak.  

3) As you all reported, the glitches are really strong in SRCL and PRCL, ASC REFL channels, etc. hveto would veto ~60% of yesterday's glitches using SRCL and PRCL, as shown in this time-frequency plot. But the rest of the glitches would still get through and hurt the searches. 

So we haven't found the right suspension, or there is something more complicated going on. Sorry we don't have a smoking gun - we'll keep looking. 

I grabbed Andy's images and lined them up in keynote. Thought folks might want to see how convincing this is for DAC zero crossing glitches. 

Jim, Dave

we checked to see if the 18bit DAC card for SUS MC2 M3 happened to be close to the new DC power supply. It is not, in fact it is the furthest from the power supply.

At least twice tonight there has been glitching of MC2, similar to what Kiwamu described last week. It is visible in POP18 and AS90, as well as the MC2 witness sensors. I was not looking at the IMC REFL camera at the time, so I can't say whether it was the same kind of kick in yaw as before.