Report on DetChar DQ glitch shift for Monday Nov 28 - Wednesday Nov 30

• Duty cycle was low for 2 days leading up to O2 (29.2% Monday, 6.4% Tuesday), but recovered for first day of O1 (42.9% Wednesday)

• Inspiral range in the 70 Mpc ballpark, drooping to 60 Mpc occasionally

Looked at 3 major sources of glitches during this shift:

• Loud 20-200 Hz glitches believed to be caused by Optical Lever laser glitching, and seem to be improved by adjusting OpLev laser power

• Strong line of 1080 Hz glitches, reasonable suspicion they're coming from the PSL (often saw definite coincidence with ~270 Hz glitches in IMC-WFS_A_DC_SUM_OUT_DQ as well as what I thought was quite telling, PSL-ISS_PDA_REL_OUT_DQ - way upstream of DARM if I recall correctly), but replacing a piezo mount didn't seem to help and turning off PCal injection line didn't help

• Relatively new lines of glitches at 2 kHz and 3 kHz that were quite evil, but we're pretty sure are caused by rung-up violin modes (and can thus be gotten rid of by spending a little time damping).

Full report can be found at https://wiki.ligo.org/DetChar/DataQuality/DQShiftLHO20161128

WP6366 Increase memory in Guardian machine

Dave, TJ, Carlos:

at 08:47PDT h1guardian0 was powered back up after having its RAM size increased from 12GB to 48GB. TJ reports all nodes are operational.

Preliminary results from MCMC fitting of the sensing function measurements taken on Nov 12 to a 5 parameter model are given below:

  gain    : 1.160e+06     (nonlinear fit was 1.153e6, 0.6% different from MCMC)
  f_c     : 342.3340 Hz   (... 346.7 Hz, difference is 1.3%)
  delay   : -8.809e-08 s  (... -2e us, this value from nonlinar fit was ignored,
                               uncertainty on the value from the nonlinear fit was +/- 3.3 us)
  f_s     : 8.19 Hz       (... 7.389 Hz, difference is 9.8%)
  1/Q     : 0.0474        (... 0.0454, difference is 4.2%)

Values of the nonlinear fit were reported in LHO alog 31665.

The fitted sensing function and the corner plots are attached.

These are preliminary results, the uncertainties on the parameters have not been analyzed at the moment.

TITLE: 12/01 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Cheryl
CURRENT ENVIRONMENT:
    Wind: 7mph Gusts, 5mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.42 μm/s
QUICK SUMMARY:

Received status about how the night/morning went from Cheryl.  

16:35:  Since we are not LOCKED, performing memory upgrade for h1guardian0 (WP6366, via Carlos).

Will get back to locking afterward and investigate around DC Readout and wait for assistance.

State of H1: losing lock at LOW_NOISE_ESD

Activities:

• started shift with H1 unlocked
• some tweaks to alignment and locked up
• broke lock after LOW_NOISE_ASC multiple times
• violin modes were rung up
• relocked and worked on damping violins
• made some headway with ETMX mode4, see alog 32059
• tried to get to NLN and lock loss at LOWNOISE_ESD_ETMY both times
• handing off to Corey

• After numerous lock losses that came with many saturations, I relocked and I spent some time looking at the violin modes since all were very rung up.
• It was near impossible to damp ETMX mode4, so I turned it off and focused on modes 2 and 3, with some success.
• I returned to damping mode4, and after turning off a filter in mode2, I could damp mode4.
• When I turned off mode2 FM5, N505.710, immediately the mode2's signal decreased in amplituded, and mode4 started to decrease.

ETMX changes:

ETMY changes:

ITMX changes:

Bounce Mode Damping: I increased the gains

After relocking, the ETMX changes didn't work exactly as they did in my lock - picture attached.

• Attachment 1: ETMX mode 4 damping
• Attachment 2: same settings on relocking

TITLE: 12/01 Owl Shift: 08:00-16:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Jim
CURRENT ENVIRONMENT:
    Wind: 6mph Gusts, 4mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.47 μm/s
QUICK SUMMARY:

• after handoff I had to "fix" the fast shutter
• the "fix" was not overly obvious, test failed, init -> test failed, auto mode -> init -> test failed
• I tried something else
• I ran the ISC_LOCK down script with Down as the chosen state, and when that was done, the fast shutter was ok again
• alignment's great, and H1 is in ENGAGE_SRC_ASC

TITLE: 12/01 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY: Having troubles relocking, seems 4.7khz violins are acting up
LOG:

IFO was locked when I got here. Jeff was working on tamping down second harmonic violins. Eventually got that taken care of got to observing. After an hour (after everyone else went home) started getting ETMY saturations. Guess I didn't notice the 4.7khz modes ringing up. Haven't been able to relock since, nothing I've tried has gotten these violins to quit playing. ISC_LOCK also keeps failing to go to and come out of down, I've had to select down after each lockloss and then manually go ready. Don't know if that is a problem.
 

Just had a lockloss. It started with a long string of continual EY saturations, some broad "low frequency" noise (roughly 10-50 hz) appeared in the live DARM spectra, then a lump appear around 50 hz. This eventually became a sharp peak at about 47 hz that would get large for a few seconds, then settle down a little. Attached spectra show a measurement just before the lock loss (red) with the 47 hz feature, a measurement with the low frequency stuff before the 47hz peak came up (brown), and a measurement 2 hours ago (green). 47 hz sounds familiar, but I know not from whence.

J. Kissel, E. Merilh, J. Warner, T. Hardwick, J. Driggers, S. Dwyer

Prompted by DetChar worries about glitching around the harmonics of violin modes, Ed, Jim, and I went on an epic campaign to damp the ~1kHz, 2nd harmonic violin modes. These are tricky because not all modes had been successfully damped before, and one has to alternate filters in two filter banks to hit all 8 modes for a given suspension. 

We've either used, or updated Nutsinee's violin mode table, with the notable newly damped entries being 
994.8973    ITMY     -60deg, +gain      MODE9: FM2 (-60deg), FM4 (100dB), FM9 (994.87) 
997.7169    ITMY       0deg, -400gain   MODE9: FM4 (100dB), FM6(997.717)                 VERY Slow
997.8868    ITMY       0deg, -200gain   MODE10: FM4 (100dB), FM6(997.89) 

Also, we inadvertently rung up modes around 4735 Hz, so we spent a LONG time trying to fight that. We eventually won by temporarily turning on the 4735Hz notch in FM3 of the LSC-DARM2 filter bank and waiting a few hours. I had successfully damped the ETMY mode at 4735.09 Hz by moving the band-pass filter in H1:SUS-ETMY_L2_DAMP_MODE9 's FM10 from centered around 4735.5 to centered around 4735 Hz exactly, and using positive gain with zero phase. However, there still remains a mode rung up at 4735.4 Hz but it's from an as-of-yet unidentified test mass, and we didn't want to spend the time exploring. These 4.7 kHz lines have only appeared once before in late October (LHO aLOG 31020).

Attched is a before vs. after ASD of DELTAL_EXTERNAL. I question the calibration, but what's important is the difference between the two traces. Pretty much all modes in this frequency band have been reduced by 2 or 3 orders of magnitude -- better than O1 levels. Hopefully these stick through the next few lock losses and acquisitions.

Thanks again to the above mentioned authors for all their help!

The free memory size on the guardian machine is about 4GB. At the current rate of usage we predict a reboot is needed before next Tuesday. At the next opportune time, we will increase the memory size from 12GB to 48GB and perhaps schedule regular reboots on Tuesdays. 

Plot of available memory for the month of November is attached (Y-axis mis-labelled, actually MB).

J. Kissel, S. Aston, P. Fritschel

Peter was browsing through the list of frame channels and noticed that there are some differences between H1 and L1 on PR2 (an HSTS), even after we've both gone through and made the effort to revamp our channel list -- see Integration Issue 6463, ECR E1600316, LHO aLOG 30844, and LLO aLOG 29091.

What difference he found is the result of the LHO-ONLY ECR E1400369 to increase the drive strength of the lower states *some* of the HSTSs. This requires the two sites to have a different front-end model library part for different types of the same suspension because the BIO control each stage is different depending on the number of drivers that have been modified;
At LHO the configuration is
    Library Part        Driver Configuration            Optics
    HSTS_MASTER.mdl     No modified TACQ Drivers        MC1, MC3
    MC_MASTER.mdl       M2 modified, M3 not modified    MC2
    RC_MASTER.mdl       M2 and M3 modified              PRM, PR2, SRM, SR2

At LLO, the configuration is
    Library Part        Driver Configuration            Optics
    HSTS_MASTER.mdl     No modified TACQ Drivers        MC1, MC3, PR2
    MC_MASTER.mdl       M2 modified, M3 not modified    MC2, PRM, SR2, SRM
    RC_MASTER.mdl       M2 and M3 modified       none

This model's DAQ channel list for the MC and RC masters is the same. The HSTS master is different, and slower, because these SUS are used for angular control only: 
                           HSTS (Hz)        MC or RC (Hz)
    M3_ISCINF_L_IN1        2048               16384

    M3_MASTER_OUT_UL       2048               16384
    M3_MASTER_OUT_LL       2048               16384
    M3_MASTER_OUT_UR       2048               16384
    M3_MASTER_OUT_LR       2048               16384

    M3_DRIVEALIGN_L_OUT    2048               4096

Since LLO's PR2 does not have any modifications to its TACQ drivers, it uses the HSTS_MASTER model, which means that PR2 alone is going to show up as a difference in the channel list between the sites that seemed odd Peter -- that L1 had 6 more 2048 Hz channels than H1. Sadly, it *is* used for longitudinal control, so LLO suffers the lack of stored frame rate.

In order to "fix" this difference, we'd have to create a new library part for LLO's PR2 alone that has the DAC channel list of an MC or RC master, but have the BIO control logic of an HSTS master (i.e. to operate M2 and M3 stages with an unmodified TACQ driver). That seems excessive given that we already have 3 different models due to differing site preferences (and maybe range needs), so I propose we leave things as is, unless there's dire need to compare the high frequency drive signals to the M3 stage of PR2 at LLO.

I attach a screenshot that compares the DAQ channel lists for the three library parts, and the two types of control needs as defined by T1600432.

I have been investigating the spike in the computed value of kappa_tst shown here in the summary pages:
https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/day/20161122/cal/time_varying_factors/
A closer look reveals a seeming correlation between the DARM line coherence and the values of kappa_tst (see the attached plot). Currently, kappa_tst computed values are gated with PCALY_LINE1 and SUS_LINE1, the only lines used to compute kappa_tst.

Things to note in the plot:
1) kappa_tst diverges from the expected range about a minute after the DARM line coherence uncertainty skyrockets, just the amount of time it takes to corrupt the 128 running median.
2) kappa_tst as computed by CALCS also diverges when the DARM coherence goes bad.
3) Ungated kappa_pu behaves similarly to kappa_tst, but since it is gated with the DARM coherence, the GDS output is not corrupted.

J. Kissel for S. Karki

I've moved the roaming calibration line to its highest frequency we intend to go, and it's also the last super-long duration we need. We may run through the lower frequency points again, given that (a) they need much less data, and (b) those data points were taken at various input powers that will likely confuse/complicate the analysis. Below is the current schedule status.

Current Schedule Status:
Frequency    Planned Amplitude        Planned Duration      Actual Amplitude    Start Time                 Stop Time                    Achieved Duration
(Hz)         (ct)                     (hh:mm)                   (ct)               (UTC)                    (UTC)                         (hh:mm)
---------------------------------------------------------------------------------------------------------------------------------------------------------
1001.3       35k                      02:00                   39322.0           Nov 11 2016 21:37:50 UTC    Nov 12 2016 03:28:21 UTC      ~several hours @ 25 W
1501.3       35k                      02:00                   39322.0           Oct 24 2016 15:26:57 UTC    Oct 31 2016 15:44:29 UTC      ~week @ 25 W
2001.3       35k                      02:00                   39322.0           Oct 17 2016 21:22:03 UTC    Oct 24 2016 15:26:57 UTC      several days (at both 50W and 25 W)
2501.3       35k                      05:00                   39322.0           Oct 12 2016 03:20:41 UTC    Oct 17 2016 21:22:03 UTC      days     @ 50 W
3001.3       35k                      05:00                   39322.0           Oct 06 2016 18:39:26 UTC    Oct 12 2016 03:20:41 UTC      days     @ 50 W
3501.3       35k                      05:00                   39322.0           Jul 06 2016 18:56:13 UTC    Oct 06 2016 18:39:26 UTC      months   @ 50 W
4001.3       40k                      10:00                   39322.0           Nov 12 2016 03:28:21 UTC    Nov 16 2016 22:17:29 UTC      days     @ 30 W (see LHO aLOG 31546 for caveats)
4301.3       40k                      10:00                   39322.0           Nov 16 2016 22:17:29 UTC    Nov 18 2016 17:08:49 UTC      days     @ 30 W          
4501.3       40k                      10:00                   39322.0           Nov 18 2016 17:08:49 UTC    Nov 20 2016 16:54:32 UTC      days     @ 30 W (see LHO aLOG 31610 for caveats)   
4801.3       40k                      10:00                   39222.0           Nov 20 2016 16:54:32 UTC    Nov 22 2016 23:56:06 UTC      days     @ 30 W
5001.3       40k                      10:00                   39222.0           Nov 22 2016 23:56:06 UTC