As requested by Jeff and the calibration review committee, I've done a number of checks related to tracking the behavior of PCAL lines in the online-calibrated strain. (Most of these checks accord with the "official" strain curve plots contained in https://dcc.ligo.org/DocDB/0121/G1501223/003/2015-10-01_H1_O1_Sensitivity.pdf) I report on these review checks below.

I started by choosing a recent lock stretch at LHO that includes segments in which the H1:DMT-CALIBRATED flag is both active and inactive (so that we can visualize the effect of both gated and ungated kappas on strain, with the expected behavior that gstlal_compute_strain defaults each kappa factor to its last computed median if ${IFO}:DMT-CALIBRATED is inactive). There is a 4-hour period from 8:00 to 12:00 UTC on 30 November 2016 that fits the bill (see https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/day/20161130/). I re-calibrated this stretch of data in --partial-calibration mode without kappas applied, and stored the output to

LHO: /home/aurban/O2/calibration/data/H1/

All data were computed with 32 second FFT length and 120 second stride. The following plots are attached:

• Timeseries of the ASD residuals of CAL-PCALY_RX_PD_OUT_DQ to GDS-CALIB_STRAIN (the online data with kappas applied) and TEST-CALIB_STRAIN (my recalibrated data without kappas), as computed at the PCAL lines (36.7, 331.9, and 1083.7 Hz).
• Overlayed median spectra of these three channels, zoomed in at each PCAL line.
• Overlayed ASD ratios zoomed in at each PCAL line.
• Full, overlayed median spectra of PCAL and the strian channels.
• A timeseries of the computed BNS range with and without kappas, compared against the behavior of each kappa (except the coupled cavity pole frequency, which is not applied online).

The script used to generate these plots, and a LAL-formatted cache pointing to re-calibrated data from the same time period but without any kappa factors applied, is checked into the calibration SVN at https://svn.ligo.caltech.edu/svn/aligocalibration/trunk/Runs/PreER10/H1/Scripts/TDkappas/. A similar analysis on a stretch of Livingston data is forthcoming.

Now that we have a few days of O2 under H1's belt.  I wanted to give a shout out to the absolute latest Operator Sticky Notes we have so far.  And a reminder is that these Sticky Notes are a wiki page here:  https://lhocds.ligo-wa.caltech.edu/wiki/OperatorStickyNotes.  Some of the older ones have been moved to an "old sticky notes" section at the bottom of the page.  Anyone should feel free to update this list to make it pertinent and useful for operations.

Operators please note these latest Sticky Notes:

H1 Sticky Notes & Alogs/Work Permits

• 12/2-12/9: High freq Calib Line Roaming changes (WP#6368) 

• 12/1: TCS Power Stabilization Guardian knocking out of Observing (Kiwamu alog#32090) 

• 12/1: Surviving rung up resonant modes via "RemoveStageWhitening" (Jenne alog#32089) 

• 12/1: ISI_CONFIG during earthquakes (Sheila's comment for alog#32086) 

• 12/1: Resonant Mode Scratch Pad! (Jeff's alog #32077) 

• 11/30: LVEA SWEPT on 11/29&11/30 by Betsy, Dave, & Fil (alogs 31975, 32024). 

• 11/30: Run A2L ( once a day until ~12/8. Always run A2L on Tues Maintenance Day. (Jenne alog#32022)) 

Dave, TJ:

A recent plot of free memory showed that the rate-of-decrease increased around noon Tuesday 15th Nov. TJ tracked this to a DIAG_MAIN code change wherein a slow channel is being averaged over 120 seconds every 2 seconds. Doing the math, this equates to 0.33GB per day. This matches the increased memory consumption rate seen since Nov 15.

To test this, during the lunch time lock loss today, we killed and restarted the DIAG_MAIN process. Attached is a plot of free memory from 9:30am Thursday PST (after the memory size of h1guardian was increased to 48GB) and 2:30pm PST today. The last data points show the memory recovered by the restart of DIAG_MAIN, and it agrees with 330 MB per day.

With the increased memory size we anticipate no memory problems for 3 months at the current rate of consumption. However we will schedule periodic restart of the machine or the DIAG_MAIN node during maintenance.

• useism is well above the 90percentile.  (we had a 20+hr lock end...most likely due to this?)
• Back up to OBSERVING, but can clearly see useism effects:
  • See it on the Tidal Strip Tool
  • SEI BLRMS is above 90percentile
  • Glitches of PRC Gain & POP18 look glitchy/noisy on PRMI striptool
  • Range is just above 70Mpc
  • Continue to run in the standard WINDY ISI_CONFIG

Additional CBC injections are scheduled:

1164747030 H1L1 INJECT_CBC_ACTIVE 1 1.0 Inspiral/{ifo}/bbhspin_hwinj_snr24_1163501502_{ifo}_filtered.txt
1164751230 H1L1 INJECT_CBC_ACTIVE 1 1.0 Inspiral/{ifo}/bbh_hwinj_snr24_1163501502_{ifo}_filtered.txt
1164755430 H1L1 INJECT_CBC_ACTIVE 1 0.5 Inspiral/{ifo}/imri_hwinj_snr24_1163501530_{ifo}_filtered.txt
1164759630 H1L1 INJECT_CBC_ACTIVE 1 0.5 Inspiral/{ifo}/imri_hwinj_snr24_1163501538_{ifo}_filtered.txt

Continuing the schedule for this roaming line with a move from 2501.3 to 3001.3 Hz.

Frequency    Planned Amplitude        Planned Duration      Actual Amplitude    Start Time                 Stop Time                    Achieved Duration
(Hz)         (ct)                     (hh:mm)                   (ct)               (UTC)                    (UTC)                         (hh:mm)
---------------------------------------------------------------------------------------------------------------------------------------------------------
1001.3       35k                      02:00                   39322.0           Nov 28 2016 17:20:44 UTC  Nov 30 2016 17:16:00 UTC         days    @ 30 W  
1501.3       35k                      02:00                   39322.0           Nov 30 2016 17:27:00 UTC  Nov 30 2016 19:36:00 UTC         02:09   @ 30 W         
2001.3       35k                      02:00                   39322.0           Nov 30 2016 19:36:00 UTC  Nov 30 2016 22:07:00 UTC         02:31   @ 30 W
2501.3       35k                      05:00                   39322.0           Nov 30 2016 22:08:00 UTC  Dec 02 2016 20:16:00 UTC         days    @ 30 W
3001.3       35k                      05:00                   39322.0           Dec 02 2016 20:17:00 UTC
3501.3       35k                      05:00                   39322.0           
4001.3       40k                      10:00                   39322.0           
4301.3       40k                      10:00                   39322.0                
4501.3       40k                      10:00                   39322.0           
4801.3       40k                      10:00                   39222.0           
5001.3       40k                      10:00                   39222.0           

Frequency    Planned Amplitude        Planned Duration      Actual Amplitude    Start Time                 Stop Time                    Achieved Duration
(Hz)         (ct)                     (hh:mm)                   (ct)               (UTC)                    (UTC)                         (hh:mm)
---------------------------------------------------------------------------------------------------------------------------------------------------------
1001.3       35k                      02:00                   39322.0           Nov 11 2016 21:37:50 UTC    Nov 12 2016 03:28:21 UTC      ~several hours @ 25 W
1501.3       35k                      02:00                   39322.0           Oct 24 2016 15:26:57 UTC    Oct 31 2016 15:44:29 UTC      ~week @ 25 W
2001.3       35k                      02:00                   39322.0           Oct 17 2016 21:22:03 UTC    Oct 24 2016 15:26:57 UTC      several days (at both 50W and 25 W)
2501.3       35k                      05:00                   39322.0           Oct 12 2016 03:20:41 UTC    Oct 17 2016 21:22:03 UTC      days     @ 50 W
3001.3       35k                      05:00                   39322.0           Oct 06 2016 18:39:26 UTC    Oct 12 2016 03:20:41 UTC      days     @ 50 W
3501.3       35k                      05:00                   39322.0           Jul 06 2016 18:56:13 UTC    Oct 06 2016 18:39:26 UTC      months   @ 50 W
4001.3       40k                      10:00                   39322.0           Nov 12 2016 03:28:21 UTC    Nov 16 2016 22:17:29 UTC      days     @ 30 W (see LHO aLOG 31546 for caveats)
4301.3       40k                      10:00                   39322.0           Nov 16 2016 22:17:29 UTC    Nov 18 2016 17:08:49 UTC      days     @ 30 W          
4501.3       40k                      10:00                   39322.0           Nov 18 2016 17:08:49 UTC    Nov 20 2016 16:54:32 UTC      days     @ 30 W (see LHO aLOG 31610 for caveats)   
4801.3       40k                      10:00                   39222.0           Nov 20 2016 16:54:32 UTC    Nov 22 2016 23:56:06 UTC      days     @ 30 W
5001.3       40k                      10:00                   39222.0           Nov 22 2016 23:56:06 UTC    Nov 28 2016 17:20:44 UTC      days     @ 30 W (line was OFF and ON for Hardware INJ)

   Ops has been seeing high dust count alarms in the PSL. Trended the PSL and LVEA dust monitors for 15 and 90 days. The trends show constant major alarm dust levels in the PSL enclosure (Monitor PSL101) in the 0.3u particles and several 0.5u alarm level events. This trend holds for the PSL Anti-Room (PSL102) as well, although at a lower level of alarm raising events. 
   The PSL Enclosure and the Anti-Room have not been cleaned for quite some time. Will arrange for a cleaning of the PSL during the next maintenance window. If this does not lower the PSL dust counts, will need to start looking into the air filtration system feeding the PSL enclosure.         

This was a quick EQ during the OWL shift at 6:16amPST (14:16utc).  H1 rode through it.

USGS link

• Magnitude of EQ (according to Terramon):  4.9
• Location of EQ (according to Terramon):  11km ENE of Petatlan, Mexico
• Starting time of Event:  14:16 UTC (saw small peak over a few minutes on 0.03-0.1 BLRMS)
• Lock Status:  H1 Remained in OBSERVING
• Terramon posted 1.0 um/s.  LHO BLRMS saw 0.25 um/s

TITLE: 12/02 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 73.5385Mpc
INCOMING OPERATOR: Nutsinee
SHIFT SUMMARY: Mostly a quiet shift, but there were a couple SDF issues
LOG:
Corey had just relocked after an earthquake when I arrived. Shortly after going to OBSERVE the TCS guardians knocked us out, as Kiwamu logged. Then quiet until just a couple minutes ago. SDF_DIAG kick us out of OBSERVE, looking at the log I find:

2016-12-02T07:51:08.18971 DIAG_SDF [RUN_TESTS.run] USERMSG 0: DIFFS: ngn: 1
2016-12-02T07:51:10.70676 DIAG_SDF [RUN_TESTS.run] USERMSG 0: DIFFS: ngn: 3
2016-12-02T07:51:17.18839 DIAG_SDF [RUN_TESTS.run] USERMSG 0: DIFFS: ngn: 1

I can't find an (the? I didn't know we had one) NGN guardian, but I know where the CS_BRS guardian lives. When I looked at that guardian, it had just started a recenter cycle at the same time:

2016-12-02T07:51:07.99681 CS_BRS JUMP target: TURN_OFF_DAMPING
2016-12-02T07:51:07.99694 CS_BRS [RUN.exit]
2016-12-02T07:51:08.05901 CS_BRS JUMP: RUN->TURN_OFF_DAMPING
2016-12-02T07:51:08.05920 CS_BRS calculating path: TURN_OFF_DAMPING->RUN
2016-12-02T07:51:08.05959 CS_BRS new target: TURN_OFF_PZT_CTRL

...

Did the CS_BRS guardian throw an SDF difference in NGN that dropped us out of OBSERVE?
 

USGS Link

• Magnitude of EQ (according to Terramon):  6.3
• Location of EQ (according to Terramon):  43km NE of Huarichancara, Peru
• Starting time of Event:  22:49UTC (started seeing 0.03-0.1 BLRMS increase
• Lock Status:  H1 Lockloss at 23:25UTC

I have looked at all the A2L data that we have since the last time the alignment was significantly changed, which was Monday afternoon after the PSL PZT work (alog 31951).  This is the first attached plot.

The first data point is a bit different than the rest, although I'm not totally sure why.  Other than that, we're mostly holding our spot positions quite constant.  The 3rd-to-last point, taken in the middle of the overnight lock stretch (alog 32004) shows a bit of a spot difference on ETMX, particularly in yaw, but other than that we're pretty solid.

For the next ~week, I'd like operators to run the test mass a2l script (a2l_min_lho.py) about once per day, so that we can track the spot positions a bit.  After that, we'll move to our observing run standard of running a2l once a week as part of Tuesday maintenence.

The second attached plot is just the last 2 points from the current lock.  First point was taken immediately upon lock, second was take about 30 min into the lock.  The maximum spot movement in the figure appears to be about 0.2mm, but I think that is within the error of the A2L measurement.  I can't find it right now, but once upon a time I ran A2L 5 or 7 times in a row to see how consistent the answer is, and I think I remember the stdev was about 0.3mm. 

The point of the second plot is that at 30W, it doesn't seem to make a big difference if we run a2l immediately or a little later, so we can run it for our once-a-days as soon as we lock, or when we're otherwise out of Observe, and don't have to hold off on going to Observe just for A2L.

Could someone on site check the coherence of DARM around 1080 Hz with the usual jitter witneses? We're not able to do it offsite because the best witness channels are stored with a Nyquist of 1024 Hz. What we need is the coherence from 1000 to 1200 Hz with things like IMC WFS (especially the sum, I think). The DBB would be nice if available, but I think it's usually shuttered.

There's indirect evidence from hVeto that this is jitter, so if there is a good witness channel we'll want to increase the sampling rate in case we get an SN or BNS that has power in this band.

Betsy, Keita, Daniel

As part of the LVEA sweep, prior to the start of O2, this morning, we spent over an hour doing a cleanup of misc cables and test equipment in the LVEA and electronics room.  There were quite a few cables dangling from various racks, here's the full list of what we cleaned up and where:

I also threw the main breaker to the OFF position on both of the free standing unused transformer units in the LVEA - one I completely unplugged because I thought I could still hear it humming.

No monitors computers appear to be on except the 2 VE BECKHOFF ones that must remain on (in their stand alone racks on the floor).

We'll ask the early morning crew to sweep for Phones, Access readers, lights, and WIFI first thing in the morning.

Kyle R., Gerardo M., Richard M. 

(completes WPs #6332 and #6360)

Initial pressure indication shortly after being energized was 3 x 10-3 torr (PT180 is a "wide-range" gauge).  If real, this would be higher than expected for the ~17 hrs. of accumulation -> "burped" the accumulated gas into the connected pump setup while Gerardo monitored its pirani gauge -> it gave no indication of a change and remained steady at 1.9 x 10-3 torr which is as expected.  Neither gauge is calibrated in this pressure region of interest.  Noted PT180 responded as expected to being combined with local turbo -> Isolated temporary local pump setup, valved-in (exposed/combined) PT180 to site vacuum volume, vented locally mounted turbo and removed from PT180 hardware -> Installed 1 1/2" O-ring valve and 2.75" CF to NW40 adapter in place of the turbo and pumped to rough vacuum the space between the two 1 1/2" pump port valves.  

Jenne, Sheila Keita

We had another instance of a jump in POP90, in which both the I and Q phases increased. We think this is a problem with the readback, similar to what is described in 31181.  

We were acquiring lock, and had no ASC running on the SRC.  We looked at witness sensors for all the interferometer optics, and it looks like none of them moved at the time.  We also don't see changes in other RF sensors, like AS36, AS90, or POP18. We looked at both quadratures of POP90 before rotation and it seems to have both a phase shift and a 3.5 dB increase in the sum of I+Q. The RFmon and LO mon on the demod don't have any jumps nearly that large, so if it is a problem in the demod it is probably downstream of the directional coupler for the RFmon. 

This seems not to be the same as the jumps in SRC alingment that started after last tuesday's maintence, (31804 31865 and other alogs), but since the symptom is very similar it would make debugging the other problem easier if this issue could be fixed. Since we use POP90 for a dither lock of the SRM angle durring lock acquisition, this can cause a lockloss if it happens while we are trying to lock. 

I've been working on a prototype epics interface to the seismon system. It currently has several moving parts.

EPICS IOC

Runs on h1fescript0 as user controls (in a screen environment). Code is

/ligo/home/controls/seismon/epics_ioc/seismon_ioc.py

It is a simple epics database with no processing of signals.

EVENT Parser, EPICS writer

A python script parses the data file produced by seismon_info and sends the data to EPICS. It also handles the count-down timer for future seismic events.

This runs on h1hwinj2 as user controls. Code is

/ligo/home/controls/seismon/bin/seismon_channel_access_client

MEDM

A new MEDM called H!SEISMON_CUST.adl is linked to the SITEMAP via the SEI pulldown (called SEISMON). Snapshot attached.

The countdown for the P,S,R waves are color coded for the arrival time of the seismic wave

ORANGE   more than 2 mins away

YELLOW  between 1 and 2 minutes away

RED  less than 1 minute away

GREY in the past

If the system freezes and the GPS time becomes older than 1 minute, a RED rectangle will show to report the error.