No obvious seismic jolt for lockloss, BUT we had been riding some high useism.

Locking Notes:

1) IMC not locking!  (it did look misaligned on Trans when it tried grabbing lock)

• Going through Jenne's troubleshooting doc, the first thing I did was clear WFS history for IMC:  NOTHING
• Check MC1/2/3 witness sensors.  There was a step in most of these channels, so I steered all of the MC's to values when the IMC was locked & this did the trick.  (see attached plot for the step which occurred after the lockloss).

2) H1 Locking Status:  Have had a couple of failed attempts after & before CARM Reduction steps.

• To allow locking of PRMI, I had to adjust the BS & PRM quite a bit.
• I have been able to lock DRMI & made it past CARM reduction steps once (dropped at ANALOG CARM when all signals became noisy & ASAIR spot looked ugly), and also lost lock before  twice CARM reduction at SWITCH TO QPD steps.
• Next going to check Bounce/Roll modes & then try CARM reduction steps by hand.
• (texting Jim about useism and he recommends staying in the WINDY states.

Crystal Chiller topped off with 265mL this morning (Diode Chiller had GREEN light for its level).  This closes FAMIS 6506.

TITLE: 01/21 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 63Mpc
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
    Wind: 5mph Gusts, 4mph 5min avg
    Primary useism: 0.19 μm/s
    Secondary useism: 0.87 μm/s
QUICK SUMMARY:

Not much to hand-off from Patrick other than the Configuration Change note for H1OAF on the CDS Overview. 

useism has inched up in the last 8-10hrs (above the 90percentile and almost clearing it!) & you can see the signal on the Tidal Striptool (nuc1).

Beautiful clear morning with no issues driving in & a crisp 28degF out.

TITLE: 01/21 Owl Shift: 08:00-16:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 65Mpc
INCOMING OPERATOR: Jim
SHIFT SUMMARY: Remained in observing the entire shift. The filter changed by Sheila that Cheryl reported in her shift summary remains unloaded.
LOG:

09:14 UTC Damped PI mode 28 by changing sign of gain
14:45 UTC GRB alert

Have remained in observing. No issues to report.

TITLE: 01/21 Owl Shift: 08:00-16:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 65Mpc
OUTGOING OPERATOR: Cheryl
CURRENT ENVIRONMENT:
    Wind: 4mph Gusts, 3mph 5min avg
    Primary useism: 0.08 μm/s
    Secondary useism: 0.67 μm/s 
QUICK SUMMARY:

No issues to report.

TITLE: 01/21 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 61Mpc
INCOMING OPERATOR: Patrick
SHIFT SUMMARY:

• two GRBs
• one lock loss
• relocking was ok, lost lock after DRMI a couple times and then made it to NLN
• had to accept a -2e-16 change on H1:PSL-ISS_REFSIGNAL to go to Observe
• red dot on CDS overview, filters changed by Sheila, not loaded, no change to IFO, snapshot attached

I have determined new frame data rates for H1 and L1.  This was last updated during ER10 ( see LHO aLOG 31567  )

You can find the scripts (work at either site) at https://llocds.ligo-la.caltech.edu/data/keith.thorne/DataRates
You can find the L1 data at https://llocds.ligo-la.caltech.edu/data/keith.thorne/DataRates/2017-01-20/, the H1 data at https://lhocds.ligo-wa.caltech.edu/exports/keith.thorne/DataRates/2017-01-20/.

I have attached the summary data sheets in PDF and XLS formats as well as channel lists. These are all raw (uncompressed) rates.

The raw H1 fast data size has shrunk from 45.59 to 43.47 MB/s (decrease of 5%). [Removal of now-unneeded spurious 64KHz channels from PI models]
The raw L1 fast data size has increased slightly from 40.56 to 40.60 MB/s (increase of 0.1%)

The raw H1 fast data rates are now 7% larger than L1 fast data rates
Sub-systems leading to higher H1 rates are PSL (0.5 MB/s), SUS (0.5 MB/s), ISC (1.0 MB/s), PEM (1.0 MB/s)

From DAQ MEDM screens during lock
H1 frames are ~1630 MB / 64 sec -> 25 MB/s
L1 frames are ~1420 MB / 64 sec -> 22 MB/s

Background - 
Both CP3 and CP4 are experiencing obstructed "high side" sensing lines which prevent their level transducers from measuring the level of LN2 in the 80K pumps.  We consider that this is due to accumulated ice at the bottom of the pumps inner LN2 vessel where the "high side" sensing line transitions from the relatively large ID aluminum tube to the relatively small ID stainless tube near a 90 degree elbow.  While Chandra R. has developed a procedure for a permanent post-O2 fix, we, nevertheless, would like to pursue more accommodating fix attempts in the meantime.  One of these is to see if by applying a vacuum (for a prolonged period) to the sensing line enough ice could be liberated through reverse sublimation that the obstruction could then be freed by alternating applications of modestly pressurized UHP N2 followed by vacuum pumping.  

Today - 
WP #6442 
Kyle R., Chandra R., Dave B., Gerardo M., Marc P.
I reconnected the "clogged" sensing line (with added parallel valved line used for vacuum pumping and a parallel valved line used for back filling) to the CDS level transducer (Rosemont model 1151DP4E52B404).  I then applied various "on-scale" pressure differentials via the back fill line (as indicated by the parallel connected magnahelic DP gauge) while Chandra R., Dave B. and Gerardo M. monitored the CDS MEDM screen. This confirmed that the transducer was operating and was in rough agreement with the mechanical gauge.  I also switched to the application of vacuum and they noted that the 4-20ma transducer output fell to 0 mA or something less than 4 mA (thought 4 mA output was minimum output?).  I then rearranged the plumbing fittings so as to permit selecting between a vacuum pump or a back fill line (10 psi UHP N2 @ 2 LPM) without the need to undo any tubing fittings as had been the case initially.  During this plumbing rearrangement, the sensing line was open to the room and began to flow GN2 past the obstruction.  This phenomenon had been observed on previous dislodge attempts (see entries from last month?) the difference this time was the significant increase in volume and duration - "Almost got it!"  I experimented with various durations of pumping and back filling etc. and observed on a few occasions that the audible gas load to the diaphragm pump would change spontaneously if let alone for several minutes.  

Anyway, this is all very encouraging and will likely be continued next week.  I halted tonight's attempts in observance of the Vacuum Team's Friday Rule and to allow others (the larger vacuum community) to consider if this approach is thermally cycling the bi-braze joint and poses a risk.  I am leaving the sensing line back filled to 10 psi and isolated from both the vacuum line and back fill line.  

WP6442, alarm on positive LN2 level in CP4 pump

Chandra, Kyle, Gerardo, Dave

CP4 LN2 Pump Level channels were added to the cell phone text alert system ( H0:VAC-MY_CP4_LT250_PUMP_LEVEL_PCT,  H0:VAC-MY_CP4_LT250_PUMP_LEVEL_PCT_ERROR).

Unfortunately the ERROR channel was in Error this afternoon which caused many cell phone alerts. Trending this channel shows that over the past month it has been transitioning between Error and No Error states, which the majority of the time being in Error.

Until this channel's intermittent errors can be resolved, we cannot rely on this as a critical alarm. WP6442 work has been postponed until Monday.

TITLE: 01/21 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 62Mpc
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY: 2 locklosses, one I'm not sure of the cause, the other was from PI mode 27. I changed a BP filter for the PI and it seemed to have fixed it.
LOG:

• 16:42 Joe starting some ice clearing near receiving. Only using a shovel.
• 16:54 Joe stopping, ice too thick to break without bringing in heavier equipment, but not while we are Observing.
• 18:02 Kyle to MY to turn pump on.
• 18:31 Lockloss
• 19:23 Chandra to Mids
• 19:50 Observing
• 22:20 Kyle to MY
• 22:50 Kyle going past the card reader to get a tank.
• 22:55 Kyle back
• ~22:57 Kyle made "a big bang" loading the tank off a ledge.

Kyle went to grab a Helium tank just past the card reader at 22:51 UTC and found the card reader already ON. This is on our LVEA Sweep checklist but must have been missed. It is now OFF

I changed the PI BP and accept the difference in SDF, let's try it this way.