Fil, Daniel, Jenne, Keita, Patrick, Richard, Sheila, Kiwamu,

The investigation continues. No solid conclusion or resolution yet.

• We tried locking the laser to the Y arm by only using the PDH servo board in order to test the PDH board.
  • The output of the PDH board was connected to the cable which is nominally connected to the output of the PLL board for this test operation.
  • The same filter settings as usual but with a lower gain (I had to lower the fast path gain as well).
  • This locked the laser somewhat but kept unlocking roughly every 10 sec. Note that this behavior is different from the glitches we have seen recently.
  • It is inconclusive to say whether the PDH board is to blame for the glitches.
• Regardless of whether the PDH board is the culprit, we decided to swap the board.
  • See 36622.
  • The first replacement unit turned out to be not working (non funzione in Italian) because it didn't responds to changes in the remote offset adjusters.
  • Fil then installed another spare unit with which we were able to control the electronic offset using the MEDM sliders.
• We then found that the FET demodulator gave us a big offset.
  • We drove to the end station and then determined that the demodulator unit was non funzione either. The I-signal was just flat 5 V.
  • We switched the system to the second half of the same demodulator unit.
• Now we are waiting for the Alaskan earthquake (LLO 34072) to settle so that we can test out the new PDH board.

TITLE: 06/02 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
    Wind: 7mph Gusts, 4mph 5min avg
    Primary useism: 3.58 μm/s
    Secondary useism: 0.87 μm/s
QUICK SUMMARY: ALS work continues. 6.9M earthquake off of Alaska will be hindering us for a bit.

TITLE: 06/02 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: TJ
SHIFT SUMMARY:
LOG:

15:00 Sheila to EY to investigate ALSY PLL.

15:15 sensor correction turned of at EY

15:46 Jim doing TFs on HAM5

16:00 Sheila back

16:05 SC turned back on at EY

16:11 Granted Aiden remote access

16:14 Richard transitioning LVEA to LASER SAFE

16:31 SC back OFF on EY - Kiwamu heading back out

16:54 Betsy in LVEA to look through TCS cabinets

16:57 Ken into LVEA to look for a ladder

17:07 Ken out

17:10 Richard and Kiwamu restarting ISC EY computers

17:21 Betsy out. Heading to the MX in 15 minutes.

17:38 Richard back from EY

17:58 Bubba into the LVEA

18:38 Kiwamu back

18:44 Gerardo going into LVEA

18:54 Gerardo out

19:18 Fil going to EY to switch out a common mode servo chassis for ALS

20:02 Fil back

20:46 Fil goin back to end station. Newly installed chassis not responding

20:46 Peter into the optics lab

22:00 Crew back at EY working on ALS

22:30 6.9mag EQ in Alaska 30um/s

The faulty regulator which turned out to be only a bad o-ring in the regulator itself, has been repaired, reinstalled, and air is flowing again. I unlocked the valves with Chandra watching the vacuum screen in the control room.
While I was repairing the regulator, Kyle was ordering a couple of new regulators for spares. 

Chandra and I also replaced the check valves on the corner station instrument air compressor.

J. Kissel

Since I've been staring at the thermometers at the end stations for the past few weeks, I surmised that our troubles with the Y-end ALS glitching might have something to do with temperature. It's a wild guess, but we're running out of ideas. Sadly, this one was not fruitful either. No obvious connection between the end station temperature and the ALS glitches.

To test (refute) the hypothesis, I looked at the temperature of the relavent VEA during all of the times that we've complained about the ALS glitching,
LHO aLOG 36602
    2017-06-02 Y End
    2017-05-27 Y End
LHO aLOG 31053
    2016-11-01 Y End
LHO aLOG 25523
    2016-10-14 Y End
LHO aLOG 22184
    2016-02-12 X End
    2016-02-12 Y End
LHO aLOG 15242
    2015-02-10 Y End
I take a day or two's work of temperature, and mark the times of complaint but solid black vertical lines. The times are derived from the dataviwer or DTT traces that are posted to that log.

There's no consistent answer, so it's mostly inconclusive if not refuting. Some of the days, there appears to be a temperature swing in the transmitter module, but FMCS and the receiver module are OK. Other days, there's no excursion. Some days the temperature is high and moving slowly around, other days it's normal and rock solid.

So, I think this is a red herring, unless there are suggestions to further refine the study.

CP3's PID control is now actively maintaining 92% LN2 level in the pump. Chandra has confirmed that we will not run the daily CP3 autofill until further notice.

WP 7014

Installed spare ALS Common Mode Servo Chassis S1102633 in ISC-R1 rack (EY). Unit was installed to help troubleshoot the ongoing glitches seen on the ALS system. Unit was mounted below original unit S1102632.

Jenne, SHeila, Patrick, Ketia

We are still not able to lock ALS for very long.  We tried locking only ALS COMM, and were able to stay locked for more than 10 minutes.  We have been sitting here watching signals from the green arms, and we see the same ALS glitches (in the Yarm) that are described in many old alogs: 31053 lists som.  

Keita Patrick and I followed Jenne's instructions in alog 30519 to lock the green laser to the arm directly by passing the PLL.  In this configuration we did not see the glitches, but now that we have swapped the cables back to the original configuration the glitches are bac.  So it seems likely that the PLL is somehow involved in these glitches. I am not sure if these glitches are the reason we cannot lock, but in the past they have prevented locking. 

Edited attachment with annotations to make it more clear. (ignore the grey smudge)  We had the green arm locked bypassing the PLL from about 6-2-2017 1:38 UTC to 1:44 UTC

The second screenshot shows a repeat of this test using a gain of 6dB instead of 10dB. I think that the loop was oscillating with the 10dB setting, so this setting is better. 

CP3 log file DOES NOT exist!
CP4 Fill completed in 300 seconds. Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 300 seconds. LLCV set back to 40.0% open.

Chandra, Dave:

CP3 is under PID control and its level sensor started reading below 100.0% around 10am this morning. So CP3 auto overfill was cancelled, and CP4 proceeded as scheduled at 11:02 PDT.

I made some changes to cp4_fill.py to add the summary text and correct some indentation isues.

Richard, Dave:

We power cycled the h1iscey computer and IO Chassis as part of the ALS-EY glitch investigation. Sequence was:

h1iscey, stop all models, take FE out of Dolphin fabric, power down comuter

IO Chassis: power cycle

h1iscey, power on computer, models restarted automatically

We had a brief (5 minute) IRIG-B excursion on h1iopiscey. After this cleared, DAQ and IPC were working correctly.

2017_06_02 10:17 h1caley
2017_06_02 10:17 h1iopiscey
2017_06_02 10:17 h1iscey
2017_06_02 10:17 h1pemey
2017_06_02 10:18 h1caley
2017_06_02 10:18 h1iopiscey
2017_06_02 10:18 h1iscey
2017_06_02 10:18 h1pemey
2017_06_02 10:19 h1alsey

CP3's reservoir level is back on scale - reading around 97% full. I lowered the PID lower limit of LLCV to 15% open when it was ~99% full to measure percentage per hour to get a feel for how low the reservoir got over the weekend.

PID is programmed to hold the reservoir at 92% full. Once it drops to that level, PI will kick in.

The pneumatic regulator connected to the supply line for the cylinder on GV8 has failed. The cylinder was still getting enough pressure to hold the gate open.
 
The ball valve used to isolate the pneumatic regulator is up stream of both GV 7 & 8 so I mechanically locked out both valves.  I talked with Chandra and Kyle before I locked out the valves. 
I will fill out a FRS for to replace the regulator this morning and confer with Chandra before any pressure is applied. This also has ties to FRS 8259.