Remote logged in to toggle PT-134, CC was off when I logged in.
I turned on and off the CC several time to try to get it to get on scale, but no luck. Left the CC on, but it is not on scale, sometimes they take long to do so.
CP3 log file DOES NOT exist! CP4 Fill completed in 298 seconds. Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 298 seconds. TC A did not register fill. LLCV set back to 41.0% open.
Bubba, Kyle Fixing Corner Station instrument air. LVEA and chiller yard. Will make comment to this entry when leaving site.
The o-ring on the same regulator that was repaired yesterday blew out again. This time I replaced the entire regulator, slightly different style but no leaks on this one. I believe the old regulator had just worn out. GV 7 & 8 gate cylinders are back under normal supply air pressure and open.
1255 hrs. local -> Leaving site now.
here is a minute trend (mean) plot of CP3's pump level and LLCV position over the past 24 hours showing good control of the LN2 level at the 92% set point. Reminder that only CP4 will autofill in 20 minutes time.
TITLE: 06/03 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: None
SHIFT SUMMARY: I tried to lock ALS a few times after commissioners left to no success. Y would complain of VCO problems and would not hold the a lock longer than a few tens of seconds. No one relieving me, and no day operator tomorrow.
LOG:
1925 hrs. local I found the rotometer ball off of the stop about 1/2 the diameter of the ball (< 0.4 LPM the smallest graduation of the flow scale). When the isolation valve at the plumbing manifold was closed, this small flow stopped, i.e. the ball rested. This was repeatable. This could be flow across the sensing line blockage or it could also be leakage flow out of the 4 or 5 pipe joints which can't be isolated. Anyway, this wasn't occurring the last time I inspected it. The UHP N2 bottle is at 1800 psi tonight, down 300 psi from 24 hrs. ago. While there, I repeated the exercise of switching between pumping the sensing line with a small vacuum pump then switching to pressurizing with the UHP N2. I had made a few attempts at this yesterday? the day before? but could never establish any flow. Tonight, I was able to achieve flow for 10s of seconds at a time. I would have like to have continued this as it is so encouraging but I don't want my children to grow up without knowing their daddy + plus I'm hungry plus ..... I am leaving the apparatus at 60 psi regulator output and adjusted for 2 LPM max.
Fil, Daniel, Jenne, Keita, Patrick, Richard, Sheila, Kiwamu,
The investigation continues. No solid conclusion or resolution yet.
Just to summarize what we think we know so far:
Things that were exonerated by bypassing PLL test:
Other things that we think are exonerated:
Things that we haven't eliminated:
TITLE: 06/02 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
Wind: 7mph Gusts, 4mph 5min avg
Primary useism: 3.58 μm/s
Secondary useism: 0.87 μm/s
QUICK SUMMARY: ALS work continues. 6.9M earthquake off of Alaska will be hindering us for a bit.
TITLE: 06/02 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: TJ
SHIFT SUMMARY:
LOG:
15:00 Sheila to EY to investigate ALSY PLL.
15:15 sensor correction turned of at EY
15:46 Jim doing TFs on HAM5
16:00 Sheila back
16:05 SC turned back on at EY
16:11 Granted Aiden remote access
16:14 Richard transitioning LVEA to LASER SAFE
16:31 SC back OFF on EY - Kiwamu heading back out
16:54 Betsy in LVEA to look through TCS cabinets
16:57 Ken into LVEA to look for a ladder
17:07 Ken out
17:10 Richard and Kiwamu restarting ISC EY computers
17:21 Betsy out. Heading to the MX in 15 minutes.
17:38 Richard back from EY
17:58 Bubba into the LVEA
18:38 Kiwamu back
18:44 Gerardo going into LVEA
18:54 Gerardo out
19:18 Fil going to EY to switch out a common mode servo chassis for ALS
20:02 Fil back
20:46 Fil goin back to end station. Newly installed chassis not responding
20:46 Peter into the optics lab
22:00 Crew back at EY working on ALS
22:30 6.9mag EQ in Alaska 30um/s
The faulty regulator which turned out to be only a bad o-ring in the regulator itself, has been repaired, reinstalled, and air is flowing again. I unlocked the valves with Chandra watching the vacuum screen in the control room. While I was repairing the regulator, Kyle was ordering a couple of new regulators for spares. Chandra and I also replaced the check valves on the corner station instrument air compressor.
The regulator failed again the next day, so we replaced it with an odd ball spare. Awaiting order delivery of direct replacements.
J. Kissel Since I've been staring at the thermometers at the end stations for the past few weeks, I surmised that our troubles with the Y-end ALS glitching might have something to do with temperature. It's a wild guess, but we're running out of ideas. Sadly, this one was not fruitful either. No obvious connection between the end station temperature and the ALS glitches. To test (refute) the hypothesis, I looked at the temperature of the relavent VEA during all of the times that we've complained about the ALS glitching, LHO aLOG 36602 2017-06-02 Y End 2017-05-27 Y End LHO aLOG 31053 2016-11-01 Y End LHO aLOG 25523 2016-10-14 Y End LHO aLOG 22184 2016-02-12 X End 2016-02-12 Y End LHO aLOG 15242 2015-02-10 Y End I take a day or two's work of temperature, and mark the times of complaint but solid black vertical lines. The times are derived from the dataviwer or DTT traces that are posted to that log. There's no consistent answer, so it's mostly inconclusive if not refuting. Some of the days, there appears to be a temperature swing in the transmitter module, but FMCS and the receiver module are OK. Other days, there's no excursion. Some days the temperature is high and moving slowly around, other days it's normal and rock solid. So, I think this is a red herring, unless there are suggestions to further refine the study.
CP3's PID control is now actively maintaining 92% LN2 level in the pump. Chandra has confirmed that we will not run the daily CP3 autofill until further notice.
Installed spare ALS Common Mode Servo Chassis S1102633 in ISC-R1 rack (EY). Unit was installed to help troubleshoot the ongoing glitches seen on the ALS system. Unit was mounted below original unit S1102632.
CP3 log file DOES NOT exist! CP4 Fill completed in 300 seconds. Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 300 seconds. LLCV set back to 40.0% open.
Raised CP4 to 41% open.
We'll continue to report on CP3 for now, but hopefully once it is determined CP3 no longer needs autofills, I'll switch to reporting CP4 only.
CP3's reservoir level is back on scale - reading around 97% full. I lowered the PID lower limit of LLCV to 15% open when it was ~99% full to measure percentage per hour to get a feel for how low the reservoir got over the weekend.
PID is programmed to hold the reservoir at 92% full. Once it drops to that level, PI will kick in.
This proves LIGO is not a hoax. Nobody could make this story up.
A story worth documenting! Can we warm up CP4 next? ;)
PI performance looks good. And there is no smoothing factor set. At 15% open on LLCV, rate of decline is 2.3% per hour given today's temperatures.
https://services.ligo-la.caltech.edu/FRS/show_bug.cgi?id=8265