Here is another snapshot of Y-arm pressures over five days. We don't see a response in Y2 from changes in CP4 temperature, but Y1 side trends closely with CP4. GV11 (nearer to mid-station) has an outer o-ring leak in gate, but we evacuated this annulus and valved out. GV12 (Y2 boundary valve) has both inner and outer o-ring leaks on gate - also evacuated and valved out.

When BSC6 was replaced with spool at mid-Y several years ago, GV11 was exposed to air and the spool was not baked after installation (was it baked at factory?), so we expect outgassing from these components during the CP4 bake, but the sharp cuffs in Y1 pressure are not what we would predict with temperature changes in steel.

CP4 air temp trend over five days in enclosure is also attached.

Kyle and I replaced a bad solenoid valve on the compressed air drying tower skid outside after getting alarms 'after hours'. Loss of compressed air caused the safety valves on the three turbo stations to close and thus spin down turbos. It also caused GV 6,8 to sag. After we regained compressed air we spun turbos back up, but YBM turbo is being difficult and keeps shutting itself down due to vibration (a known issue). We tried all the tricks with bypass valve and air admittance to load rotors. We'll let it spin 100% down tonight and try again tomorrow. I closed all three turbo isolation gate valves in a timely manner so the corner should not have been affected from back streaming. Pressure rose for a period while turbos were valved out. I didn't have time to check on in-vacuum high voltage equipment - don't think anything was turned on.

FYI: we did not replace the solenoid portion of the valve at the drying tower.

The guardian upgrade has been completely aborted, due to the unexplained segfaults described in elog thread 40765 .  The entire guardian system is back to the exact same configuration as it was two weeks ago (guardian 1.0.3, h1guardian0 ubuntu12).  All nodes are up and running nominally.

h1guardian1 has been left in place to run demon excision tests: 20 fake guardian nodes are running under valgrind in the hope of catching a crash that would point to the problem.  These tests should not affect interferometer operation in any way.

In our hunt for issues, we swapped a BOSEM on ETMY Side R0 (reported below).  The BOSEM wasn't the problem tho.

OLV has been updated in medm:

• 08-03-18 15:48 JeffB in optics lab
• 08-03-18 16:01 Bubbas contractor for pump room work arrived - red truck
• 08-03-18 16:09 Richard to CER
• 08-03-18 16:49 Terry to HAM6, ISCT6
• 08-03-18 16:49 Richards contractor for the card access system arrived
• 08-03-18 17:14 JeffB done in optics lab and cleaning area
• 08-03-18 17:17 Karen done with cleaning at EY
• 08-03-18 17:18 Christina done with cleaning EX
• 08-03-18 17:18 Ken at EY for card access work
• 08-03-18 17:50 Richards card access contractor at EX and MX today
• 08-03-18 17:56 Mark to CER for cosmic ray equipment
• 08-03-18 17:56 Niko to HAM6
• 08-03-18 17:57 Dave looking at python error on verbal alarm machine
• 08-03-18 18:01 Sheila to optics lab and HAM6
• 08-03-18 18:10 as of 18:10 we are having an earthquake and also moving guardian nodes
• 08-03-18 18:11 many guardian nodes are in fault but Jim beleives all ISIs are locked or in damped
• 08-03-18 18:16 Robert to EX
• 08-03-18 18:17 Robert to EX to take pictures
• 08-03-18 18:18 Jim and Mark to EY to lock the ISI
• 08-03-18 18:23 Chandra to MY
• 08-03-18 18:43 Travis and Betsy to EY
• 08-03-18 19:00 Karen to MY
• 08-03-18 19:16 Jim to EX to start Pcal baffle install
• 08-03-18 19:38 Karen done at MY
• 08-03-18 19:43 Jim back from EY
• 08-03-18 19:49 Richard to EY to talk to Ken
• 08-03-18 20:21 Betsy and Travis back from EY
• 08-03-18 20:24 Robert to EY to take pictures
• 08-03-18 20:32 TJ, Sheila, Terry, Gerardo out of LVEA and HAM6 for lunch
• 08-03-18 20:45 Jim, Mark, and Niko, to EX to work on Pcal baffles
• 08-03-18 20:52 Dave and Fil hard cycling EY SUS, optic and TMS in damped
• 08-03-18 21:38 Richard and Fil swapping binary card at EY, after power cycle didnt fix
• 08-03-18 21:38 Kyle to EX to check on purge air, turn up
• 08-03-18 21:41 Chandra to MY
• 08-03-18 21:49 Dave testing a CS to EY connection, ETMY pitch
• 08-03-18 21:57 Terry heading to HAM6
• 08-03-18 22:01 Fil and Mark to MX to pull cable for new access system
• 08-03-18 22:05 Kyle back from EX
• 08-03-18 22:10 Matt to LVEA to take a picture
• 08-03-18 22:12 Nutsinee to HAM6
• 08-03-18 22:14 Dave has restored EY
• 08-03-18 22:17 Mark testing a table on HAM4
• 08-03-18 22:26 Matt to MY to look for IO 3IFO parts
• 08-03-18 22:32 Sheila to HAM6
• 08-03-18 22:45 TJ to HAM6
• 08-03-18 22:46 TJ to HAM6
• 08-03-18 22:53 Matt back from MY
• 08-03-18 22:56 TJ back from HAM6
• 08-03-18 23:08 Richard to LVEA with contractors
• 08-03-18 23:17 Roberts back from EY
• 08-03-18 23:54 crew still at HAM6
• 08-03-18 23:55 Jim and crew still at EX
• 08-03-18 23:58 Richard's contractor are probably still on site at MX

With the connecting and disconnecting the cables in  hunting and fixing ground loops at EY the side coil on the RO failed to drive the suspension.  Investigating it was obvious we had no coil connected. (Could not see 40 Ohms on the pins)  Check the feed through and everything appeared good and tight but it must have been seated oddly.  Reseated and tightened the connection and the coil was fine.  40 Ohms at he Sat amp 45 at the rack.

Fil and I also took this opportunity to replace the Binary IO card that had failed for the read back.  All signals seem to work now.

all are aligned except ETMX and TMSX, which are damped

Adjusted CP4 Dewar pressure relief valve (aka economizer valve) by turning screw 2 turns clockwise to increase head pressure. It was at 12 psig with tank 62.1% full. This may help increase GN2 flow. However, the PSI document specifies a head pressure of 5 psig for regen exercise. 

It will take a day or so for the head pressure to stabilize to new setting.

Dust6 appears to be off, JeffB checking

After ~3 days of ground loop hunting, fixing, breaking, fixing as well as a relocking of the ISI, we finally are getting good results on the 18 DOF transfer functions for all 3 suspended chains in BSC10 (ETMY main, ETMY reaction, and TMS).

Attached are results of the TMS.  Peaks from today overlap peaks from previous closeout sessions.

Kissel has cast an eye on these and calls them good.

Will post the ETM set shortly.

Next up:  First contact and electrometer optic readings, chamber clean-out, then closeup!

• HEPI: red EY cleared with Hugh
• Instrument Air: red at EY, EX, vent, MY, MX, historically red
• Annulus Ion Pump: red MY, bake
• Pressure: EY, EX, LX, vent

The h1susey computer was power cycled twice this afternoon. First to see if a power cycle of the IO Chassis would clear the binary error, the second to replace the Contec 6464 binary card. Even though h1susey had been taken out of the fabric, the first power up glitched the dolphin'ed models (h1seiey and h1iscey). This was not an issue since ISI was locked and the HWWD has the ISI coil drivers powered down.

After h1susey was working again I restarted all the other EY models with no problems.

2018_03_08 13:18 h1iopsusey
2018_03_08 13:18 h1susetmy
2018_03_08 13:18 h1sustmsy
2018_03_08 13:19 h1iopsusey
2018_03_08 13:19 h1susetmy
2018_03_08 13:19 h1sustmsy
2018_03_08 13:20 h1susetmypi
2018_03_08 13:45 h1iopsusey
2018_03_08 13:45 h1susetmy
2018_03_08 13:45 h1sustmsy
2018_03_08 13:46 h1iopsusey
2018_03_08 13:46 h1susetmy
2018_03_08 13:46 h1sustmsy
2018_03_08 13:47 h1susetmypi
2018_03_08 14:07 h1hpietmy
2018_03_08 14:07 h1iopseiey
2018_03_08 14:07 h1isietmy
2018_03_08 14:14 h1alsey
2018_03_08 14:14 h1caley
2018_03_08 14:14 h1iopiscey
2018_03_08 14:14 h1iscey
2018_03_08 14:14 h1pemey

Chandra, Dave:

the regen overtemp interlock alarm was added to the alarm system, this alarms with no latency. Also, the CP4 thermocouple low alarm level was raised to 70C.

Guardian:
    •    Reverted to previous config.
    •    BRS in CS has been removed, so is white
    •    New nodes to be added
    •    No work next Tuesday
CDS:
    •    Possible model reboots during Maintenance
    •    DAQ reboot
PSL:
    •    Alignment and mode matching for new 70W amp continues
HAM6:
    •    Locking
    •    Next week, smaller crew
    •    Ready for sei to start balancing
    •    Doors? Later
    •    Platform: Mark is working on it
    •    Chamber cover may need to be up without people in the cleanroom
LVEA:
    •    Kyle turning purge air off for about an hour – Tuesday Maintenance
EY:
    •    Card access work continues
    •    Card access contractors
EX:
    •    Card access work continues
    •    Card access contractors
    •    Pcal work started
    •    Kyle to check purge air
OpLevs:
    •    ITMY is blocked by GV1

GN2 flow survived the night (but did Kyle, who monitored the screen?). We now have an alarm enabled that will text the vacuum group almost immediately if the GN2 heater trips, in which case relatively cold GN2 would continue to flow through warm CP4, across bibraze joints. Our bake set up is different from the original PSI design, which I don't think was intended to run hot GN2 for days on end.

I increased the regen setpoint to 105C. Gas temp. was pretty steady at ~85C last night, with a setpoint of 95C and prop. gain of 7. There seems to be an offset (~10C) between setpoint and temp reading that I haven't converged on by changing the gain, so for now I increase the setpoint to make up for it.  The GN2 flow is ~ 20 scfhx100 and 1/4 of the vaporizer is frosted (see photo). Dewar consumption varies but doesn't look to be more than the typical consumption of normally operating pumps. May need to open up vaporizer feed valve more to allow more flow.

Continued with hunting of ground loops at EY. After various attempts all shorts for ETMY are now fixed. Some of the steps taken to remedy shorts:

1. Betsy loosen and re-tighten some of the connections at the feedthru (in chamber).
2. Shorts on the UIM and PUM were no longer present. Short on ETMY MO still present. New short on MO/RO.
3. Disconnected cables at feedthru (air side) and verified pin 13 was not shorted to ground inside chamber.
4. Checked that the shield and backshell on connectors were not touching (feedthru air side).
5. Disconnected all cables coming from chamber at the SUS Amplifier boxes.
6. Checked that all pins and backshells were not shorted to chamber ground. All passed.
7. Reconnected all cables back to SUS Amplifier units.

This morning, Travis had to repoint the ETMX with some bias to get it back on the Oplev which I found odd.  The Oplev had been zeroed to the ETMX SUS last ~Monday reportedly by Jason.  However, besides that jump in the trend data, there is another jump on Tuesday.  I am guessing that the ISI EX model work/bootfest changed the pointing of the floating ISI/SUS.  Travis used the ETMX SUS to repoint back to zero.

This morning, Jim locked the ISI so hopefully there will be no more unexpected shifts due to anything ISIish.

Side note - try not to bump the OPLEV piers.  Ever.