Shut down purge air station and booster pump. Left compressor energized. Closed WP 6026.
I raised CP3's LLCV from 17% to 18% since it took 16 min. to overfill yesterday.
HAM 6 N & S doors and gauge assembly are leak tight. Attached is pump down curve. We will open GV 5 & 7 after the ISCT6 table is positioned and all viewport work is complete.
Due to the failure of the fast shutter system on WHAM6 and the damage done to the OMC, we will be operating under the following conditions:
The input power is to be kept to less than 25 W.
Tests of the fast shutter system shall be made on a regular schedule to verify its readiness.
This exceptional condition of operation will remain in place until a TRB (technical review board, reporting to Systems) has reviewed the risks to the OMC, the failure mechanisms, and the methods to deal with them (both administrative and engineering based). The TRB will then issue a return to normal operations order. We expect and hope that these issues can be resolved quickly and commissioning will resume at the 50 W power level.
Adjacent pump port volumes isolated after Turbo at full speed, Aux. pump carts at HAM5 and HAM6 annulus pump ports pumping combined annulus volume in parallel. Corner Station vent/purge air supply left running in the event that final leak testing tomorrow results in a needed vent cycle to fix leak. Also, Y-end RGA bake cycle completed (maybe get background scans tomorrow - crisis permitting!) 1840 hrs. local -> Leaving site now.
Web view of vacuum MEDM screens has been restarted to pick up the modified screens.
Wow, that pumped down fast! Current pressure at 9:20pm is 1.5e-4 Torr. Love the new web based MEDM screens! New PT-110 gauge readout is not updated on MEDM screenshot.
Chandra's "Wow!" comment above only makes sense considering my "1840 hrs. local -> Leaving site now". In fact it was "2040 hrs". Those two extra hours makes this pump down "typical"
Ah, ok - normal 3 hr roughing pump down till we transition to turbo.
Gerardo, Kyle, Chandra (and Bubba on doors) Started pumping down HAM 6 with rough pump at 5:10pm local. Kyle graciously offered to stay late until system can transition to turbo pump. Thursday morning Chandra will leak check the new gauge assembly. Crew can connect ISCT6 table first thing in the morning. If all is well with leak check, gate valves can be opened in the morning following leak check. Expect pump noise for over a week from maglev turbo on top of HAM 6 and 2-3 aux carts - pumping on annuli and one at vertex RGA (between HAM 4 & 5). Thank you Bubba for helping install doors today! We spotted a new nick on north door at 4 o'clock position in the path of the outer o-ring but leak checking confirmed there is no leak. We leak checked both north and south doors by pumping on the annulus volume. Dew point of chamber during blow down was -19.9 deg C. Gerardo & Kyle took some particle count measurements at south side after north door was sealed. They can report the #s. New wide range ion gauge was installed. Thanks Richard and Fil and others for running cables/power and rebooting for immediate monitoring!
Chandra rightly pointed out that the procedure we actually use when pumping HAM1 & HAM6 is sufficiently unique as to require their own distinct procedure (" I'll get right on that Chandrohn! ")
Dust counts:
Took a reading before entering cleanroom with door, counts remained at 0, another reading with the door inside the cleanroom with counts still at 0. removed both covers (door and chamber) and took a third reading with 9 for 0.3 um and 4 for 0.5 um.
[Jenne, Stefan, Peter, TJ]
We now have a new guardian node, FAST_SHUTTER, to monitor the function of the fast shutter. So far this does not interact with the PZT shutter.
Attached is the guardian graph of how it works for now.
So far, this is not incorporated into the main ISC_LOCK guardian. However, ISC_LOCK Down should probably request Fast_Shutter's Down. Somewhere around DRMI_LOCKED we should have the ISC_Lock guardian request Ready from the FastShutter guardain. We shouldn't be able to continue locking unless the FastShutter says that it's Ready. Somewhere like DC Readout, we should put the FastShutter guardian in IFO_Locked.
We hope to be able to test the functionality of this new guardian tomorrow, after ISCT6 is put back in place.
I noticed that an uknown script constantly "opens" (i.e., unlatches) the trigger logic for the AS port protection shutters. This is probbaly not a good idea, since it can confuse an operator. If we need the auto-unlatch feature, it should be added to the shutter control in a transparent way.
The intended design of this shutter controller, (called a trigger reset controller in the beckhoff) is to reset itself.
The new HAM6 vacuum gauge channels were added to H0EDCU_VAC.ini and the DAQ was restarted.
We took this opportunity to configure h1fw0 to write trend frame files again. This made fw0 very unstable, crashing before it could write out a second trend file (on the 10-minute mark). I reverted back to fw0 only writing the 64second frame files and it became stable again.
here is the current configuration:
fw0 writing only science and commissioning frames. In past 14 days has not restarted itself.
fw1 writing all frames (sci, com, sec, min). In past 14 days has restarted 3 times.
Jonathan is working on adding further diagnostics to daqd to help with this investigation. For now we will keep with this stable configuration, we have zero full frame loss and minimal trend frame loss.
WP 6073 I updated h0vaclx to change PT110 to a BPG 402 Inficon gauge. This required restarting the TwinCAT3 software and the EPICS IOC. I burtrestored the IOC to midnight yesterday. I also took the opportunity to enable and set the low trip point for PT110, PT170 and PT180 to 1E-5: (800E:02) Low Trip Point Enable TP1: TRUE (800E:14) Low Trip Point Limit TP1: 1E-5 (800F:02) Low Trip Point Enable TP2: TRUE (800F:14) Low Trip Point Limit TP2: 1E-5 The following channels were removed: H0:VAC-LX_Y0_110_PIRANI_INTLK H0:VAC-LX_Y0_110_PWR_REQ H0:VAC-LX_Y0_HS110_PWR_CTRL H0:VAC-LX_Y0_PT110A_PRESS_TORR H0:VAC-LX_Y0_PT110A_PRESS_TORR_ERROR H0:VAC-LX_Y0_PT110A_PRESS_VOLTS H0:VAC-LX_Y0_PT110A_PRESS_VOLTS_ERROR H0:VAC-LX_Y0_PT110A_PRESS_VOLTS_SMOO H0:VAC-LX_Y0_PT110B_PRESS_LOGTORR H0:VAC-LX_Y0_PT110B_PRESS_LOGTORR_ERROR H0:VAC-LX_Y0_PT110B_PRESS_TORR H0:VAC-LX_Y0_PT110B_PRESS_TORR_ERROR H0:VAC-LX_Y0_PT110B_PRESS_VOLTS H0:VAC-LX_Y0_PT110B_PRESS_VOLTS_ERROR The following channels were added: H0:VAC-LX_Y0_PT110_ERROR H0:VAC-LX_Y0_PT110_INFODATA_STATE H0:VAC-LX_Y0_PT110_MOD1_OVERRANGE H0:VAC-LX_Y0_PT110_MOD1_PRESS_TORR H0:VAC-LX_Y0_PT110_MOD1_PRESS_TORR_ERROR H0:VAC-LX_Y0_PT110_MOD1_SENSOR H0:VAC-LX_Y0_PT110_MOD1_TRIP H0:VAC-LX_Y0_PT110_MOD1_UNDERRANGE H0:VAC-LX_Y0_PT110_MOD1_VALID H0:VAC-LX_Y0_PT110_MOD2_OVERRANGE H0:VAC-LX_Y0_PT110_MOD2_PRESS_TORR H0:VAC-LX_Y0_PT110_MOD2_PRESS_TORR_ERROR H0:VAC-LX_Y0_PT110_MOD2_SENSOR H0:VAC-LX_Y0_PT110_MOD2_TRIP H0:VAC-LX_Y0_PT110_MOD2_UNDERRANGE H0:VAC-LX_Y0_PT110_MOD2_VALID H0:VAC-LX_Y0_PT110_WCSTATE_TOGGLE H0:VAC-LX_Y0_PT110_WCSTATE_WCSTATE
In order to easily assess the fast shutter during operations, I added code to the DIAG_MAIN guardian that is watching for orm power drops, and if it finds one, it writes out a text file that can be ingested by the lockloss tool.
The medm button that calls up the plot is on ISC_GUARDIANS.adl for now.
Related files In /opt/rtcds/userapps/release/sys/h1/scripts:
===================================================
fastsutterlockloss : wrapper file that launches lockloss
channels_to_look_at_FastShutter.txt : lockloss channel file for the fast shutter
FASTSHUTTERTIMES.txt FASTSHUTTERTIMES_1.txt FASTSHUTTERTIMES_2.txt FASTSHUTTERTIMES_3.txt : files containing the timing information for the last four arm power drops.
Related code in DIAG_MAIN:
==========================
# this check finds an arm power drop, and creates a text file for plotting the fast shutter performance
fastshutterMemory = {
'LOCKED' : 0,
'ARM_ON' : 1000,
'ARM_OFF' : 800,
'TIME_LAST': 0}
@SYSDIAG.register_test
def HAM6FASTSHUTTER():
armon =fastshutterMemory['ARM_ON']
armoff=fastshutterMemory['ARM_OFF']
lockedpast=fastshutterMemory['LOCKED']
timelast=fastshutterMemory['TIME_LAST']
arm=ezca['LSC-TR_X_NORM_INMON']
timestr=gpstime.tconvert()
if arm > armon:
locked=1
elif arm < armoff:
locked=0
else:
locked=lockedpast
# upon dropping arm power: record time window for plotting
if locked==0 and lockedpast==1:
dur=int(float(timestr)-float(timelast)+1)
filefs='/opt/rtcds/userapps/release/sys/h1/scripts/FASTSHUTTERTIMES.txt'
filefs1='/opt/rtcds/userapps/release/sys/h1/scripts/FASTSHUTTERTIMES_1.txt'
filefs2='/opt/rtcds/userapps/release/sys/h1/scripts/FASTSHUTTERTIMES_2.txt'
filefs3='/opt/rtcds/userapps/release/sys/h1/scripts/FASTSHUTTERTIMES_3.txt'
os.system('mv '+filefs2+' '+filefs3)
os.system('mv '+filefs1+' '+filefs2)
os.system('mv '+filefs+' '+filefs1)
f = open(filefs, 'w')
f.write('plot -w ['+str(-dur)+',1] '+timestr+'
')
f.close()
yield "New Fast Shutter plot availabe!"
fastshutterMemory['LOCKED'] =locked
fastshutterMemory['TIME_LAST']=timestr
Ansel, Jim, Dave:
We upgraded the final timing systems. In the staging building we upgraded the Fanout, IRIG-B and IO Chassis units. We turned these systems off after we were done to save power.
The DTS fanout was the first to be upgraded, so it had the 118 version of the code. All the other fanouts were upgraded to 121 recently, and we installed this version in the DTS. Now the master is not complaining about a firmware error on port 8.
Title: 08/10/2016, Day Shift 15:00 – 23:00 (08:00 –16:00) All times in UTC (PT) State of H1: IFO is unlocked. HAM6 Vent work finishing up. Film crew on site. Commissioning: Outgoing Operator: N/A Activity Log: All Times in UTC (PT) 14:50 (07:50) Chris – Going to both Mids and both Ends 15:00 (08:00) Start of shift 15:05 (08:05) Krishna – In Biergarten working on Compact BRS 15:10 (08:10) Richard – Transitioned LVEA to Laser Safe 15:45 (08:45) Kyle – At End-Y for RGA bake out 15:45 (08:45) Jim W. & Hugh – Closing out SEI on HAM6 15:45 (08:45) Karen – Cleaning in the LVEA 15:49 (08:49) Richard – Working on phone access at HAM6 15:56 (08:56) Kyle – Back from End-Y 16:01 (09:01) Jim B. – Restart NDS1 (WP #6069) 16:02 (09:02) Corey – Going to HAM6 16:09 (09:11) Betsy – Going to HAM6 16:24 (09:24) Mike – Escorting film crew into LVEA (WP #6070) 16:34 (09:34) Filiberto – Working at HAM1/2 (WP #6065) 16:45 (09:45) Carlos – Going to End-Y for phone work 16:50 (09:50) Bubba – Going to HAM6 16:57 (09:57) Chris – Finished at Mids & Ends 17:00 (10:00) Apollo Mechanical on site for Bubba 17:04 (10:04) Gerardo – Going to HAM6 17:16 (10:16) Richard – Going to HAM1/2 18:02 (11:02) Richard – Going again to HAM1/2 18:14 (11:14) Koji – Going to HAM6 to clean up after vent 18:20 (11:20) Mike – Film crew out of the LVEA for lunch 18:52 (11:52) Koji – Out of the LVEA 19:03 (12:03) Richard & Filiberto – Out of the LVEA 19:29 (12:29) Mike – Escorting film crew 1/4 way down beam tube. Will look inside 19:45 (12:45) Bubba – Out of LVEA 19:49 (12:49) Jason – Escorting film crew into the PSL 19:52 (12:52) Gerardo – Out of the LVEA 20:30 (13:30) Kyle – Going to End-Y to adjust RGA bake out 20:31 (13:31) Bubba – Going to HAM6 20:34 (13:34) Travis – Going to HAM6 20:40 (13:40) Jason – Film crew out of the PSL 20:45 (13:45) Mike – Escorting film crew in the LVEA – May be using the crane 21:02 (14:02) Jeff K. – Running charge measurements at both Ends 21:10 (14:10) Betsy, Calum, Travis – Out of the LVEA 21:29 (14:29) Kyle – Back from End-Y 21:34 (14:34) Richard – Going to HAM1/2 21:58 (14:58) Richard – Out of the LVEA 22:10 (15:10) Jeff K. – Finished with charge measurements 22:45 (15:45) Mike – Film crew out of the LVEA – Filming in the control room 23:00 (16:00) CDS doing DAQ restart Title: 08/10/2016, Day Shift 15:00 – 23:00 (08:00 – 16:00) All times in UTC (PT) Support: Incoming Operator: N/A Shift Detail Summary: IFO is unlocked due to HAM6 vent. Doors are back on HAM6. Pump down is underway.
Krishna
I've attached some plots showing the tilt measurements with Compact-BRS. The calibration is only approximate right now (+/-50%). More accurate calibration will happen soon.
Quick Tutorial:
Recall that the instrument consists of a beam-balance with two fiber interferometer readouts at left and right ends of the balance. The fiber-tips are located on a translation stage driven by a PZT stack each (100V, 11 micron range). Photodiodes (PDs) measure the output of the interferometers. The PZTs are used in a low-frequency feedback loop to keep the PDs at mid-fringe. The low-frequency tilt signal is therefore in the PZTs, while the high frequency seismic tilt is in the PDs. The raw displacement signal is converted to angle by dividing by the arm-length (roughly 15 cm from the center). The beam-balance is also damped with capacitive actuators using the PZT control signals.
The fiber interferometers are operated with a large gap (using a collimating lens) and thus are individually limited by the frequency noise of the common laser. However, taking the difference of the PD signals allows us to subtract frequency noise to the extent that the cavity lengths are matched. Crudely,
PD 1 = Angle + L1 * Frequency noise
PD 2 = -Angle + L2 * Frequency noise
Therefore,
SUM = (PD1+PD2)/2 = Frequency noise*(L1+L2)/2
and DIFF = (PD1-PD2)/2 = Angle + Frequency noise*(L1-L2)/2
The attached plots show data measured this afternoon. The first plot shows the PZT and PD signals for each cavity. After the flexure replacement on Monday, the instrument is slowly settling but is continuing to drift in one direction as seen from the plot.
The second plot shows the ASD of these four signals. As explained above, the tilt at higher frequencies is in the PDs and the low-frequency tilt is in the PZTs.
The third plot shows the ASD of the SUM and DIFF channels. Note the smooth 1/rt(f)-ish slope in the SUM channel (at low frequencies) indicating that it is limited by the frequency noise, as expected. Unfortunately, there appears to be excess noise above 10 Hz, which look like acoustic pickup of some sort. This was taken during a noisy LVEA, so it is possible that after the clean room fans are turned off and it gets quieter, the noise will go down. The DIFF, does dip below the SUM as expected - indicating that we are getting some frequency noise suppression. It is not clear what the noise floor of the DIFF channel is yet. IF there is a factor of ~3 frequency noise suppression (at a minimum), then the noise floor ought to be a factor of 3 below the SUM channel (~ 30 picorad/rt(Hz) at 10 Hz). But if we are limited by acoustic noise, then the noise floor could be worse.
The fourth plot shows coherences between some interesting channels.
Compact BRS is performing reasonably well at the moment. Stay tuned for more data/plots when things are quieter. I'll also try to compare c-BRS with other local sensors.
I've attached some spectra recorded last night from 9:30 to 12:30 pm. I've also included the nearby STS-2 seismometer (ITMY STS) in veolcity units.
First plot shows the low-frequency spectra, which is very noisy below the resonance. This is likely all temperature noise and was expected as the instrument is not in vacuum. However, this is irrelevant for the high frequency sensititivity.
The second plot shows the DIFF (actual floor tilt) and SUM (~frequency noise) spectra along with the STS-Z. The floor of the DIFF spectra dips down to ~ 50 picorad/rt(Hz) at ~2.5 Hz and the bump there seems to be real, as it is seen in the STS as well. Unfortunately, the fans were still not OFF yesterday so many high frequency peaks are visible in both instruments.
Found the FSS AUTOLOCK was oscillating. Could not adjust the gain due to Guardian control. Per Jason: (1). Took the Guardian node PSL_FSS to AUTO (2). Set the state to IDLE (3). Adjust the COMMON GAIN to 16dB (4). Set the state to READY_FOR_MC_LOCK Jason and TJ are looking into an elegant long term solution.
I did a few things today.
1) Updated the final FSS Common Gain to its new value of 16 (previously 20).
2) Rather than just throwing in the new gain when the FSS is no longer oscillating, it will make four steps for one second each to get there. If the FSS begins to oscillate while it is stepping, then it will start over when the oscillation is done.
I tested it a bit and all seemed well, but if there are any issues then just go back a revision.
[Corey, Betsy, Fil, Peter, Calum, Stefan, Koji]
Summary
- The ISC portion of the HAM6 vent work has been completed.
- Shield isolation of the in-vacuum cables was confirmed.
- The fast shutter was reinstalled on the table after some modifications.
=> Ready for the SEI mass balancing and other exiting procedures. OMC PZT HV is still on. Remeber to turn it off before pumping down.
Some details
- Shield isolation: It is always confusing to check the shiels isolation on HAM6 because of several reasons.
We initially had several cables shorting to the chamber but all of them but one happened at the slack of the cables right inside of the flange. The last one happened on the DB25 cable before it climb up to the vertical wall of the ISI.
- The fast shutter modification / reinstallation
(Photos are supposed to come later.)
Photos From Yesterday's Fast Shutter Work
Photos are on Resourcespace here:
https://ligoimages.mit.edu/?c=1705
Cable Grounding Check
(Corey, Fil, Koji)
Documents of interest:
We went through as many of the ISC/SUS cables as we could, which is mainly all the DB25 connectors (most were on a single flange, D6). The non D6 cables [i.e. SUS OMC & WFS Heads] were checked at the CDS rack (east of HAM6) & the SUS rack (south west of HAM6). We did not check the RF WFS cables at D5 (not sure how to make check on these connectors, and Koji was worried about disconnecting the connectors since it could make things worse. We also did not check SEI since those cables were not touched and are nicely clamped & out of the way.
Some notes:
OMC Cables (D6: F1, F2, & F3): These cables all go to a harness on the OMC breadboard. Their sheilds are all tied to each other. Since we checked these cables at the flange, You must disconnect all three cables at the same time. Then you can do the ground loop check (otherwise, if you do one at a time, you will trick yourself into seeing ground loops, but this is because their sheilds are all connected).
For the Picomotor cables, I thought Koji said we should disconnect the cable from the picomotor and then check for grounding, but I can't remember if that is really necessary.
Bird's Nest!: We found some shorts. When one finds a short, the job is to go in chamber and then "wiggle" the cable in question until you no longer see the short. I believe we were able to do this in-chamber right next to the D6 flange. This is where there is a "bird's nest" of ISC cables (unfortunate...I reckon we could clean this up by carefully clamping ISC cables down to HAM6's Stage0.).
At any rate, whenver one fixes a cable with a ground loop, one must then re-check all cables! This is because all these cables are in this "bird's nest". And if you remedied one by moving it around, you don't know whether you made things worse for a neighboring cable in the "nest".
At the end of the day, we were happy with our ground loop checks for these cables.
OH, a To Do Item:
At the D6 Flange, it was easier for Fil to plug/unplug cables when he removed some protective bars were removed from the flange protector. At the nearest convenience, those bars should get re-installed.
Re the To Do Item--removal of the 'protective bars' aka strain relief, to make it easier to unplug cables. These should be replaced and returned to relieving strain by the remover. There is plenty of instances where this has not been done and I expect is the SOP, sadly. The job is always so much easier if you don't have to return to it later, after you've remembered it.
> For the Picomotor cables, I thought Koji said we should disconnect the cable from the picomotor and then check for grounding,
> but I can't remember if that is really necessary.
No. It turned out that the picomotors have no shields at the mighty mouse connectors and have no shorting the the table.
HAM6 CC wafer placement - it is now back in the same place as the last one was pulled from.