Sheila, Stefan,

We debugged Daniel's Beckhoff based AS port protection logic (more on that in Sheila's alog) and updated the guardian wrapper code to work with it.

MEDM screen description (figure 1):
AS Trigger PD (top left): Is the trigger PD (ASC-AS_C_SUM), as seen by Beckhoff within alowable power limits (i.e. there is power on the PD, but also not too much)
AS Trigger Logic (bottom left): Control of the fast trigger logic - feeds into the fast shutter and into the PZT shutter. (Note: manually closing this trigger actually fires the fast shutter.)
Fast Shutter (top middle): Control of the fast shutter (Note: manually closing the shutter here closes  the fast shutter SLOWLY.)
PZT Shutter (bottom mddle): Readbacks for the PZT shutter.
Protection (right): Logic and readback for the shutter status and testing.

The protection implements Daniels ECR E1600248 testing. More on that in SHeila's alog.
Note that the Beckhoff code cannot check the light on PDs downstream of the shutter. We indend to add a check of the downstream diodes into the FAST_SHUTTER guardian.

FAST_SHUTTER guardian:
This guardian is intended to implement the rest of ECR E1600248. Currently, it checks whether the AS_PORT_PROTECTION needs a test (Beckhoff readback), runs that test by poking Beckhoff if needed, and verifies that all the light levels are adequate before going to READY.

If any of the conditions are not met, it stops in SHUTTER_FAILURE and waits for human action.

The ISC_LOCK guardian will be updated to require the FAST_SHUTTER guardian to be ready before proceeding to full lock.
 

Items of ECR E1600248 still to be implemented:

1) Automatic check on lock-loss that the shutter worked as intended. If not, command PROTECTION_AS into Fault, and raise audio warning. This part however can only be implemented when seeing real lock-losses. So we intend to initially require a human to look at every lock-loss - the corresponding medm button is implemented - see alog 29001.

2) Augment the FAST_SHUTTER guardian to monitor the photodiode levels during the Beckhoff testing, and require that the downstream light is cut off and doesn't come back.

We triggered the fst shutter a couple of times by lowering the threshold (i.e. ensuring that the alanog electronics fires the shutter), and wee verified that we didn't see any bounce after 60ms to 100ms (as loged before in alog 28958 ).

Attached are plots for one trigger, on different time scales.

J. Kissel

I took an opportunity between shutter testing and beckhoff problems to gather the final at-vacuum measurements of the HAM6 SUS. I'll process the data in complete detail tomorrow (I attach a screenshot of an example DOF from the OMC), but the message is that the TFs look just as good as they did before the vent: free, clear of rubbing, and all resonances are at the same frequency with roughly the same Q. 

Excellent!

Watch this space for full analysis and comparison with previous results.

/ligo/svncommon/SusSVN/sus/trunk/OMCS/H1/OMC/SAGM1/Data/
2016-08-11_2305_H1SUSOMC_M1_WhiteNoise_L_0p02to50Hz.xml
2016-08-11_2305_H1SUSOMC_M1_WhiteNoise_P_0p02to50Hz.xml
2016-08-11_2305_H1SUSOMC_M1_WhiteNoise_R_0p02to50Hz.xml
2016-08-11_2305_H1SUSOMC_M1_WhiteNoise_T_0p02to50Hz.xml
2016-08-11_2305_H1SUSOMC_M1_WhiteNoise_V_0p02to50Hz.xml
2016-08-11_2305_H1SUSOMC_M1_WhiteNoise_Y_0p02to50Hz.xml

/ligo/svncommon/SusSVN/sus/trunk/HTTS/H1/OM1/SAGM1/Data/
2016-08-11_2249_H1SUSOM1_M1_WhiteNoise_L_0p03to50Hz.xml
2016-08-11_2249_H1SUSOM1_M1_WhiteNoise_P_0p03to50Hz.xml
2016-08-11_2249_H1SUSOM1_M1_WhiteNoise_Y_0p03to50Hz.xml

/ligo/svncommon/SusSVN/sus/trunk/HTTS/H1/OM2/SAGM1/Data/
2016-08-11_2151_H1SUSOM2_M1_WhiteNoise_L_0p03to50Hz.xml
2016-08-11_2151_H1SUSOM2_M1_WhiteNoise_P_0p03to50Hz.xml
2016-08-11_2151_H1SUSOM2_M1_WhiteNoise_Y_0p03to50Hz.xml

/ligo/svncommon/SusSVN/sus/trunk/HTTS/H1/OM3/SAGM1/Data/
2016-08-11_2257_H1SUSOM3_M1_WhiteNoise_L_0p03to50Hz.xml
2016-08-11_2257_H1SUSOM3_M1_WhiteNoise_P_0p03to50Hz.xml
2016-08-11_2257_H1SUSOM3_M1_WhiteNoise_Y_0p03to50Hz.xml

Attached is an SEM (1500 volts) scan of the Y-end RGA following a multiday bakeout ( > 160C ) while still isolated from the Y-end and while being pumped by a ~60 l/s turbo which was bolted to the pump port (net pump speed 10 L/sec or less due to connection).  The suspect "dirty" Nitrogen and Krypton calibration gas isolation valves were open during the ramp up and heat soak portion of the bake but were then closed at the start of the cool down portion. The half of the calibration gas lecture bottles containing the pinched capillary tube was wrapped and heated along with the bottle isolation valves and the rest of the RGA components.  The calibration gas bottles were left isolated for this scan. 

Comments from vacuum folks?  

Introduction:  I investigated variations in seismic band levels with month of the year, time of day, direction (X, Y, and Z), and station (CS, EX, and EY). 

Methods:  Hourly trends from seismometers at the Corner Station (CS), End Station X (EX), and End Station Y (EY) were exported from DataViewer, and were analyzed using the R statistical computing and graphics environment.  Three years of hourly maxima were analyzed, 2013 through 2016. 

Results: 

0.1 – 0.3 Hz
This band is dominated by the microseismic peak caused by pressure exerted on the ocean floor by waves and deep-sea storms.  No significant differences in ground motion were observed in this band between hours of the day or days of the week, which is expected for the microseismic peak.  Average ground motion in December was about 4 times what occurred in summer months.  Based on the 50% in lock level for O1, ground motion in this band contributes to difficulties maintaining lock in up to 40 times as many hours in December as in summer months.  This difference is attributed to an increase in ocean storms in the Pacific during the winter months.  Hourly trends showed that ground motion in this band was about 10% lower on average at EY than CS or EX, whereas monthly trends showed EY between 10% to 15% lower than the other stations from December through April.    

0.3 – 1 Hz
Ground motion in this band was about 60% to 80% higher during winter months than summer months.  This annual pattern is much less pronounced than in the 0.1 – 0.3 Hz band, possibly due to the influence of wind that increases in spring and summer [1].  Between 9 a.m. and 5 p.m. local time, ground motion in this band increased by between 25% to 40%.  This diurnal pattern is much less pronounced than the 1 – 3 Hz band.   Hourly and monthly trends showed ground motion at EX was about 25% to 50% higher than EY and between 5% and 20% higher than CS, which may be due to its closer proximity to the vitrification plant construction at the Hanford site.  

1 – 3 Hz
This band is dominated by human activity, which can explain a clear diurnal pattern.  Hourly trends showed that ground motion in this band increased by 60% to 75% between 8 a.m. and 5 p.m. local time.  There may also be evidence of a swing shift.  The higher ground motion at EX compared to EY can be explained by its closer proximity to Hanford site activities, such as the ongoing vitrification plant construction.  

3 – 10 Hz
Hourly and monthly trends in this band showed that ground motion at EY was between 30% to 3 times higher than ground motion at EX, which is likely due to its closer proximity to traffic on Hwy 240.  Ground motion at EY was also about 30% higher in the y direction than the x and z directions, because Hwy 240 is in the y direction.  Ground motion at EY decreased by about 30% in January, which may be due to a decrease in transportation of agricultural products during winter months.  Ground motion at EY was higher than at CS throughout the year, reaching up to about 75% higher than CS in June.  While truck traffic on Hwy 240 is still the largest contributor in this band, it is a factor of 2 lower than iLIGO because of the resurfacing of Hwy 240.  We should be able to watch it increase with time.  A diurnal pattern consistent with human activity can also be clearly seen in hourly trends, especially at CS.  Hourly trends showed that ground motion at CS between 7 a.m. and 11 a.m was up to 1.5 times higher than EX and about 15% higher than EY.   

10 – 30 Hz
Hourly trends showed that ground motion in this band highest at CS and EY overall, however, CS was up to 1.5 times higher than EY between 7 a.m. and 3 p.m. local time.  Hourly and monthly trends showed EY was between 25% to 30% higher than EX throughout the year, except in January.  Monthly variations were very small, however, an increase of up to 30% was observed at CS in this band in March and April; EX and EY both saw a similar increase in June.  Ground motion in the z direction (vertical) was about four times as high as the horizontal directions (x and y) at CS, which is explained in more detail here (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=15819).

References:
[1] Vidrio, Margarita, Schofield, R. Statistics for 8 years of wind at LHO. Aug 2014.  https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=12996

Title:  08/11/2016, Day Shift 15:00 – 23:00 (08:00 –16:00) All times in UTC (PT)
State of H1: IFO is unlocked. HAM6 Vent work is complete. Pump down is well underway. PSL tripped off. 
Commissioning: Recovering IFO
Outgoing Operator:  N/A
 
Activity Log: All Times in UTC (PT)

14:45 (07:45) Chandra – Going to leak check new gauge on HAM6 
15:00 (08:00) Start of shift
15:00 (08:00) PSL is down. Notified PSL team
15:13 (08:13) John – Going into LVEA
15:13 (08:13) Gerardo – Going into LVEA
15:45 (08:45) Vacuum crew out of LVEA
16:18 (09:18) Koji, Kiwamu, Stefan – Going into LVEA to connect ISCT6 table
16:25 (09:25) Krishna – Going to Biergarten to work on compact BRS
16:35 (09:35) Filiberto – Going to HAM6 to connect ISCT6 table
16:46 (09:46) Kyle – Going to End-Y to scan RGA
16:50 (09:50) Chandra – Opened CB to shutdown HAM5/6 cleanroom. All cleanrooms ore off.
16:54 (09:54) Koji, Kiwamu, Stefan – Finished connecting ISCT6 table
17:00 (10:00) Kiwamu – Transitioning LVEA to Laser Hazard 
17:15 (10:15) Ed – Going to End-X to install cabling for wind sensor
17:26 (10:26) Kyle – Back from End-Y
17:46 (10:46) Apollo Mechanical on site to meet Bubba
17:47 (10:47) Filiberto – Finished with ISCT6 - Out of the LVEA 
17:50 (10:50) Vern – In the LVEA to turn on TCS Lasers
17:54 (10:54) Jason – Recovered Laser – Going into LVEA to check PSL UPS
18:15 (11:15) Vern – Power cycle PSL Rotation Stage EtherCAT chassis 
18:30 (11:30) Kiwamu & Koji – Alignment work on ISCT6
18:30 (11:30) Hugh – Going to HAM6 to unlock HEPI
18:47 (11:47) Ed – Back from End-X
19:05 (12:05) Hugh – Out of LVEA – HEPI is unlocked
19:30 (12:30) Koji & Kiwamu – Out of the LVEA
19:32 (12:32) Richard – Going to check TCS interlock
20:06 (13:06) TJ – Going into LVEA to recycle TCS laser interlock
20:35 (13:35) Vern & Richard – Going into LVEA to check TCS lasers
20:36 (13:36) Chandra & Calum – Going to Both end stations
20:37 (13:37) Kyle – Going to End-Y 
20:45 (13:45) Vern & Richard – Out of the LVEA
20:55 (13:55) Kiwamu & Koji – At ISCT6 for alignment work
21:17 (14:17) Cheryl – Going to ISCT6 
21:25 (14:25) Cheryl – Out of the LVEA
21:33 (14:33) Kiwamu & Koji – Finished with ISCT6 alignment
21:41 (14:41) Chandra & Calum – Back from end stations – Going into LVEA


Title: 08/11/2016, Day Shift 15:00 – 23:00 (08:00 – 16:00) All times in UTC (PT)
Support:  
Incoming Operator: N/A

Shift Detail Summary: IFO is unlocked. PSL down. Jason working on bringing the lase back up. Cleanup and close out work has finished at HAM6. Vacuum crew has reopened GV5 & GV7. LVEA transitioned to Laser Hazard. Laser are back up. Commissioners recovering IFO.   

WP 6077

Koji, Stefan, Kiwamu,

Today we put the ISCT6 table back into the final position, connected the light pipes and re-aligned the optics in the enclosure. The table is ready for locking.

[some details]

The table was put back in to the position where the table was before the vent with an eye-ball precision of a few mm. The table legs were subsequently secured to the ground, and the light pipes were connected to the view ports. In a single-bounce configuration, we were able to realign the optics on the table. The ASAIR beam was a bit off horizontally from the center by 1/4 inch or so at the upper periscope mirror, but since this is a 2" mirror, we decided not to improve it. We steered this mirror to fix misalignment in the downstream.

In contrast, OMC_R and _T paths needed the periscopes to move. For OMC_R, we shifted the entire periscope structure and all the optics by about 2". The beam coming out of the viewport was too high by an inch or so, so we slid the upper periscope mirror up. As for the OMC_T path, the periscope was shifted by an inch. The upper periscope was also moved up by an inch or so. We confirmed that the alignment of all the cameras are good.

Krishna,

The location of the c-BRS is shown in the attached map, indicated by the green '+'. The center of the instrument is very closely aligned with the center of the ITMX BSC chamber along the x direction and is located ~ 46 cm (18 inches) from the horizontal beam between the piers (seen a little below the green + sign).

c-BRS Locking Instructions

The attached screenshot shows the MEDM screen to control c-BRS. The control flow is described in T1600325. The two interferometer cavities are locked using the PZT1 and PZT2 filter boxes. The second cavity feedback signal is used to apply damping to the beam-balance. If the outputs of the PZT control signals (top right) look railed, this indicates that the beam-balance has drifted out of the range of the piezos. We can reset the range simply by locking to a different fringe as described below.

1. First, turn off damping, by setting the gain of the DAMP filter box to 0.

2. Open PZT1. Set Gain to 0. The ramp time should be set to 3 s.

3. Hit 'CLEAR HISTORY'. Now turn the PZT1 Gain back to 0.25. First cavity should be locked.

4. Open PZT2. Set Gain to 0. The ramp time should be set to 3 s.

5. Hit 'CLEAR HISTORY'. Now turn the PZT2 Gain back to 0.25. Second cavity should be locked.

6. Wait at least 30s and then set the DAMP Gain to 0.1 This sets the Q of the beam-balance to ~20, as opposed to the natural Q of ~150.

Note that if we know the direction of the drift, the Piezo offset can be set to take advantage of that fact. Currently the Piezo 1 drifts negative hence the Piezo 1 offset is set close to the positive maximum (2^17 ~ 130,000 cts) and vice versa for Piezo 2. I expect that this drift will go away in a few days time and then we will just have the normal daily temperature modulation, in which case the PZT offsets could be set to 0.

Krishna

Here are the recorded c-BRS associated channels and their calibrations (I think it is accurate to <20%):

1. H1:NGN-CS_CBRS_RY_PZT1_CTRL_IN1_DQ  :   PhotoDiode 1 output  :  1.6e-10 rad/count

2. H1:NGN-CS_CBRS_RY_PZT1_MASTER_OUT_DQ : PZT1 output  :  2.8e-10 rad/count

3. H1:NGN-CS_CBRS_RY_PZT2_CTRL_IN1_DQ  :   PhotoDiode 2 output  :  1.23e-10 rad/count

4. H1:NGN-CS_CBRS_RY_PZT2_MASTER_OUT_DQ : PZT2 output  :  2.8e-10 rad/count

I've also attached more spectra recorded this morning after the clean room fans were turned off. Many of the noise peaks went away, especially above 100 Hz, but some lines remain which are common to the seismometers and c-BRS. The DIFF signal above 100 Hz, drops to ~10 picorad/rt(Hz), indicating that it's noise floor is indeed well below the SUM channel. I expect that the current SNR at ~10 Hz is between 10-30. I've attached Matlab code showing the analysis.

Recent mods to the ETM ESDs (alog 28939) allow PI signals to drive even while ETM ESD is in the HV mode. I've edited the SUS_PI and ISC_LOCK guardians to account for this. Both reloaded successfully.

Previously the PI damping path for ETMs required the ESD be in LV mode == not be in HV mode. Because of this, we could not damp PI on ETMX until the ISC_LOCK state LOWNOISE_ESD_ETMY (as prior to this we are actively using ETMX ESD in HV mode for lock aquisition). PI damping for the 3 other test masses was turned on during DC_READOUT. ECR E1600230 now allows PI drive even when the LV driver is in HV mode. Now all damping turns on during DC_READOUT. 

J. Kissel

I had told the HAM6 team verbally (and even signed a piece of paper indicating) that all HAM6 SUS were freely suspended before the doors went on, but forgot to put in an aLOG about it. Mia culpa! 
 
Anyways, for the record: the in-air, after re-installation of the OMC shroud was complete, just before chamber close, transfer functions of H1 SUS OMC, OM1, OM2, and OM3 (still) look great. See attached .pdfs.

We need one more check after HAM6 is pumped down, and then we can declare victory.

Per E1600247 and ECR1600246, the following electronics were modified or relocated:

HAM6 shutter controller S1203609 was relocated from ISCT6 table (top) to rack ISC-R5, slot U2/3.

Changed R23 from 26.7K ohms to 12.4Kohms in the QPD transimpedance amplifier chassis S1301506. This was to increase the maximum threshold for the AS port trigger PD as seen by the shutter controller. See alog entry.

Cabling to ISCT table has been reconnected. The high voltage supplies for both the fast shutter and PZT are now power on.

PT-523 cold cathode vacuum gauge tripped around 10am. I restarted it from the controls console and then noticed PT-510 tripped. I restarted that one too. Looks like the pirani gauges spiked which caused the CC trip.

J. Oberling, J. Bartlett, P. King (from Pasadena)

Jeff came in this morning and found the PSL to be off.  Trends indicated the laser turned off at around 3:25 am PDT, with errors showing the flow was zero and the NPRO was turned off (no power watchdogs were tripped); the crystal chiller was found to be off, while the diode chiller was still running.  The crystal chiller was restarted and allowed to run for ~15 minutes and no glitches or errors were seen (last time we had a flow problem the chiller would glitch when just running by itself, no laser).

In discussion with Peter he suggested the possibility of the NPRO tripping (likely due to a power glitch or a glitch from the UPS).  When this happens the NPRO power supply, located in the PSL rack in the LVEA, needs to be manually re-enabled.  Upon inspection, the NPRO power supply indeed had to be manually re-enabled, which points to either a power glitch or a glitch of the UPS the NPRO is plugged into.  We don't think it's a power glitch as that would cause issues with other IFO systems, and this was not seen.  This leaves the most likely culprit at this time as a glitch from the NPRO UPS.

The laser is now back up and running, all stabilization systems enabled.  I took this opportunity to reset the HPO LRA back to its reference position and reset the relock counter for the injection locking system.

Adjacent pump port volumes isolated after Turbo at full speed, Aux. pump carts at HAM5 and HAM6 annulus pump ports pumping combined annulus volume in parallel.  Corner Station vent/purge air supply left running in the event that final leak testing tomorrow results in a needed vent cycle to fix leak.  

Also, Y-end RGA bake cycle completed (maybe get background scans tomorrow - crisis permitting!) 

1840 hrs. local -> Leaving site now.

[Jenne, Stefan, Peter, TJ]

We now have a new guardian node, FAST_SHUTTER, to monitor the function of the fast shutter.  So far this does not interact with the PZT shutter. 

Attached is the guardian graph of how it works for now. 

• Down is currently an empty state.  The only thing I can think to perhaps add here is re-requesting that the shutter be closed. 
• CheckReady is the big state that does most of the work.
  • If the Beckhoff logic says that we need to run the tests (either it's been more than 48 hours, or something is wrong), then execute the test script that's also accessible from the new medm screen.
  • If we don't "need" tests, then check several of the ready bits and error flags to make sure they're all clear.  Also check that we have some non-zero amount of light on the trigger photodiode (threshold currently set to 0.25, typical value during 2W DRMI lock is 0.3 or so).
  • If everything is fine, go to the Ready state
• Ready is an empty state like that of ISC_LOCK, to help make it clear that you've successfully completed the Check state.
• TestShutter executes the Beckhoff test script.  If the shutter passes the tests, the script should clear the TestNeeded EPICS bit, so that when TestShutter returns to the CheckReady state, it'll go through with the rest of the checks, and hopefully return Ready.  If the shutter fails the test, the guardian instead returns ShutterFailure.
• ShutterFailure should be impossible to get out of without human intervention.  It's an empty state, but it'll keep things stuck until someone checks on things.
• IFO_Locked is currently empty, but should be made to watch for locklosses.  This is the nominal state for Observing.
• Lockloss should run some small analysis to ensure that the shutter did successfully close.  If it did not close, this should return to the ShutterFailure state, and keep us stuck there until a human checks things out.  Otherwise, it'll return Down and be ready for the next acquisition sequence.

So far, this is not incorporated into the main ISC_LOCK guardian.  However, ISC_LOCK Down should probably request Fast_Shutter's Down.  Somewhere around DRMI_LOCKED we should have the ISC_Lock guardian request Ready from the FastShutter guardain.  We shouldn't be able to continue locking unless the FastShutter says that it's Ready.  Somewhere like DC Readout, we should put the FastShutter guardian in IFO_Locked.

We hope to be able to test the functionality of this new guardian tomorrow, after ISCT6 is put back in place.