Peter, Ed
Investigation currently underway regarding potential water leak in HPO.
Peter & Ed are addressing it in the H1 PSL Room. Vern notified.
Peter said we will be down rest of today and possibly tomorrow. When TCSy work is complete, we could transition to LASER SAFE....but that will have to be after the active TCSy work.
This is FAMIS task #6109
FRS has been filed for the water leak: FRS#6063
J. Kissel This is your weekly assessment of charge accumulation, resulting effective bias voltage, and actuation strength change of the ETM ESD systems. The extremes of each test mass are now showing ~30 to 40 [V] of effective bias voltage, which is an actuation strength change of ~10% from nominal. ETMX's quadrants are all consistently at +30 [V]; ETMY's quadrants are less consistent, with some showing as little effective bias voltage as -10 [V]. Sadly, we've still not fully recovered the IFO since ER9 so we've still not yet been comfortable with the lock acquisition sequence to commission regular bias flipping (or even flipping it once), but let's hope we can get there soon! (I encourage the commissioning vanguard to not wait for me to be present to try...)
Required climbing on HAM5/HAM6. Two pump carts left running in LVEA for rest of week, one at South door of HAM5 and the the second on North side of tube near HAM4.
Digging into the HAM6 ISI and shutter issues, I found that the "high" isolation filters had very low phase margin in DTT open loop measurements, something like ~10 degrees or less. These have not been changed since before O1, so either something has changed the plant or we've been living on the edge. The Matlab commisioning scripts say the installed loops should have had more like ~20 degrees. I've done a quick adjustment on them, and now the DTT measurments show more like 20-30 degrees. First attached plot shows the X DOF DTT OL measurments. Blue is before, red is after UGF is now about 32 hz and phase margin is about 22 degrees.
I don't think this will have large effect on the performance of the ISI, the difference in gain at 1 hz is a factor of <10, but it went from 10000 to ~ 1000, the gain at 10 is different by less than a factor of 2. The attached pdf shows the matlab commissioning plots for X, Y and RZ, the first 3 pages are the old loops, last 3 pages are the new. Pages 1 and 4 are for the X dof, which should be compared to the first image. Still more digging to be done, I'm not sure why the commissioning scripts are off so much.
The shutter still saturates the actuators, but I've triggered the shutter a half dozen times this morning and the actuators get ~3000-5000 saturations each time it shuts. There are a few hundred more when the shutter resets, so we can probably ride out one cycle, before clearing the saturations, which I think the ISC_LOCK guardian does. I don't believe this fully avoids trips, yet, but re-isolating should be better in the event of a trip.
Subsystems
PNNL Tour this afternoon
H1 request granted for this morning:
| Work Permit | Date | Description | alog |
| 6092.html | 2016-08-16 17:41 | Activity: Reprogram the CPS fanouts to shutoff the LED. Area of Activity: SEI CPS FanOut | 29125 |
| 6091.html | 2016-08-16 17:20 | Activity: Upgrade tcioc to most recent version which supports TC3 (should be transparent for TC2) | 29131 |
| 6090.html | 2016-08-16 17:15 | Activity: Install updated protection library (stat reset added, catch abort, improve lock loss stat and cleanup) Make sure all is svn checked Reboot machine to test clean boot | |
| 6089.html | 2016-08-16 14:59 | Activity: Pull signal cables from annulus ion pumps (IBM122, OBM136, IBM192 and OBM196) to the LY rack. Cables will be landed at both ends. Area of Activity: Vacuum, LY Rack, LVEA | |
| 6088.html | 2016-08-16 12:58 | Activity: Clean all condensate pans for the cooling coils. Vacuum all loose debris and scale and unplug the drains. Area of Activity: HVAC @ EX-EY-MX-MY and LVEA Supply Fans | |
| 6087.html | 2016-08-15 21:06 | Activity: Update GDS to version 2.17.4 for bugzilla 897, 936, and 952 for ramping excitations up and down. Area of Activity: GDS control room software | 29126 |
| 6086.html | 2016-08-15 19:52 | Activity: Pull in a DB25 cable for the readbacks of the TCS RF Amp. Cable will be run from TCS-R1 to ISC-C3. FRS 6025. Area of Activity: H1 TCS LVEA | 29133 |
| 6085.html | 2016-08-15 19:29 | Activity: Install a new copy of the daqd frame write software on h1fw0. This build of the frame writer includes some enhancements to help address stability. Issues and some expand diagnostic channels which may be put into the frame. | |
| 6084.html | 2016-08-15 17:21 | Activity: Install new ISS software to support testing of new outer loop chassis. Will require restart of all models to reallocate DAC chans. New binary IO card will be used. Fast chans added to commissioning frame. DAQ restart needed. Area of Activity: PSL front end software | 29139 |
| 6083.html | 2016-08-15 17:12 | Activity: Swap the CO2 laser on the TCSy table for a spare; align the new laser to the original beam path. The TCSy CO2 laser had a power drop from ~55W to ~40W. The RF driver was swapped and now the laser is only outputting ~15W, therefore we are now swapping the laser to return to ~55W operation. The TCSy table will be opened and a 50W Class IV CO2 laser will be operating, therefore this will require the LVEA to be in LASER HAZARD. | |
| 6082.html | 2016-08-15 15:59 | Activity: Start and run locally mounted turbo backed by pump cart and scroll pump at Vertex RGA * Bake RGA while isolated from Vertex volume and with calibration gas isolation valves open. Area of Activity: Vacuum, Vertex RGA, LVEA | |
| 6081.html | 2016-08-15 14:43 | Activity: Install new ISS servo. As planned at the moment install the new servo in the PSL racks. Extend the Photo Diode cable from the Ham chamber to the Rack. Connect the new servo to IO system. Make software changes as needed test. Area of Activity: PSL ISS outer loop Servo | |
| 6080.html | 2016-08-12 18:13 | Activity: Isolate and remove locally mounted turbo pump from RGA pump port * Install 1 1/2" O-ring valve in place of removed turbo * Pump out volume between pump port valve and O-ring valve - Close O-ring valve * Open 2 1/2" RGA isolation valve and expose RGA to Y-end volume * Energize RGA filament and start continuous scanning * Stop scanning at the end of the day and de-energize filament * Leave RGA exposed to Y-end volume (new "nominal" state). Area of Activity: Vacuum, RGA, Y-end | |
| 6079.html | 2016-08-12 17:33 | Activity: During Tuesday maintenace I will restart both servers to access the BIOS and setup the correct parameters on the IPMI to prevent DHCP requests. | |
| 6078.html | 2016-08-12 15:20 | Activity: Using a tripod, mount the Pcal ITM camera and optical tube in front of the dedicated viewport and capture an image of the ITM. Note that the housing had interference issues with the bellows on the adapter and modifications are being designed. Lexan protector will stay in place until all adjustments have been made. It will only be removed for the final photos after the setup has been secured. Lexan protector will be replaced immediately after capturing the images and the cover on the viewport protector will be replaced. Area of Activity: "Pcal" ITM camera, Xarm manifold, LVEA | |
| 6077.html | 2016-08-11 17:32 | Activity: Remove the lexan plate out of each of three view ports on the north side of HAM6 for aligning the optics in ISCT6. The lexan plates will remain out permanently. Area of Activity: ISC, north side of HAM6 | |
| 6076.html | 2016-08-11 16:59 | Activity: The ultrasonic Gust meter will be placed in the desert next to the wind fence at end-X. Cables will be pulled through the underground conduit from the outside to the inside of the VEA area and will be terminated into the electronics rack. | |
| 6075.html | 2016-08-11 16:57 | Activity: Opening GV 5 & 7 after HAM 6 vent & pump down. Area of Activity: VE, LVEA, GV 5 & 7 OPENING | |
| 6074.html | 2016-08-10 18:59 | Activity: Swap PEM power supplies in CER for Kepco JQE. This is to make room for the ±24V power supplies that will be used to power Beckhoff/Baffle PD Amplifier/ Spool X/Y camera ect. Pem power distribution chassis and microphones to be powered down. | 29135 |
| 6073.html | 2016-08-10 18:11 | Activity: Update and restart h0vaclx to change PT110 gauge pair to BPG 402 Inficon. DAQ restart required. | |
| 6072.html | 2016-08-10 16:00 | Activity: PNNL will be coordinating an OTDR test of the fiber pair that feeds the observatory's data connection to make sure that signal strength is appropriate for upgrading to 10Gb/s up from 1Gb/s. Wireless, phones, and internal wired networks at the observatory will be unaffected. Internet access will be interrupted during this work. Area of Activity: GC Internet connection | |
| 6071.html | 2016-08-10 15:59 | Activity: Pull new shielded network cables for the GigE cameras on ISCT1. Work will require climbing over HAM 1 & 2. Install new patch panel on ISCT1 for new POP GigE camera. Area of Activity: H1 Gig E Camera | 29123 |
Title: 08/16/2016, Evening Shift 23:00 – 07:00 (16:00 – 00:00) All times in UTC (PT) State of H1: Relocking the IFO after maintenance Commissioning: Outgoing Operator: TJ Activity Log: All Times in UTC (PT) 23:00 (16:00) Start of shift 23:05 (16:05) Jason – Out of the LVEA 23:38 (16:38) Jason – Going into the LVEA to turn on the TCS-X laser 23:43 (16:43) Jason – Out of LVEA – TCS-X laser has been restarted 00:05 (17:05) Rick S. and crew – Out of the LVEA 01:18 (18:18) HAM5 ISI and SRM WDs tripped. – Perhaps commissioning activities 03:47 (20:47) Sheila & Terra – Going to ISCT1 to align camera 05:10 (22:10) Sheila & Terra – Out of the LVEA 05:15 (22:15) Jeff – Into the LVEA to check doors and run particle counts 05:25 (22:25) Jeff - Out of the LVEA Title: 08/16/2016, Evening Shift 23:00 – 07:00 (16:00 – 07:00) All times in UTC (PT) Support: Sheila, Jenne, Terra Incoming Operator: N/A Shift Detail Summary: The commissioners spent the shift working stability and robustness of the IFO. There were high particle counts recorded on both dust monitors (Beirgarten and at HAM6) and in the PSL Anti-Room starting at 09:30. The high counts were mostly in the 0.3um particle sizes. Elevated counts were not recorded in the PSL Laser room. Counts were tapering off by the start of the evening shift and by shifts end had returned to somewhat normal levels. Have not been able to determine the source, yet.
Terra, Sheila
Tonight we had trouble engaging the ASC again.
Losing optical gain in POP X
We rang up what we think is a PR3 bounce mode when engaging the ASC the same way as last night. We found that we could avoid ringing this mode up by keeping the PRC2 gain low (digital gains of -500). Right before the OMC damage/vent, the POP X path was reworked and the optical gain seemed suspiciously low.
Tonight we found that the optical gain has decreased even more. Terra changed the demod phase by dithering PR3 pit (500 counts to M3) and rotated the phase positive 65 degrees, (Q1, Q2, Q3, Q4 from 55, 53, 54, 51 to 120, 118, 119, 116 ) to maximize the signal in I (minimize the Q signal). The 2 attached figures show Terra's before and after OLG measurements (excitation gain of 50), both with Jenne's gain of -5000, showing a 10dB increase in optical gain which is about what we expected based on the dither amplitude change.
After optimizing the phase, we did not see the 28 Hz mode get rung up, but this seems to come and go because we also didn't see it yesterday. We quickly tried moving L2 on the POP X path, while watching the amplitude of the PR3 dither line in the POP X signal. We moved the lens about 4 inches closer to POP X and about 3 inches further away, and didn't find any location that had more signal for PR3 so we replaced it as we found it.
We are going to leave the IFO locked in DC readout 2 Watts with the request set to down so that it will not try to relock. The noise is bad as expected.
POPX whitening gain is 0dB but should be odd, see alog 26307. FRS 6057 filed.
The whitening gain on POP X was changed from a gain step of 7 (21 dB) to 0 (0dB) on August 12th. This whitening chassis has a problem and we must use odd gain settings, or else it will return an error and not set the gains equally on all quadratns, as Keita and Hang noted 26307
The change in gain probably happened during a beckhoff restart for the shutter code, but we could have been saved from this problem by SDF. I cannot find a record for these whitening chassis in any SDF table.
Also, this does not explain the drop in gain that Jenne saw, which happened before the whitening settings changed.
The stuck whitening gain bit is the LSB of the Q3 channel. In the past this was typically an indication of a cable problem (short).
Sheila Daniel Terra
Connected the AM laser to the POP X head, and saw that we have very similar response in the electronics to what Evan measured in 27069
we had 3.3 mW out of the AM laser with a whitening gain of 21 dB, used -40 dBm of RF drive at 45.501150 MHz. We saw about 600 counts on each quadrant (except quadrant 3 which had 350 counts and also the least amount of DC light because of way the laser was mis centered on the diode).
We saw that there are rather large offsets when we changed the whitening gain, so Daniel reset the offsets. The large offsets might have contributed to problems last night, along with confusion about the whitening gain.
Also, we remembered that a factor of 6.7 of the mystery gain loss was due to adding a beamsplitter and forgetting to comensate for it on July 11.
(Edit: Actually, Haocun and I did remember to correct for this gain change, we just compensated for it in the digital loop gain. )
So to summarize:
loops were intially commisioned with a whitening gain of 21, a digital gain of -21, a 1 Hz ugf, and electronics gain similar to what we have now. (late may)
Edit: loops were originally commisioned with a filter gain of -200 for pit, -0.1 in the input matrix, an analog gain of 21 dB, and the WFS head electronics performing in a way simlar to what we have now. This is when the reference that I think Jenne used was saved, and within a few days the pit input matrix was reduced by a factor of 2.
Edit: Around June 16th, we had difficulty staying locked when these loops were engaged, which was noted in the alog. Terra and I just looked at trends of the filter gains, and it seems like we also reduced the digital gain from -220 to -3.4 although this was not noted in the alog. This, together with the input matrix change explains most of the missing gain that Jenne found.
On July 11th I forgot to compensate for the beamsplitter causing a gain reduction of 6.7 that no one noticed.
On July 26th, Evan and Keita relocated POP X and Jenne noticed that the digital gain had to be increased by a factor of 250 (or 500 for yaw) to keep the ugf the same.
August 12th the whitening gain was reduced to 0 dB from 21 dB by mistake in a beckhoff reboot.
August 16th Terra and I noticed this further reduction in gain, which is explained by the whitening gain. We also changed the demod phase which increased the gain by about 10 dB. We checked that small movements of the L2 don't change the optical gain much, and moving it by a few inches can decrease the signal.
So, we are missing about a factor of 40 gain, which we cannot explain with electronics.
In the end only a factor of 2 of Jenne's gain change in unexplained. It seems that we have had stable high power locks with both the high gain and low gain settings for PRC2, so we can decide which we want to use. We also should have a factor of 3 increase in gain because of the phasing Terra and I did.
More complicated than that.
|
Whitening (dB) |
POPX digital gain before rotation |
Input matrix | PRC2_P_GAIN |
BS |
Overall gain relative to original |
alog | |
| Originally | 33 | 1 | -1 | -220 | none | NA | |
| May 24 ~1:02 | 33 | 1 | -0.05 | -220 | none | 0.5 | |
| Jun. 17 | 33 | 1 | -0.05 | -3 | none | 6.8E-3 | |
| Jun. 22 ~noon | 21 |
2.8 |
-0.05 | -3 | none | 4.8E-3 | 27901 |
| Jul. 11-12 | 21 | 2.8 | -0.05 | -21 | inserted | 5.0E-3 | 28324 |
| Jul. 27 ~4:20 | 21 | 2.8 | -0.05 | -5000 | inserted | 1.2 | 28666 |
No mystery optical/electronic gain reduction any more. Maybe a factor of 1.2 came from the rework on the table.
It's not clear to me why the PRC2 filter gain was reduced by a huge amount on Jun. 17 but I haven't searched through alog.
Typo in the above table, originally the input matrix was -0.1, not -1.
PSL model changes. WP6084
Ben, Jim, Dave:
Three psl models were changed today:
h1ioppsl0: Had the new Contec 6464 binary IO card added.
h1pslpmc: Removed its grounding of the last four DAC channels, these channels will now be used by the new outerloop system
h1psliss: added the new outerloop test code for this week's chassis install and test by Ben. Current code just monitors the ADC input channels and writes them at 2kHz to the commissioning frame. I permits the four DAC channels to be driven by filter modules, it provides binary control and monitoring of the eight BIO channels.
I have created a simple MEDM and attached itto the site map (PSL -> 'ISS BEN TEST').
We anticipate more h1psliss code changes as this commissioning progresses during the week.
New GDS code
Jim:
please see Jim's alog.
Reboot h1guardian0
TJ, Dave:
h1guardian0 was rebooted after running 29 days. The loading came down a bit but starts off in the mid-20s.
New digital video camera
see today's alog for details
Wind gust meter attached to h1pemex
Jim W, Jim, Fil, Richard, Dave:
LHO's Young ultrasound 3d anemometer was attached to three spare ADC channels on h1pemex. The channels are ADC_0_12,13,14.
Frame Writer Code Testing
Jonathan:
New daqd code was tested on h1fw0. The new code has been left running.
Beckhoff C1-PLC1 code change
Daniel:
please see Daniel's alog
DAQ Restart
Daniel, Dave:
DAQ was restarted to sync to new h1psliss channels and the latest Beckhoff h1ecatc1plc1.
Ben has created a ISS OUTERLOOP medm. It is attached to the SITEMAP by navigating PSL -> 'ISS OUTERLOOP'
After running a low battery shutdown test, dust monitor #7 has been removed from the LVEA.
Isolated pump carts from HAM5/HAM6 annulus system but leaving carts running. AIPs were started earlier in the day and pump currents look low enough as to not need assistance -> Will shut down 2 of 4 running pump carts in the morning if not needed. Other pump carts will need to run until Friday morning judging from current data
TITLE: 08/16 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Aligning
INCOMING OPERATOR: Jeff
SHIFT SUMMARY: Pretty busy mintenance day. Much more than I expected at least. TCS crew is still working on diagnosing TCSY, commissioners are trying to lock without disturbing them too much.
LOG:
PEM power supplies in the CER mezzanine were swapped out for Kepco model JQE supplies. This model is a 1/4 rack version, and allows for four power supplies to be mounted on one shelf. This is to make room for the ±24V power supplies that will be used to power Beckhoff/Baffle PD Amplifier/ Spool X/Y camera ect. All PEM instrumentation microphones, magnotometers, ect. were powered down from around 10:30 am to 1:00 pm. All power is now restored.
Fil, Terra
I did a quick check of power spectra for all corner station mics and mags, before and after power supply swap. Attached spectra show pre-swap in darker colors as references, post-swap in lighter.
Found post-swap saturation in several mags, strongest in MAG_LVEA_INPUTOPTICS (first attachment at 21:03 UTC ). Fil switched this mag to battery power, which has temporarily fixed the problem (second attachment at 23:03 UTC). He took a look at the mag box in the lab but found nothing wrong. We left it battery powered for now. Will investigate more tomorrow.
Additionally, it looks like MIC_LVEA_HAM7 got disconnected around 23:03 UTC (top right of second attachment).
Channel 2 of AA Chassis S1300104 found to be bad.
MIC_LVEA_HAM7 started negatively railing yesterday ~3 hours after the power supply swap, trend attached (first drop then raise is from power supply swap). Turns out that AA channel has gone bad (bottom most AA chassis in PEM rack). Fil will pull it and have a look tomorrow morning.
AA chassis repaired, HAM7 mic back up and running. MAG_LVEA_INPUTOPTICS was also fixed. At this point, all corner station mics and mags are running well. Attached are power spectra for all.
The PSL power watchdogs tripped at 20:38:13 UTC (13:38:13 PDT). The cause appears to be the NPRO turning off. In the attached trend the NPROOK channel, which monitors if the NPRO is working or not. goes to zero (indicating the NPRO turned off) at 20:38:08 UTC, 5 seconds before the power watchdogs trip. The cause of the NPRO turning off is currently unknown. I am bringing the PSL back up.
PSL is now back up and running; came up without issue. Unknown at this time what caused the NPRO to shut down, will investigate. I reset the injection lock reset counters. Filed FRS 6060 to document the NPRO shut down.
The OMC alignment servos keep turning on, even though the OMC guardian is paused. I found that it gets turned on in engage_wfs_centering whenever we do any ASC since it's in a generate states function. So, I have commented out that line, but we must uncomment it when we have the OMC back and are ready to use it again. It is currently line 113 in ISC_GEN_STATES.
Maybe this should have a work permit to make sure this isn't forgotten; or at least some mechanism to undo the change in a timely manner.
Not sure when it happened, but looks like this has been restored to normal & uncommented (probably some time after the HAM6 work last week).
Making a note in Operator Sticky Notes wiki.