Richard, Fil, Jim W, Jim B, Dave
The voltage outputs from the gust meter when it was installed at EX yesterday did not look correct and did not correlate to wind conditions. Today Jim W brought the unit back to the shop and we tested it with a fan. There was some variation in voltage output but it did not look correct when referencing the manual. Jim Bconfigured the system for RS232 serial communications and logged into the unit. The serial readbacks for wind speed, azimuth and elevation looked correct. Jim reprogrammed the three voltage outputs to be Speed, Azimuth and Elevation. We then set the system back to analog voltage mode, and now the outputs vary by many volts and are V1=speed, V2=azimuth, V3=elevation. We suspect the system lost its programming at some time and was perhaps outputting U,V,W analog voltages.
The unit is ready for re-install at EX.
Ben, Dave:
today I modified h1psliss to read the PD5-8 FAST and SLOW channels. On paper it looked like the h1psldbb model was not using the last eight channels of its ADC (the fourth adc, or ADC3). In practice the DB25 connector for the last eight channels of this AA chassis have a cable connected. (Note to team-psl, perhaps the model should be recording these channels). I had already made the model change and restarted the DAQ before we discovered this.
Ben suggests for plan-B we re-use eight of the ISS ADC channels for reading PD5-8. We are going to disconnect the production ISS cables from ADC0 chans 16-23 and replace them with the new ISS PD5-8 cables during tomorrow's test. Specifically ADC0 chans16-19 are PD5-8FAST, chans20-23 are PD5-8SLOW.
I've modified h1psliss, we will wait until tomorrow morning the restart the model and DAQ.
This morning Ed asked me to look at some trend plots he had just generated. Clearly noticeable was a sharp
rise in the humidity and a drop in the flow rates for the heads. I checked the crystal chiller and its
water level was below minimum and the controller displayed an error informing of such.
Attached is a picture of the burst hose attached to the head 3 water cooling circuit. This is the same
connection that failed ~2-1/2 years ago. I do not recall if it has been adjusted since that time. Although
care is/was taken to prevent twisting of the hose when the fitting is tightened, it is highly likely that
there was a small twist in the hose that exerted a small shear resulting in the tear seen in the picture.
Also attached is a picture of the 4f lens for head 4, clearly showing some water droplets on it.
Apart from the obvious step of replacing the hose, the turbine flow sensor for head 4 will also be
replaced. The laser will take the better part of a day to dry out - to be on the safe side. All optical
surfaces and the fibre bundle tips will need to be inspected for cleanliness before any operation of the
high power oscillator is contemplated.
(All times in PST)
H1's state this morning: DOWN. ITMx was swinging around, and Kissel cleared history for M0 Lock to calm it down.
Turned off locking servos for the night for ISS, FSS, & PMC.
LN2 at exhaust in 8 minutes
Raised LLCV from 19% to 20%
In preparation for the next CW hardware injection test, the /hinj/Details/pulsar/ subdirectories on h1hwinj1 have been updated. Here are the major changes:
CPS sensor checks for BSCs and HAMs - measurement taken around now-6hours, or about 14:30UTC today.
Nothing remarkable - spectras look good.
Nothing unusual to report. Typical noise at both end stations and data glitching at EY. Corner station pressure shows pump removal as described in Hugh's aLog. Corresponding control activity for this action is also notable in the plots.
Peter, Ed
Investigation currently underway regarding potential water leak in HPO.
Peter & Ed are addressing it in the H1 PSL Room. Vern notified.
Peter said we will be down rest of today and possibly tomorrow. When TCSy work is complete, we could transition to LASER SAFE....but that will have to be after the active TCSy work.
This is FAMIS task #6109
FRS has been filed for the water leak: FRS#6063
J. Kissel This is your weekly assessment of charge accumulation, resulting effective bias voltage, and actuation strength change of the ETM ESD systems. The extremes of each test mass are now showing ~30 to 40 [V] of effective bias voltage, which is an actuation strength change of ~10% from nominal. ETMX's quadrants are all consistently at +30 [V]; ETMY's quadrants are less consistent, with some showing as little effective bias voltage as -10 [V]. Sadly, we've still not fully recovered the IFO since ER9 so we've still not yet been comfortable with the lock acquisition sequence to commission regular bias flipping (or even flipping it once), but let's hope we can get there soon! (I encourage the commissioning vanguard to not wait for me to be present to try...)
Required climbing on HAM5/HAM6. Two pump carts left running in LVEA for rest of week, one at South door of HAM5 and the the second on North side of tube near HAM4.
Digging into the HAM6 ISI and shutter issues, I found that the "high" isolation filters had very low phase margin in DTT open loop measurements, something like ~10 degrees or less. These have not been changed since before O1, so either something has changed the plant or we've been living on the edge. The Matlab commisioning scripts say the installed loops should have had more like ~20 degrees. I've done a quick adjustment on them, and now the DTT measurments show more like 20-30 degrees. First attached plot shows the X DOF DTT OL measurments. Blue is before, red is after UGF is now about 32 hz and phase margin is about 22 degrees.
I don't think this will have large effect on the performance of the ISI, the difference in gain at 1 hz is a factor of <10, but it went from 10000 to ~ 1000, the gain at 10 is different by less than a factor of 2. The attached pdf shows the matlab commissioning plots for X, Y and RZ, the first 3 pages are the old loops, last 3 pages are the new. Pages 1 and 4 are for the X dof, which should be compared to the first image. Still more digging to be done, I'm not sure why the commissioning scripts are off so much.
The shutter still saturates the actuators, but I've triggered the shutter a half dozen times this morning and the actuators get ~3000-5000 saturations each time it shuts. There are a few hundred more when the shutter resets, so we can probably ride out one cycle, before clearing the saturations, which I think the ISC_LOCK guardian does. I don't believe this fully avoids trips, yet, but re-isolating should be better in the event of a trip.
Terra, Sheila
Tonight we had trouble engaging the ASC again.
Losing optical gain in POP X
We rang up what we think is a PR3 bounce mode when engaging the ASC the same way as last night. We found that we could avoid ringing this mode up by keeping the PRC2 gain low (digital gains of -500). Right before the OMC damage/vent, the POP X path was reworked and the optical gain seemed suspiciously low.
Tonight we found that the optical gain has decreased even more. Terra changed the demod phase by dithering PR3 pit (500 counts to M3) and rotated the phase positive 65 degrees, (Q1, Q2, Q3, Q4 from 55, 53, 54, 51 to 120, 118, 119, 116 ) to maximize the signal in I (minimize the Q signal). The 2 attached figures show Terra's before and after OLG measurements (excitation gain of 50), both with Jenne's gain of -5000, showing a 10dB increase in optical gain which is about what we expected based on the dither amplitude change.
After optimizing the phase, we did not see the 28 Hz mode get rung up, but this seems to come and go because we also didn't see it yesterday. We quickly tried moving L2 on the POP X path, while watching the amplitude of the PR3 dither line in the POP X signal. We moved the lens about 4 inches closer to POP X and about 3 inches further away, and didn't find any location that had more signal for PR3 so we replaced it as we found it.
We are going to leave the IFO locked in DC readout 2 Watts with the request set to down so that it will not try to relock. The noise is bad as expected.
POPX whitening gain is 0dB but should be odd, see alog 26307. FRS 6057 filed.
The whitening gain on POP X was changed from a gain step of 7 (21 dB) to 0 (0dB) on August 12th. This whitening chassis has a problem and we must use odd gain settings, or else it will return an error and not set the gains equally on all quadratns, as Keita and Hang noted 26307
The change in gain probably happened during a beckhoff restart for the shutter code, but we could have been saved from this problem by SDF. I cannot find a record for these whitening chassis in any SDF table.
Also, this does not explain the drop in gain that Jenne saw, which happened before the whitening settings changed.
The stuck whitening gain bit is the LSB of the Q3 channel. In the past this was typically an indication of a cable problem (short).
Sheila Daniel Terra
Connected the AM laser to the POP X head, and saw that we have very similar response in the electronics to what Evan measured in 27069
we had 3.3 mW out of the AM laser with a whitening gain of 21 dB, used -40 dBm of RF drive at 45.501150 MHz. We saw about 600 counts on each quadrant (except quadrant 3 which had 350 counts and also the least amount of DC light because of way the laser was mis centered on the diode).
We saw that there are rather large offsets when we changed the whitening gain, so Daniel reset the offsets. The large offsets might have contributed to problems last night, along with confusion about the whitening gain.
Also, we remembered that a factor of 6.7 of the mystery gain loss was due to adding a beamsplitter and forgetting to comensate for it on July 11.
(Edit: Actually, Haocun and I did remember to correct for this gain change, we just compensated for it in the digital loop gain. )
So to summarize:
loops were intially commisioned with a whitening gain of 21, a digital gain of -21, a 1 Hz ugf, and electronics gain similar to what we have now. (late may)
Edit: loops were originally commisioned with a filter gain of -200 for pit, -0.1 in the input matrix, an analog gain of 21 dB, and the WFS head electronics performing in a way simlar to what we have now. This is when the reference that I think Jenne used was saved, and within a few days the pit input matrix was reduced by a factor of 2.
Edit: Around June 16th, we had difficulty staying locked when these loops were engaged, which was noted in the alog. Terra and I just looked at trends of the filter gains, and it seems like we also reduced the digital gain from -220 to -3.4 although this was not noted in the alog. This, together with the input matrix change explains most of the missing gain that Jenne found.
On July 11th I forgot to compensate for the beamsplitter causing a gain reduction of 6.7 that no one noticed.
On July 26th, Evan and Keita relocated POP X and Jenne noticed that the digital gain had to be increased by a factor of 250 (or 500 for yaw) to keep the ugf the same.
August 12th the whitening gain was reduced to 0 dB from 21 dB by mistake in a beckhoff reboot.
August 16th Terra and I noticed this further reduction in gain, which is explained by the whitening gain. We also changed the demod phase which increased the gain by about 10 dB. We checked that small movements of the L2 don't change the optical gain much, and moving it by a few inches can decrease the signal.
So, we are missing about a factor of 40 gain, which we cannot explain with electronics.
In the end only a factor of 2 of Jenne's gain change in unexplained. It seems that we have had stable high power locks with both the high gain and low gain settings for PRC2, so we can decide which we want to use. We also should have a factor of 3 increase in gain because of the phasing Terra and I did.
More complicated than that.
|
Whitening (dB) |
POPX digital gain before rotation |
Input matrix | PRC2_P_GAIN |
BS |
Overall gain relative to original |
alog | |
| Originally | 33 | 1 | -1 | -220 | none | NA | |
| May 24 ~1:02 | 33 | 1 | -0.05 | -220 | none | 0.5 | |
| Jun. 17 | 33 | 1 | -0.05 | -3 | none | 6.8E-3 | |
| Jun. 22 ~noon | 21 |
2.8 |
-0.05 | -3 | none | 4.8E-3 | 27901 |
| Jul. 11-12 | 21 | 2.8 | -0.05 | -21 | inserted | 5.0E-3 | 28324 |
| Jul. 27 ~4:20 | 21 | 2.8 | -0.05 | -5000 | inserted | 1.2 | 28666 |
No mystery optical/electronic gain reduction any more. Maybe a factor of 1.2 came from the rework on the table.
It's not clear to me why the PRC2 filter gain was reduced by a huge amount on Jun. 17 but I haven't searched through alog.
Typo in the above table, originally the input matrix was -0.1, not -1.
PEM power supplies in the CER mezzanine were swapped out for Kepco model JQE supplies. This model is a 1/4 rack version, and allows for four power supplies to be mounted on one shelf. This is to make room for the ±24V power supplies that will be used to power Beckhoff/Baffle PD Amplifier/ Spool X/Y camera ect. All PEM instrumentation microphones, magnotometers, ect. were powered down from around 10:30 am to 1:00 pm. All power is now restored.
Fil, Terra
I did a quick check of power spectra for all corner station mics and mags, before and after power supply swap. Attached spectra show pre-swap in darker colors as references, post-swap in lighter.
Found post-swap saturation in several mags, strongest in MAG_LVEA_INPUTOPTICS (first attachment at 21:03 UTC ). Fil switched this mag to battery power, which has temporarily fixed the problem (second attachment at 23:03 UTC). He took a look at the mag box in the lab but found nothing wrong. We left it battery powered for now. Will investigate more tomorrow.
Additionally, it looks like MIC_LVEA_HAM7 got disconnected around 23:03 UTC (top right of second attachment).
Channel 2 of AA Chassis S1300104 found to be bad.
MIC_LVEA_HAM7 started negatively railing yesterday ~3 hours after the power supply swap, trend attached (first drop then raise is from power supply swap). Turns out that AA channel has gone bad (bottom most AA chassis in PEM rack). Fil will pull it and have a look tomorrow morning.
AA chassis repaired, HAM7 mic back up and running. MAG_LVEA_INPUTOPTICS was also fixed. At this point, all corner station mics and mags are running well. Attached are power spectra for all.