In the past two or three days, Sheila, Dan and I have worked more on switching of the ETM actuators from the ESD to L2 stage with the goal of reducing the ESD DAC noise (see Evan's latest noise budget). Last night, I was again able to reduce the ESD biases all the way to zero by actuating on the ETMY L1 and L2 stages (see previous attempts in alog 17267). The DARM spectrum improved in 30-200 Hz band as expected, but only by a factor of ~ 1.7. Lock was not so stable (which seemed to be related to a peak at 12.52 Hz) and it stayed locked only less than 10 minutes. I was not able to test different bias voltages yet.
(DARM noise)
Here are two DARM spectra, one with the 380 volts ETMX ESD bias (in purple) and the other with zero bias voltage (in red). The ETMY ESD had zero DAC voltage in all the quadrants and bias all the time. As you can see, the noise floor went down in 30-200 Hz band with a slight increase at around 20 Hz. Also a peak at 12.5 Hz has been prominent in this week (for example, look at alog 17355) and it looks like this peak rings up every time before we lose lock. The peak usually becomes clearly visible in the DHARD signals in time series several seconds before the lock loss.
(Actuation Scheme)
We used to switch off the ETMX ESD and switch on the ETMY L2 stage while keeping the ETMX L1 stage active (see alog 17267) in the last week. This time, we tried a different scheme in which we actuate the L1 and L2 stages of ETMY and we do not actuate ETMX at all. In a sense, this is a transition of the ETM suspensions from X to Y. Before trying the transition, we measured the transfer function of the L1 and L2 stages at different times in order to check the cross over frequency. The attached below is the ratio of the two transfer functions (i.e. L1 / L2) together with a LSC DARM open loop, L1 and L2 transfer function models:
As shown in the plot, the measurement can be mostly explained by the model. However, according to the model, the phase at 1.8 Hz touches 180 degrees and thus unstable at this frequency. Unfortunately our measurement incidicated that the ratio of L1/L2 exceeds unity gain at around 1.6 Hz and I noticed that this would be a potential risk of unstable loop. Indeed, this configuration gave me unstable DARM which rang up at 1.6-ish Hz as expected when I tried transitioning to ETMY. I decided to insert a lead filter around this frequency. With the lead filter (a simple zero-pole pair at 0.3 and 1.5 Hz respectively), the L1/L2 transfer function should now look like the following:
This has two cross-over points, one at 0.45 Hz and 1.8 Hz with phase mergines of 57 and 43 degrees. I installed the lead filter in FM8 of SUS-ETMY_L1_LOCK_L. This allowed a repeatable transition. The transition process is also coded in the ISC LOCK guardian.
Fred, Kiwamu,
So the 12.5 Hz oscillation happens not only when the interferometer is locked on DC readout but also when it is locked on ASQ. We had more than three times of lock losses today where we lost lock right after (typically in a few seconds) the 12.5 Hz peak rang up. Taking a close look, we noticed that a 17 Hz peak also rang up at the same time. This two-peaks-behavior seems consistent in each lock loss. Their peak heights typically show a slow modulation in the DARM spectrum (on the order of a few seconds) and eventually reaches as high as 10-13 Hz/sqrtHz (0.3 Hz fft-bandwidth) at which we lose lock. The attached is a screenshot of the DARM spectrum from a time when the interferometer was almost unlocking.
It turns out these 12 and 17Hz oscillations were my fault. Earlier in the week, we switched the HAM6 DC centering toplogy to use AS_A and the OMC degrees of freedom. But, I forgot to edit the part of the DRMI Guardian that sets the ASC input matrix. As a result, for the past two days we have been asking the DC3 centering loops to center the beam on AS_A, but we weren't giving them any input. With the correct DC centering loops on the instability does not appear.
We were able to replicate the 12Hz oscillation by disabling the AS_A centering loop before we increased the power. Also I was able to make it reappear by putting an offset into the AS_A pitch setpoint. At 10W input power, an offset of 0.1 counts was sufficient to excite whatever this mode is and break the lock. This is kind of a small offset - we should figure out how much margin we have during normal operations.
(It could be that we're sensitive to the spot position on AS_B, since any offset in the centering loops will also move the beam on AS_B. But we think that this noise comes in through DHARD, and that error signal is derived from AS_A. AS_B is used for MICH and SRC1.)
Attached is a trend around the time when we saw the oscilation grow during the first steps in the power-up sequence. The DC3 centering loops are turned on about halfway through the trend, and the spot on AS_A is immediately servoed to zero.
Slowly, over the past week, I have found a set of filter settings that damp the violin modes. In the tables below I identify the mode frequencies that are matched to each test mass; the matching was done based on which lines were suppressed or excited using bandpass filters in the L2_DAMP filter banks for each test mass. I also indicate which filter damps that particular line.
The filter settings are given at the bottom of each table. For all the L2_DAMP filter modules, FM1 is the bandpass filter (with +120dB gain in the pass band), FM2 and 3 are phase shifters (+/-60deg respectively), and FM4 is a 100dB gain. FM1 and FM4 are always on. FM2 and FM3 are used as indicated. The gain settings are included in the Guardian - the filters shouldn't change in the locking process, but I haven't updated the safe.snap files for the suspensions!
ITMX | |
f (Hz) | damped using... |
500.053 | MODE3 |
500.210 | MODE3 |
501.091 | MODE6 (still too high) |
501.253 | MODE3 |
501.450 | MODE3 |
502.620 | MODE3 (still too high) |
502.743 | MODE3 |
MODE3 | 0deg, -200 |
MODE6 | -60deg, +400 |
ITMY | |
f (Hz) | damped using... |
501.680 | MODE5 |
501.747 | MODE6 |
503.007 | MODE3 |
504.857 | MODE2 |
MODE2 | 0deg, -100 |
MODE3 | +60deg, |
MODE5 | -60deg |
MODE6 | +60deg |
ETMX | |
f (Hz) | damped using... |
504.870 | MODE6 |
505.585 | MODE6 |
505.708 | MODE6 (still too high) |
505.805 | MODE4 (still too high) |
506.921 | MODE6 |
507.157 | MODE6 |
507.389 | MODE6 (still too high) |
MODE4 | -60deg, -300 |
MODE6 | -60deg, +100 |
ETMY | |
f (Hz) | damped using... |
508.008 | MODE5 |
508.145 | MODE5 |
508.204 | MODE5 |
508.219 | MODE5 (still too high) |
508.583 | MODE5 |
508.659 | MODE5 |
1008.49 | MODE3 |
1009.03 | MODE3 |
1009.44 | MODE3 |
1009.49 | MODE3 |
~1484.5 | MODE6? |
MODE3 | +60deg, +100 |
MODE5 | -60deg, -100 |
MODE6 | -60deg, +10k (?) |
Unaccounted for: | |
f (Hz) | best guess |
501.606 | ITMY |
501.810 | ITMY |
503.119 | |
504.803 | |
507.192 | ETMX |
507.991 | |
508.288 | ETMY |
508.659 | ETMY |
The attached figure compares the noise spectrum from Friday and today. The first harmonics of the violin modes are now the dominant feature in the spectrum, along with 60Hz and the 2.5kHz BS butterfly mode.
I forgot to note the gains for the ITMY filters. They are:
Scott L. Ed P. Chris S. Cleaned the remaining 12 meters of tube to X-1-6 double doors. Results are posted here. Relocated lights and equipment this afternoon. New generator arrived this afternoon. John & I removed from shipping crate, added engine oil, hooked the up the battery and test fired the engine. It is ready to go tomorrow morning.
Only ETMy and ITMy ISI SDFs have anything 'DIFF.' These are the Blends Jim is running alogged 17295. Attached is the DIFFs at ITMY; the numbers at endy are the same. We'll likely save these as nominal soon. Either the BURT or PRESENT settings will work if the SHTF.
Karen and Cris in LVEA 08:16 Hugh setting SEI guardian nodes to 'Ready', charging HEPI accumulator 08:22 Gerardo to LVEA to look at pump carts by HAM1 and HAM2 08:26 Hugh and Gerardo back 08:36 Corey to squeezer bay 09:04 Gerardo turning on power supplies for annulus ion pumps at HAM1 and HAM2 09:39 Jodi and Gary to LVEA near HAM6 to tag boxes 09:41 Gerardo done 09:45 Jim B. to mechanical room mezzanine 09:56 Jim B. back 09:56 Andres to LVEA to look for beam splitter part in west bay 10:15 Andres done 10:18 Jodi and Gary done 11:22 Pepsi truck through gate 11:33 Gerardo to HAM1 and HAM2 11:40 Gerardo done 12:57 Tour in CR 13:01 Jodi and Gary to west bay 13:03 Andres in LVEA 13:09 Corey to squeezer bay 13:23 Jodi and Gary done 14:34 Cheryl to VPW 16:17 Cheryl in optics lab
Plots are found in SeiSVN: HEPI/H1/Common/2015-03-18_H1HPI_PumpControllerNoise.xml
The reference traces are from Monday 17 March 0900utc before the Tuesday Maintenance Accumulator Charging. The current traces are 24 hours later after the charging effort (BSC2SupplyNorth not done.)
My assessment is this assessment is a bit too noisy and this didn't make much difference. All the controller improvements have made things pretty good already. The largest coherence was in the BS RZ which shows some broadband signal between 10 and 100mhz and that is reduced a good bit. However, there are spikes of greater coherences in other dofs. I would never say this is not worth doing. Jeff may have some suggestions to better assess this is worhwhile.
I added the plots now.
Summary: The twin bumps at around 3300Hz comes from the OMC LSC loop, which has a 100Hz UGF.
Of course OMC length dither at 3300Hz should make some sidebands in the DC readout by design, but these twin bumps look really unnecessarily big.
We can reduce these by making the OMC length bandwidth much smaller.
Details:
In the attached, Red is with nominal setting (H1:OMC-LSC_SERVO_GAIN=60 and H1:OMC-LSC_OSC_CLKGAIN=6).
Blue is with 50% servo gain but with nominal dither amplitude (SERVO_GAIN=30, CLKGAIN=6), so it has half the open loop transfer function of red.
Green is is with nominal servo gain and 50% dither (SERVO_GAIN=60, CLKGAIN=3), so again it has half the OLTF of red.
IF the OMC length signal is sensing the true OMC length as opposed to just some noise that is not proportional to the dither amplitude, the OMC length control signal (middle row) for green and blue should be the same. But in reality they're 6dB apart from 8Hz and up.
Also you will expect that the OMC length error signal (bottom row) for blue and green have the same shape but different by 6dB, as the OLTF is the same but the sensing is 6dB different. That's only the case for f<8Hz.
So it's all noise for f>8Hz, and probably that's just the noise floor of DC readout at 3300+-f Hz downcoverted by the demodulator (not dither) to f Hz, fed back to the OMC length, and upconverted by the dither back to 3300+-f. See the left column. Blue and green on the top left are the same because the sideband amplitude around 3300Hz produced by this mechanism is proportional to the product of the feed back gain and the upconversion gain (dither), the sensing part is independent of the dither amplitude.
In summary,
Later Dan will try some low BW filters.
(John W, Gerardo M)
Turned on the ion pumps at 9:05 am local time with both pump carts valved in, both pump carts were on the low 10-06 torr
At 11:40 am I valved out the pump carts, but left the pump carts ON (connected to the annulus system and running).
Yeah :)
Obtained an alternate head to the charging hose. Took down corner SEIs (Guardians to ready), turned Pressure servo to 0 psi. Charged accumulator to 70psi. Returned pressure servo to nominal and brought SEI back on via guardian. Couple platforms tripped while deisolating and a couple (HAM2 & HAM3) tripped a couple times on the way back up. All nominal now.
CDS Filiberto moving SEI teststand from staging building to H2 building Facilities Bubba working by metal recycling container Bubba looking at possibility of moving crates at mid X Beam tube washing continues Gerardo working on vacuum pumps by HAM1 and HAM2 3IFO Corey working in squeezer bay Other work on floor TBD Operations Suresh is giving training on optical levers at 11 in the large conference room
There has been an intermittent issue with the OMC_LOCK Guardian in which it runs a portion of code twice. I've observed this most often in the OMC_LSC_ON state, where the LSC controls for the OMC are ramped up. It's easy to notice, because if the gain steps are repeated for the OMC length servo the loop quickly becomes unstable and the cavity unlocks. I've noticed this happen a handful of times in the past few weeks. I've also seen lines of code in other main() function get executed twice, although I don't have screenshots to prove it.
Attached are screenshots of the OMC_LOCK log, from an event last week (March 14), and another tonight. In both screencaptures, the OMC guardian enters the OMC_LSC_ON state, completes the instructions in main()...and then starts all over again. In both cases the requested state was well downstream of OMC_LSC_ON, the guardian should not have looped there. (And anyways, how does it repeat the main() function?)
I've committed the latest version of the OMC_LOCK guardian to the SVN, if experts want to check the code to make sure I'm not doing something heinous in the function calls or definitions.
In other locking notes from tonight...
After several tries at handing off the DARM drive to ETMY L2/L1, we are leaving the IFO locked. 16Mpc.
Dan, Keita, Kiwamu, Sheila
During a long, patient lock this evening I was able to measure the DHARD pitch loop down to 0.2Hz. This follows Keita and Sheila's filter modifications to get some additional phase around the 3Hz UGF. The attached plot is a record of the measurement (look at the RED trace), I have saved the xml file with the filename at the top of the plot.
The phase margin at the UGF is good (~40deg), and the loop does not cross unity gain at higher frequency. There is almost a unity gain crossing at lower frequency, we have about 3dB of gain margin at 0.9Hz.
We're fine without aggressive boost.
It's clear that the boost (FM6) was not on in this measurement.
Yet, the measured TF looks OK in that even if the dip at around 0.9Hz changes somewhat and crosses the unity gain, it will be very stable.The phase margin at around the dip is between 140 and 180 degrees.
Also the phase at UGF was improved by 10+ deg due to the new FM2 and by disabling redundant notches (FM7 and FM9).
With the boost, we'll get close to 50dB gain at 0.1Hz at the expense of 13 degree phase at UGF, about 7dB gain at 1.5Hz peak, and 2dB or so higher high frequency (f>7Hz or so) response. That sounds kind of excessive to me.
Since the second UGF at 0.9Hz will not be a problem I'd rather leave that guy off. If we need more DC gain we can make a milder boost without messing with the gain at 0.9Hz.
Jamie, Dave, Jim:
Jamie reported that the location of the MEDM Print selection, being at the top of the right-mouse pull-down menu, has resulted in many accidental printer jobs. This explains the printouts of things like the sitemap screen with no data content and no one picking the print-outs up.
Jim modified the LHO CDS MEDM this afternoon to move the print section from the top to second from the bottom of the pull down menu. Please see the attached before and after images.
During tomorrow's maintenance we will stop all old MEDMs in the control room so the new feature will be picked up. This is only a linux change, it was not applied to Mac OS.
Awesome. Thanks, Jim and Dave. The forests thank you.