Jeff, Kiwamu, Stefan
We had trouble engaging the ISS 2nd loop at 40W. The core issue is that the digitized signal (H1:PSL-ISS_SECONDLOOP_SIGNAL_OUTPUT) swings rail to rail without the loop, making it difficult to properly pre-set the offset adjuster (H1:PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA).
As a result, when engaged, the ISS 1st loop tended to swing into saturation.
I patched the Guardian as follows:
- Instead of trying to rely on a calculation for the offset adjuster (H1:PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA), I hard-coded it to 0.987 V, which is where it likes to sit at 40W. (THis is obviously not a long-term solution.)
- As soon as the servo comes on, I ramp H1:PSL-ISS_SECONDLOOP_REF_SIGNAL_SERVO_GAIN to zero, clear the history, and re-engage it on the diffreation power.
This seemed to engage the 2nd loop without excessive transient - back to full locking.
PS: I might have already mentioned this: but the ISS SHOULD BE AC COUPLED!
Code:
In PREPARE_ISS:
# read current servo signal value. If it is too far from the operating point, correct the offset.
#second_loop_out = cdu.avg(1, 'PSL-ISS_SECONDLOOP_SIGNAL_OUT16')
#if abs(second_loop_out) > 1.0:
# # measure current input PD value
# # Sets the ref. signal value to bring it to close to the operating point
# pd_avg = cdu.avg(1, 'PSL-ISS_SECONDLOOP_PD_14_SUM_INMON')
# ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA'] = round(0.5*((pd_avg*2.29*.0006103515625)+0.101), 6)
# time.sleep(5)
# above section commented out - instead command the value that worked (Stefan Ballmer 20160629 for 40W)
ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA']=-0.9826934814453125
In CLOSE_ISS, once the loop is closed:
# the loop is successfully locked, ramp off the loop (Stefan Ballmer 20160629 for 40W)
ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_SERVO_GAIN'] = 0
and a little later:
# clear and ramp on the loop (Stefan Ballmer 20160629 for 40W)
ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_SERVO_RSET'] = 2
time.sleep(0.1)
ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_SERVO_GAIN'] = 1
After talking with Rana and Evan during the NSF review, I made some independent estimates of the thermal noise that could come from the composite SRM which had a measured resonance frequency of 3.34KHz with a Q of 150. I get the same thermal noise displacement spectra of the mirror surface as Evan in log 27488,27490,and 27490. Tend to favor the viscous case as Peek is a thermoplastic as is Nylon which was measured to have a viscous spectrum. The model is a shear motion of Peek 8-32 screws clamping the 2 inch diameter 1 inch thick mirror into the metal carrier. The motion is a displacement of the mirror allowed by shearing the 8-32 bolts in and out of the plane of the carrier - longitudinal modulation of the light beam. Thermo elastic damping is much too small to explain the low Q. The time constant for thermal diffusion in the Peek screws is about 10 seconds. The thermo elastic delta is about 10^-2, so that loss tangent phi varies as 1/f with a value of 5x 10^-8 at 3.4kHz and about 2 x 10-6 at 100Hz. My guess unfortunately is the SRM is not responsible for the excess low frequency noise we are seeing. I think Evan and Rana had come to the same conclusion.
The OMC would not lock as we entered DC_READOUT_TRANSITION, and the DARM spectrum had risen around 100Hz. Sheila tracked it down to H1:LSC-ARM_INPUT_MTRX_RAMPING_1_3 having a gain of 20 and not 8. She investigated further and realized that the matrix was not being loaded beore the actual value was in place, something that a sleep of 0.1s can cure. With her help. I added in these short sleeps or just loaded later.
J. Kissel Several lock losses were caused by DC coupled ISS loops today, so in my desperation to make things better, I modified the PSL ISS 2ND Loop MEDM screen to include all the relevant PSL AOM Diffracted Power information on the same screen (formerly only available on the 1st loop ISS screen). Also, to facilitate the turn on dance, I've decreased the default increment on the slider used to tune the analog reference signal (H1:PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA) and increased the precision which the reference signal value is displayed. The updated screen has been committed to the user apps repo here: /opt/rtcds/userapps/release/psl/common/medm/ISS/PSL_CUST_SECONDLOOP.adl
Sheila, Haocun
We designed some new filters for the low bandwith CHARD Yaw at 40W.
We plan to lower the gain for about 6dB, which will give us a UGF at 3.5Hz and use the new filters 6 ('40W') and 7 ('ctrlHBW') to stablize the loop (blue curve in the 2nd Figure). The new cutoff filter for 40W is filter 9 ('cutoff40W') (Figure 1). The pink curve in the second curve is what we expect from the simulation.
yesterday fw0 was more stable than fw1. In the evening the new QFS-NFS server was serving fw0's files to both nds0 and nds1. I also stopped fw1 from writing science frames to see if this would improve its stability. Overnight fw0 restarted several times, meaning that fw1 needs to write its science frames again (it had not become more stable). Again we are seeing both framewriters sometimes restart around the same time, a synchronisation we had not previous seen.
The configuration for fw1 was changed to write science frames again and I waited until 4pm for it to restart itself, but fw1 went into a period of stability. At 4pm I restarted fw1 and it became very unstable. After power cycling both h1fw1 and h1ldasgw1 it is now stable.
During the day Stefan needed nds1 to link to fw1's frames to see data for a time fw0 was not running, nds1 was reconfigured and restarted.
Support: Jeff K., Jenne, Kiwamu, Peter K., Sheila At the beginning off the shift I changed the power_sepoint in lsc_params.py from 20 to 40 and loaded it. After a lock loss from 40 W we started having FSS and ISS issues similar to what TJ (alog 28034) and Sheila (alog 28045) had reported last night. Kiwamu was able to find and address the problem (alog 28058). A couple of times the guardian did not appear to recognize a lock loss and thus did not take the IFO to down. I had to do so manually. There were occasional notifications that the POP X PZT had railed and that the IMC WFS were not centered. There were also notifications that the TCSX and TCSY CO2 power was off by more that 0.05 W. Stefan fixed a bug in the DRMI down state (alog 28053). I ended the shift having trouble locking DRMI and handed it over to TJ to run an initial alignment. 14:58 UTC Peter to LVEA PSL racks to reset noise eater 16:08 UTC Bubba and Nicole to mid X 16:57 UTC Karen and Christina to mid X and mid Y to clean 17:02 UTC Ed to CER to reset DBB AA/AI chassis (alog 28049) 17:52 UTC Karen leaving mid Y 18:07 UTC Christina leaving mid X 18:19 UTC Ryan patching alog 21:44 UTC Kyle refilling CP3 (alog 28062) 22:03 UTC Kyle back
biggest = prm pitch changing by 2.5urad
close second = im4 pitch changing by 1,15urad
plot and chart of top 16 optics/DOFs attached
plot of optics/DOFs in the chart - meant to attach yesterday. I
used OpLev signals when available and OSEMs when not.
UPDATE to my alog's less than descriptive title: I think the original title suggests these changes were during a 40W lock, but this is not the case.
The optic alignment changes are actually happening over 8 minutes of full lock while the input power increased from 22W, the O1 power level, to 40W.
1450 - 1505 hrs. local -> To and from Y-mid Opened exhaust check-valve bypass, opened LLCV bypass valve 1/2 turn -> LN2 at exhaust in 50 seconds -> Restored valves to as found state. Next CP3 over-fill to be Friday, July 1st.
Kiwamu, Stefan
We were trying to nail down what is responsible for the Power Recycling Gain change during power-up, so we went through the whole sequence slowly, and gave the system time to settle in each state:
All times UTC of 20160629:
- 18:54:37 / Green dot: Power-up starts from 2Watt to 10Watt
- 19:00:32 / Red dot: Power-up restarts from 10Watt to 40Watt
- 19:06:10 / Black dot: : Power-up to 40W finishes, but additional SOFT offsets remain left off.
- 19:10:15 / Blue dot: Soft offsets in yaw start ramping on over 2 minutes
- 19:12:17: Soft offsets ramp in yaw is done
- 19:15:17: Soft offsets in pit start ramping on over 3 minutes
- 19:18:19: Soft offsets ramp in yaw is done
- 19:21:20: End of undisturbed period - guardian continued
In particular, we now there is a significan power recycling gain improvement with a soft yaw adjustment of the x-arm. Additionally, the beam splitter cannot affect the relative alignment of x-arm and power recycling cavity. SO attached are only yaw signals from x-arm and power recycling cavity.
Conclusions:
- During the very first step in power up (green dot) there is a momentary shift in PRM yaw and IM4 yaw, but it does not continue during the additional power increase.
- During the main power increase (between red dot and black dot). There is a consiostent signature in the ETMX control, oplev and IR camera signal, sggesting that we are slightly moving the x-arm with the ASC system. THe ITMX doesn't see much of this in control signal and oplev signal (its camera is broken and not plotted). However no significant motion is visible in any of the power recycling cavity mirrors (this includes PR2, which is not plotted).
- Once we start moving the soft loops with soft offsets, ETMX, ITMX, PR3 and BS clearly respond. Interestingly, the ETMX moves to the same direction as during the power increase.
Attached are 220days of recycling cavity alignment data (Nov 15 2014 to now)
The alignment offsets for IM1-3 are not a valid way to track the optic alignment, since IM1, IM2, and IM3 all have significant alignment shifts after a HAM2 ISI trip.
I wrote a procedure to correct these after restoring the ISI and the optics but adjusting the alignment offsets to drive the optics back to their nominal alignments.
During O1 the procedure was to drive the optic back to it's most recent good value, however this can be hard to decipher, so in May I posted a Nominal Alignment for the IMs and have maintained it since.
I've been maintaining an alignment of the IMs based on our last good high power lock in April 2016, and this alignment choice is explained in alog 26916, and recently updated in alog 28016.
On a longer time scale, the alignment of the IMs is harder to track since I only started restoring their alignments after an ISI trip around September 2015, so before that time when IM2 pitch shifted 40urad we spent time realigning H1 to follow that change.
In my investigation, I see that the IM alignment shifts have caused down time in H1, and IM1-3 now have an alert in guardian to prevent the IM alignment shifts from causing down time in O2.
Here are some pertinent alogs from my investigation:
Last night at about 23:30 UTC last night, we had an earthquake arrive at LHO. I don't know where it was or how big, because Terramon has been down for a while. The peak BLRMS displacement seen by the ITMY STS in the .03-.1hz region was .28 microns. The first thing I noticed was the IMC F and LSC Y TIDAL on the Tidal StripTool were showing large ~20 second oscillations. When Sheila switched to the "Earthquake" SEI_CONF state, these oscilations went away, even though the earthquake band ground motion continued going up. My first plot shows the SEI_CONF state ( 40 is the windy state (SC on everywhere), 17 is the earthquake state) ITMY Z .03-.1hz BLRMS displacement in nm, IMC-F and LSC-Y_TIDAL_MON. You can see that at about the time the ground motion BLRMS gets to about 100nm, the IMC and LSC TIdal starts ringing up. As soon as the sensor correction is turned off, the IMC and Tidal settle down. This lock lasted in this configuration past the earthquake, until the commissioners tried to move to a higher guardian state.
The next two images are two times later in the evening, one when the SEI_CONF was still in the earthquake mode (6/29/2016 1:10 UTC), the other after the later lockloss when the commissioners had changed back to the "Windy" SEI_CONF state (6/29/2016 2:13 UTC). IMC and Tidal don't seem to be bothered by either state. The newt four images are various spectra comparing IFO control signals at these two times. Unfortunately it looks like there was maybe another small earthquake and higher winds when the SEI_CONF was still in "earthquake". All "REF" traces are with the earthquake configuration, the live traces are with the windy configuration. I think these spectra show that for now we can be in either state, but some of the suspension length signals show lower drives with the "windy" config. Additionally, the earthquake config will probably not be okay with higher microseism, we have been sitting at about 10th percentile levels for a while now. The last plot just shows the effect of turning of the (BRS subtracted, STS based) sensor correction on the ETMY ST1 T240s. The Y blend is the same for both blue and pink traces. The subtraction at the microseism doesn't look good right now, but I think that is because of the low microseism.
Hey Jim, great post, I'm happy to see there is already a SEI_CONF in place for EQ.
About the epics variable for EQ, we implemented one here at MIT that seems to work fine (more details here). Right now it just does what terramon is doing (arrival time and predicted amplitude), which is a start but not perfect. The goal would be to have a real warning system, telling us how likely we are to loose lock. We are curently working on that, and hopefully we could install something at the sites soon.
DBB:
.001 files= Lo Power
,002 files = Hi Power
As expected beam pointing and mode matching in the Hi Power path remain to be addressed.
ISS_RPN:
It looks like AOM diffraction noise is high consistent with the higher-than-usual measured diffracted power.
PDB is "in-loop" and noise is ≈ 1 oreder of magnitude higher than reference.
Pointing noise in X-axis seems to be elevated in the regions between 12Hz & 80Hz. Also, there is some Y-Axis noise between 100Hz & 850Hz.
Lisa asked me to run a BruCo scan on data from this elog entry: 27990 (June 28, 6:00 - 6:10 UTC)
The report is available at the following address:
https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_1151128817/
Instead of CAl_DELTAL I used directly the OMC_DCPD_SUM signal. The reason is that the usual calibration of CAL_DELTAL does not seem to be correct anymore.
In summary:
For future reference, to run BruCo you can follow this procedure
./bruco.py --ifo=H1 --channel=OMC-DCPD_SUM_OUT_DQ --gpsb=1151128817 --length=600 --outfs=4096 --naver=300 --dir=~/bruco_1151128817 --top=100 --webtop=20 --xlim=1:1000 --ylim=1e-10:1
Thank you, Gabriele.
So, I've taken the NPRO "RRO" (which is LASER speak for Resonant Relaxation Oscillation - what is that?? right?) status indicator from the PSL_LASER.adl screen and copied it to the FSS_LASER.adl screen and aptly enhanced the label to include the term "NOISE EATER" which is a term that we would more likely be familiar with. It's located in some prime real estate just South of the "Oscillation Threshold [v]" setting. I did this at the advice of Peter King in response to the issues that have recently arisen regarding the FSS trying to re-lock after being "kicked" by a lock loss. In the usual tradition, GREEN=Good and RED=Bad.
Marie, Sheila, Kiwamu, Lisa, Stefan
We tried a few things to improve the recycling gain tonight. We believe that the problem is with something in our PRC. We can see that the green power transmitted through the X arm stays stable as we power up, we think this means the optics are stable. When we move the soft offsets to improve the recycling gain we change the X arm alignment to follow something in the vertex.
As TJ noted, since the computer problems earlier we are having trouble relocking the PSL after locklosses, with noise eater, PMC and FSS and ISS problems. It seems like trying to lock the FSS causes the noise eater to oscillate again after it has been reset.
Jeff K, Stefan, Peter, Kiwamu
We kept having the same issues with the PSL in this morning as reported by Sheila in the above entry. We now think that it is all fixed. Here are what we have done for fixing the issues.
Not that all of the settings that Kiwamu describes were not found by the SDF system because I'd mistakenly set the SDF system to look at the OBSERVE.snap record instead of the SAFE.snap record after maintenance yesterday. The OBSERVE.snap was out of date, having not been updated since O1 and/or the HPO turn-on. Both the safe and OBSERVE.snaps have not been updated. Another case of the difficulty of maintaining several .snap files per front-end model...
The OMC DCPDs have proven to be useful for monitoring the test mass acoustic modes around 15 kHz, but there is a lot of low-pass filtering in the readout chain that render them less useful for monitoring higher frequency acoustic modes. This is now being changed with modifications to the electronics that will provide separate, faster channels for PI monitoring:
The attached plot shows the magnitude response of the low-pass filtering in the previous case, and with the poles removed from the whitening and AA channels. It is no wonder that no PI modes have been seen above the 15-16 kHz grouping, as there is 40 dB of relative attenuation already at 25 kHz.
I also attach a 1kHz - 10 MHz transfer function of the in-vacuum DCPD readout that Koji measured at Caltech:
Here are the transfer functions with the AA notch included. I had forgotten that the notch is a passive twin-T type, which by design has a Q = 1/4, so it is quite wide and should be taken into account. In future the AA notches should also be removed.
Jenne, Lisa, Stefan,
Still running at 40W, for 2h.
Tried to damp some PIs and go to low-noise.
- We successfully damped 15009Hz (ETMY) and 15542Hz (ETMY) (damp settings picture attached.)
- 15541Hz (ETMX) was ringing up.So we tried switching to ETMY and transition the ETMX coil driver to low noise.
- We successfully switch the coil drivers, but...
- We switched to ETMY, low noise, held lock for a while, but saturated the ETMY ESD with 20Hz noise from the ASC, as well as a 532.77Hz line, that we don't know the origin off. (PI?) (picture)
===================================
Also, we realized that DTT is a bit too smart: The 64kHz channel H1:OMC-PI_DCPD_64KHZ_A_DQ can in principle look at PIs above the Nyquest through aliasing. However, since DTT only alows you to select a freqency about 10% below the Nyquist frequency, there is an effecive dead band where DTT cannot see PIs between ~29500Hz and ~36036Hz. The MATLAB script below produces a spectrum up to the Nyquist. Indeed, we found an elevated mode at 30018.3Hz, although it was not quite saturating.
MATLAB code to get spectrum up to Nyquist frequency:
addpath /ligo/svncommon/NbSVN/aligonoisebudget/trunk/Externals/SimulinkNb/Utils/
gwd = GWData();
gwd.make_kerberos_ready;
gwd.site_info(2).port = 31200;
gwd.site_info(2).server = 'nds.ligo-wa.caltech.edu';
H1.channels = {'H1:OMC-PI_DCPD_64KHZ_A_DQ'};
time_fetch = tconvert('26 Jun 2016 01:15:00');
[data, t, ~] = gwd.fetch(time_fetch, 1000, H1.channels);
Fs=4*16384;
pwelch(data,[],[],[],Fs)
[Lisa]
Attached is Jenne's PI knowledge, she also updated the PI wiki page.
This last 40W lock ended at June 26, 2.11 UTC while trying transitioning to low noise to damp the 15541 Hz ETMX PI.
The 532.7Hz matches the beat frequency between the 15541.9Hz ETMY and 15009.2Hz ETMY modes that were at very elevated amplitudes during this lock. 15541.9Hz appears to have been unstable ringing up with a time constant of 190 seconds, damping was engaged with a gain of 1000, so it may have been PI or being driven up, while the 15009Hz damped over the duration of the lock with someone actively changing the control gain. See the figures of 15540Hz and 15kHz mode group amplitudes over the last half hour.
There were two peaks ringing up in the unmonitorred region between 29000Hz and 32768Hz, one appearing in the OMC DCPD signals at 30551.09 and another at 31083.90. See third figure. The beat note between these two peaks is also around 532.7Hz. The connection can be seen in the fourth figure when the beginning of lock spectrum (green) is compared to the end of lock spectrum (blue). The largest green peak is the sensing harmonic of the 15009Hz mode. The largest blue peak is the first sensing harmonic of the 15541Hz mode. These mixing with the 532.7 produce the center frequency and the 31620Hz peak. There appears to be something else producing messy ~480Hz 'sidebands' on some of these peaks.
When people were engaging the offset adjuster servo with 40W into IMC, eventually H1:PSL-ISS_SECONDLOOP_REF_SIGNAL_MON_OUTPUT settled down to -0.98V and H1:PSL-ISS_SECONDLOOP_SUM14_DC_OUT16 to 39.6.
This PSL-ISS_SECONDLOOP_SUM14_DC_OUTPUT thing is a digital sum of four digital outputs, each with dewhite, so it's well behaved even with high power, unlike PSL-ISS_SECONDLOOP_PD_14_SUM which is just a readback of the second board input without dewhite.
pd_avg = cdu.avg(2, 'PSL-ISS_SECONDLOOP_SUM14_DC_OUT16')
ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA'] = round(-pd_avg*0.98/39.6, 6)
instead of
ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA']=-0.9826934814453125
But you might have to fine tune the above to account for some offset in the board. I haven't changed the guardian.