Summary of the PSL trips from 2015-5-1 to 2015-5-3 during the LHO mini run. Every instance of the PSL tripping was due to the same interlock, H1:PSL-IL_DCHILFLOW, tripping with no apparent loss in coolant flow for the diode chiller, H1:PSL-OSC_DCHILFLOW. The first three trips on 2015-5-3 were during the long interlock trip that Nutsinee reported in alog 18187. Data for the other PSL trips over the last 10 days have been reported in alogs 18083 and 18134. All times UTC.
Submitted bug report #1057
Using 0100pdt on Saturday morning as an okay time to check the ~middle of the 10 hour lock stretch during the run. No large coherence with DARM_CTRL for any of the three controlling pressures for the HEPI Pump Controller Servos. Everything is below ~0.6 and most all just look like noise.
Attached are power spectra of the sum signals of the 4 IR QPDs that are on the ETM Trans Mon tables (in-vac). Each Trans Mon has 2 QPDs (labeled A & B). Also included in the plot is a spectrum of QPD_B from the L1 Y-arm. As discussed in LLO log entry (hmm, LLO log is down), the L1 Y-arm QPDs show broad noise peaks at 30 Hz, 60 Hz, 90 Hz; these peaks show up in DARM and are believed to be caused by scattered light at the end station (the L1 X-arm QPDs do not show these noise peaks).
The good news is that the H1 TransMon QPDs do not show anything like the scattered light noise peaks seen in the L1 Y-arm, and in general the H1 QPD spectra are lower than the L1 spectra (all channels are normalized to unity for a single arm lock). The RMS of the X-arm transmitted power fluctuations is about 30/10,000 = 0.3%; about 1% pk-pk.
The bad news is that there is something wrong with QPD_A on the Y-arm. It shows an elevated 1/f noise above a few Hertz. This is presumably related to the QPD 'clipping' problem reported by Keita in entry 18204. Looks like some electronics problem to me. On the other hand, the X-arm QPD_A is anomolous in another way: its average value is about 30% lower than the others (see table below).
For reference, the average values of the QPD NSUM channels were:
| A | B | |
|---|---|---|
| X | 9100 | 12,000 |
| Y | 13,200 | 12,800 |
It seems like segment 2 is the bad guy (in that the excess noise comes from that segment), and it comes with about 900 counts of offset which indeed can quantitatively explain the "clipping" behavior.
Segment 4 is also strange in that the noise floor is 6 dB larger than the others.
Whitening interface is working fine in that the spectrum of all segments move up and down by the same amount at the same time for each step of the whitening slider from 0 to 45dB, and all combinations of whitening/dewhitening result in the same spectrum.
It seems like the input stage of the whitening box for Y QPDB SEG2 is broken.
Moving slider up and down, switching whitening filter on and off: All signals changes as they should.
Disconnecting the QPD cable between the chamber and the QPD transimpedance amplifer box: No change in the DC offset (QPD B SEG2_IN1 -900 to -1000 counts at the nominal whitening gain of 18dB, everything else within +-10 counts).
Disconnecting the cable between the QPD transimpedance box and the whitening box: SEG2_IN1 jumps to -32k, others no change.
Disconnecting the cable between the whitening box and the DAQ: Everything goes to zero-ish.
Since I don't want to mess with whitening I/O cables now, which have a history of acting up and taking a long time to diagnose and fix, I asked Richard to swap the whitening chassis the first thing in the morning tomorrow.
It turns out that the bad segment has been bad since about 2015/03/29 18:00 UTC, that's Sunday 11AM local time. Nothing on the alog.
Laser Status: SysStat is good Front End power is 31.4W (should be around 30 W) FRONTEND WATCH is RED HPO WATCH is RED PMC: It has been locked 0 day, 23 hr 54 minutes (should be days/weeks) Reflected power is 2.0 Watts and PowerSum = 24.3 Watts. (Reflected Power should be <= 10% of PowerSum) FSS: It has been locked for 5 h and 4 min (should be days/weeks) TPD[V] = ~1.1V (should be 1.3 - 1.6) ISS: The diffracted power is around 8.5% (should be around 7%) Last saturation event was 22 h and 57 minutes ago (should be days/weeks)
Attached are the series of plots: ASDs, Coherences, and TFs; for and between the three LVEA SEI STS2 instruments.
Last week I moved the STS2-B (ITMY) to concrete (References for it are the old unit on vinyl under the igloo.)
The current traces attached are from this morning at 0130pdt. I had a problem with the calibration and units before, but getting that straight hasn't made answers to all the questions clear.
The three Z ASD are right on top of one another to below 20mHz. The Y DOF ASDs are similar but they diverge below 40mHz. For the X DOF, the HAM2 instrument seems the outlier diverting from the other ASDs as high as 90mHz whereas ITMY and HAM5 are hand in hand until 20mHz.
The Coherences look very good on Z, other DOFs are here and there. Maybe as expect too, the Z TFs look great, and the other DOFs are 'it depends.'
I think we can say that the removed STS2 should go in for overhaul and the HAM2 machine should be huddle tested with one of the others.
Apparently I haven't aloged it last week, but oplevs recive the IR scatter from the arm. It's not a big deal, I'm writing this just for the record.
Attached shows one 7W lock when ETMY OPLEV was not working (no light). As soon as IR resonates in the arm, ETMY oplev segments jump up and the SUM goes to 560 counts (left and middle row). If the oplev laser was alive, this 560 counts would have been added on top of about 30k counts. Fortunately this is mostly common for all four segments, and the effect on the angle readout, when the oplev laser is alive, would have been about 0.03% of the full range, or about 0.03 urad.
The same thing happens to ETMX (right bottom) except that the oplev laser was alive and that the scatter increased the oplev SUM by only 360 counts.
For ITMs the SUM due to the arm scattering seems to be abount an order of magnitude smaller than ETMs, but the oplev power itself is also smaller (3000 to 4000 counts).
Along the same lines: quite a while ago during the DRMI locking, Jeff and I noticed that the BOSEMs of the HAM6 tip-tilts would pick up flashes from the DRMI lock acquisition that would generate spurious damping signals and shake the mirrors. This isn't a problem because we don't need those optics to be quiet before DRMI locks, but we thought it was interesting. I can't remember if the OMC SUS acquired some noise from the DRMI flashing.
After we noticed this, Jeff and I checked that it wasn't a problem for any of the corner-station mirrors that we don't actuate on to lock DRMI. We saw no evidence that DRMI flashing generated noise in the ITMs, PR2-3, or SR2-3. Of course it's impossible to tell if PR2, SR2, or BS OSEMs are affected by flashes since the LSC drive is banging on those optics when we're trying to acquire.
(Update: See Peter's alog.)
I looked at the lock stretch from last night and the IR QPDs on TMSY still show the sign of clipping though the saturation was fixed.
On the left column you have X QPD SUM(B)/SUM(A) data (top), which is basically just a solid flat line after the IR resonates in the arm over a large change in the IR alignment.
The top right plot for Y QPD SUM ratio has the same scale as the top left, it goes up and down in low power lock and then steps up when the power was increased. Even during the 10W lock, the ratio was not as stable as X.
It's not like Y beam is particularly moving larger than X.
J. Kissel Now that the mini-run is complete, I've reverted the "nominal" ISC_LOCK guardian state back to LSC_FF (I'd changed it to the mini-run state of "LOWNOISE_ESD_ETMY" on Friday, see LHO aLOG 18145). I've reloaded the guardian code change, and confirmed that the "nominal" box is now surrounding the LSC_FF state; see attached.
SEI nothing planned SUS nothing planned CDS continue preparations for chassis swap possible RCG upgrade on Tuesday FAC starting repair of wall around patio area wet paint all around building expecting delivery from Sunbelt PSL trips appear to all be related to flow sensor VAC preparation for vent of end stations in June need to run pump cart at Y end for 24 hours to bakeout RGA, possibly Tuesday
This morning AOM diffracted power was up to ~ 13% with a refsignal setting of -2.00 . I've adjusted the Diff power to ~ 8.5% -> refsignal = -2.05
I've taken care of the changes noted by the SDF and accepted the new settings.
FSS resonance threshold had also changed from .6V to .55V. This was also noted and accepted in the SDF.
Summary
Today I wanted to see if increasing the bandwidth of the SRC ASC loops could make the 90 Hz SRCL coupling into DARM go away or become more stationary. I cranked up the bandwidth of the AS_C loop, but saw no improvement. We should try cranking further, or else try cranking the bandwidth on the AS36I loop.
Details
I was able to bring the interferometer to dc readout at 10 W with a recycling gain of 39 W/W. To get there, I had to turn on the ITM oplev damping again to avoid 0.4 Hz instability during power-up (and sometimes during the last steps of the CARM reduction sequence). I was able to engage the MICH, SRCL, and dETM yaw boosts without issue.
Then I tried increasing the bandwidth of the AS_C pointing loops. I was able to increase the gain by a factor of 30 just fine (2 ct/ct to 60 ct/ct, pitch and yaw), which gave a ugf of about 250 mHz as measured by step response. This means that the original bandwidth was less than 10 mHz, since the loop is 1/f below the suspension resonances. Correspondingly, the low-frequency angular motion seen by AS_C is suppressed by a factor of 30 or so, and the rms angular motion seen by AS_C is reduced by a factor of 4 in pitch and some small factor in yaw. Based on the attached spectra, it seems like we would have to push the bandwidth up to several hertz in order to see any further improvement in the rms. Anyway, after doing this I did not see any improvement in the stationarity of the SRCL coupling aroung 90 Hz.
I attempted to repeat this exercise with the AS36I loops, since Gabriele had previously found that the pitch loop was the dominant signal modulating the SRCL/DARM coupling once dETM yaw was adequately suppressed. However, I could not increase the loop gains by more than a factor of 4 before they went unstable (this was the case even when I engaged the same HSTS compensation filter as used for the AS_C loops). This factor of 4 was not enough to give a noticeable improvement in the spectra of AS36I pitch and yaw, since these loops are also slow (<100 mHz, just based on watching the loops' responses to transients).
To answer Jeff's question about the wind, I looked at the ten hour lock, and the three hour lock starting about 23:30 UTC (ignoring the stochastic injection). I wrote a custom script to make the plots, since I couldn't get sensemon range from NDS. I've done a 3-minute median filter on the wind and range minute trends to smooth them out a bit. The first plot shows the wind and range for the first lock. Around 12:30 UTC, the spectrogram (second plot) shows a big increase in the noise, and there's a corresponding decrease in the range. I think this is unrelated to the wind, and will be investigated elsewhere, so I've cut this lock off about 320 minutes in (third plot). The fourth plot is the scatter of the wind versus the range. A simple fit gives a value of 8.4 Mpc at zero with an 0.03 Mpc decrease per MPH of wind. The fifth and sixth plots are this repeated for the other lock, starting after the stochastic injection ends. The fit is 9.0 Mpc at zero with an 0.01 Mpc decrease per Mpc of wind. I've attched the script I used. It needs a cache file for data of type H-SenseMonitor_CAL_H1_M called snsw.lcf. There's some values hard-coded, but it should be obvious what to change.
Sweet set of plots, thanks! Can you (and/or @DetChar, collectively) make a similar assessment of wind vs. omicron glitchiness?
J. Kissel Having processed the last two DARM open loop gains -- the first measurements out to 5 [kHz], and the first measurements after we've tuned up the ASC loops to give us a consistent recycling gain of 35-40 -- I can now make the following statements: - The DARM Coupled Cavity Pole (CCP) frequency is now consistently much closer to the "as designed" value. The measurements are consistent with a model of the DARM loop using a CCP of 355 [Hz]. - The ETMY ESD's driver pole frequency is 2.2 [kHz], not the colloquially thrown about 2 [kHz]. - I've added the violin modes to the model of the QUAD suspension, and they have no appreciable effect, but I'll leave them in for completeness. On what these measurements and changes to the model mean for the uncertainty (precision and accuracy): - The modeled unknown time delay remains at 0 +/- 5 [us]. - The frequency dependent uncertainty in the calibration model remains at +/- 2.5% in magnitude and 1 [deg], but I've data to back up that this extends out to the entire required* frequency band 10 and 2000 [Hz]. See attachment 2015-05-02_FittedCCP_H1DARMOLGTF.pdf (* OK, what *used* to be the requirement. The requirements are now extended albiet inflated out to 5 [kHz]; see T1300950) - Over these last two measurements, the scale factor used to match the model against measurement has only varied by 3%. Indeed, if you cluster the eight measurements, grouping by CCP, then the standard deviations of the three, three, and two measurements are 7%, 41%, and 3%. However, the total standard deviation of all measurements is 22%, and the latter two measurements are only 1 day apart. From the data I have, I'm still not confident in decreasing the scale-factor uncertainty below 22%; perhaps PCAL can make a better statement on this (but I fear the current Mini-Run noise is too high for the low-frequency PCAL line). See 2015-05-02_upto5kHzOnly_FittedCCP_H1DARMOLGTF.pdf - The current calibration installed in the CAL-CS model is incorrect, both in frequency dependence and scale factor. - Frequency dependence: The CAL-CS filters assume a DARM CCP of 389 [Hz] in the sensing path, and an ESD Driver Pole (ESDP) of 2.2 [kHz] in the actuation path. For the latest lock stretches, that inflates the uncertainty of CAL-DELTAL_EXTERNAL_DQ with a known, systematic error of current / correct = ( 1 / (current CCP / correct CCP) + (current ESDP / correct ESDP) ) / (properly normalized) = ( 1/zpk(-2*pi*389,-2*pi*355,355/389) + zpk(-2*pi*2e3,-2*pi*2.2e3,2.2e3/2e3) ) / 2 which translates to as much as a 7% magnitude error at 1 [kHz], and 1.8 [deg] swing in phase surrounding 1 [kHz]. This is demonstrated in 2015-05-02_H1CAL-CS_Systematic.pdf, This is also reflected in the statistical uncertainty estimate. All comparisons are shown if a 389 Hz CCP and 2 kHz ESDP were used in 2015-05-02_389HzCCP_2p0kHzESDPole_H1DARMOLGTF.pdf. - Scale Factor: this depends on whether we want to use 22% or 3% for our scale factor uncertainty. If 22%, then the last two lock stretches still fall within the 1.1e6 +/ 22% [ct/m]. But, if the scale factor is 1.2725e6 (the mean of the last two scale factors), then that indeed falls outside of 1.1e6 +/-3% - We *still* have not implemented any compensation for the analog or digital, AA or AI filters. - The GDS/DMT produced calibration is off just as much as the CAL-CS produced calibration, because GDS/DMT calibration is using the same filters. ------------------ All parameter files have been committed (with updated ESD driver poles, and fitted DARM coupled cavity poles) here: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER7/H1/Scripts/ H1DARMparams_1109994128.m H1DARMparams_1111998876.m H1DARMparams_1112399129.m H1DARMparams_1112933759.m H1DARMparams_1112942996.m H1DARMparams_1113119652.m H1DARMparams_1114541595.m H1DARMparams_1114634170.m Which are processed by the DARM model function here: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER7/H1/Scripts/H1DARMmodel_preER7.m And then compared with the script here: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER7/H1/Scripts/CompareDARMOLGTFs.m
J. Kissel
At the requested of Gabriele, I've isolated/highlighted the change in measurements' cavity pole frequency in the attachment below. In it, I
(pg 1) have divided the open loop gain transfer function *measurement* by all components of the *model* for each measurement *except* for the frequency response of the IFO (which we have modeled as a single pole filter at the designated frequency). That includes
- The entire actuation function, A
- The entire control filter and gain, D
- The sensing function's non-ifo frequency dependence from the AA (both digital and analog) filters and the uncompensated OMC DCPD whitening poles
- The DC optical gain
(pg 2) Remind people that I've filtered out every data point but the most ridiculously coherent (coh = 0.99, nAvgs = 20), such that we can be certain that all frequency points used have sqrt( (1 - 0.99) / (2 * 20 * 0.99) ) = 1.6% uncertainty. Indeed, as shown by the historgram, and the vast majority have greater than 0.999 coherence, i.e. sqrt( (1 - 0.999) / (2 * 20 * 0.999) ) = 0.5% uncertainty.
J. Kissel, N. Kijbunchoo We've tried several more times to reacquire lock after the damping the rung up violin modes (LHO aLOG 18153), a slow but steady increase in wind up to the current 30-35 [mph] (see 1st attachment), the PSL tripping (LHO aLOG 18159), a mysterious guardian failure (LHO aLOG 18162), and now a rigorous trip of the ETMX seismic isolation because of what I think is my user error. We've run out of steam, and I don't think there's a point in continuing to battle the IFO under these terribly windy conditions. We'll start again tomorrow morning. Some details from the few lock acquisition attempts after the Guardian failure: Attempt 1: Just let the guardian try and do it's thing. Lost lock at the start of TRANSITION_TO_QPDs (but didn't realize it) Attempt 2: Noticed the recycling gain (as measured by ASAIR) was low, lost lock again Because I'd heard Kiwamu talking about low recycling gain vs. high recycling gain causing a sign flip in the ASC loops, I suspected the low recycling gain was the problem. So I had Nusinee play with the alignment of PR3, and the ITMs to try and get the ASAIR signal back up into the 600s (where is was in the 500s). She found that PR3 YAW was most effective (and the recycling gain has stayed in the 600s since she touched it up) -- we still lost lock on the TRANSITION TO QPDS. Attempt 3: After relocking, the recycling gain came up awesomely with out having to touch anything. Still lost lock at the same point. Attempt 4: This time, requested DRMI_LOCKED, such that we could go manually through each of the steps leading up to SWITCH_TO_QPDs. We got as far as the step right before -- REDUCE_CARM_OFFSET -- which completed. I was *just* about to hit go on the TRANSISTION_TO_QPDS when we lost lock. Then Nutsinee noticed that the ETMX SEI isolation system had tripped. After chasing down a bunch of WD trip and lock loss tools, we found the lock loss in the following order: HEPI 03:54:12 UTC (Actuators) IFO 03:54:13 UTC ISI 03:54:19 UTC (ST1 Actuators) TMSX 03:54:24 UTC Now, because I was messing around with the ISC_LOCK guardian in manual, I have a feeling it was me somehow sending a huge Tidal impulse to HEPI that took down the chamber but I can't be sure. Looking at the plot of the lock loss, it's certainly a huge impulsive spike that kicks the chamber, not like some slow shove from the wind or as if the Tidal was huge (again because of wind) and it suddenly hit the edge of the range. Anyways, I don't think there's something systematically wrong with HEPI that we need to freak out about, this was one lock loss of MANY over the past day, and my gut feeling tells me that the problem right now is that the TRANSITION_TO_QPDs fails while it's servoing the ALS-C_DIFF_PLL_CTRL_OFFSET and even during the REDUCE_CARM_OFFSET, because there's too much uncontrolled arm angular motion (from wind) for the CARM reduction to happen. According to the guardian logs, this servo seems to stress and eventually break the IMC lock, but I'm not sure if it's a cause or effect. We're gunna try to lock one more time, because getting to DRMI_LOCKED is incredibly robust, even in these high winds. BUT, we're not gunna log the result if negative and just go home.
Oh -- one more thing: we found something suspicious in the ISC_LOCK guardian, exactly in the TRANSITION TO QPDs's state definition: (Line 714) "ezca['LSC-TR_CARM_OFFSET'] = -3.3 #reduced from -3.3 when recylcing gain went from 29 to 40"" Seems strange that this OFFSET value would be the same as what the comment says it was reduced *from*. BUT -- this Guardian code hadn't changed since the ~3 hour lock stretch this morning, so I'm not sure if it's a problem. The lock losses happen during the only other thing of substance in this state, the self.servo of ALS-C_DIFF_PLL_CTRL_OFFSET using LSC-ASAIR_A_RF45_Q_NORM_MON as the readback channel.
Jeff switching from Manual to Exec (03:54:19) does not explain HEPI tripped (03:54:12). However, it does correspond to ISI tripped (03:54:19) which probably caused the TMSX tripped (03:54:24). Thus the kick on HEPI that caused a lock loss right before REDUCE_CARM_OFFSET is still a mystery...
Evan, Sheila
We have been able to lock the IFO at 10 Watts with a recycling gain of 37. We also think that we can probably improve our inital alingment scheme by adding a servo from POP A to PR3 durring the INput align step. We've closed loops around ITMX from the TRX QPDs, these were stable for almost an hour before we dropped the lock for other reasons, and helped keep the X arm build up stable as we increased the power.
Variation on Inital Alingment
At the beginning of the evening, our alignment resulted in a rather low recycling gain (15 or something) which resulted in difficulty locking. This was probably because the ITM camera references would have moved 18033. We have now gone through an initial alignment with some extra steps which brought us back to a decent recycling gain (35 before any manual adjustment or full lock ASC).
| PIT (urad) | Yaw | |
| PD1 | 169.2 | -311.4 |
| PD4 | 134.5 | -278.9 |
| mean | 151.85 | -295.15 |
This resulted in a recycling gain of about 35, so it seems like this was a sucsesful approach at least once. If there is time for some day time commissioning tomorrow, it would probably be helpful to add to the INPUT align state a loop which actuates on PR3 to center the beam on POP A. Commissioning the dither Align script for the BS to ETMY baffle PDs might also be helpful, but this is a lower priority because we aren't sure that step is necessary.
Since we have seen this week that many things (CARM offset reduction, violin and roll mode damping, and ASC error signals, possible radiation pressure instability on the ITMs) change when we go from a bad recycling gain to a good one, it seems like we will want to improve our initial alignment scheme enough to consistently bring us to a high enough recycling gain that we can use consistent settings for lock acquisition.
Avoiding oscillations durring power up
We ran into the oscillation in ITM pitch which we think is due to mis centering and radiation pressure on the ITMs several times tonight. It seems like we are more stable when the recycling gain is high, we have also slowed down the final CARM offset reduction and the power increase.
Closing ITM loops
We were able to close loops around ITMX from the QPD error signals we found last night. Here are settings:
DSOFT P (ITMX P) error signal 1.71 TRX A -1 TRX B FM offset 0.08, FM2,3,4 gain -700,000, top mass offloading on
DSOFT Y (ITMX Y) error signal 1.84 TRX A -1 TRX B FM offset -0.1, FM2,3,4,6 gain 600,000, top mass offloading on
both of these loops respond in a 2-3 seconds.
DARM OLG
Evan made a measurement of the DARM OLG, this was when we still on ETMX, DC readout with low frequency boost, and a recycling gain of 35. 11 Watts input power.
Now a large earthquake has tripped most of the seismic platforms.
While in full RF lock with a good recycling gain, turn off the green QPD servo and align the Y green beam to the arm manually (or using green WFS feeding back to PZT mirrors), then and set the QPD offsets.
Hopefully this will fix the ETMY green beam position, and makes Y arm initial alignment less convoluted.
The BS alignment slider values for PD1 and PD4 are switched in this entry. I redid the BS→EY baffle pointing today and found
| P (µrad) | Y (µrad) | |
| PD1 | 134.3 | −280.0 |
| PD4 | 169.5 | −314.0 |
| Mean | 151.9 | −297.0 |
Also, PD4 has a large dark offset (19.7 µW) compared to PD1 (−0.5 µW). The net power reported by PD4 with the PSL beam pointing onto it is 4.2 µW, and the for PD1 it is 5.2 µW. The settings for both are 0 dB gain, 0.5 A/W responsivity, and 20 kΩ transimpedance.