model restarts logged for Sat 27/Dec/2014
2014_12_27 06:27 h1fw1
2014_12_27 12:47 h1fw0
both unexpected restarts. Conlog reporting was offline yesterday.
I have just noticed that the corner station Hartmann Wavefront Sensor EPICS IOC server (h1hwsmsr) is not responding. It failed between 8pm and 9pm Wednesday night (12/24). If anyone needs the HWS for commissioning over the next week, please call me on my cell phone and we can try rebooting.
no restarts reported. Conlog frequently changing channels report attached.
model restarts logged for Wed 24/Dec/2014
2014_12_24 11:21 h1ioplsc0
2014_12_24 11:23 h1lscaux
2014_12_24 11:23 h1lsc
2014_12_24 11:23 h1omc
2014_12_24 11:25 h1iopiscey
2014_12_24 11:27 h1caley
2014_12_24 11:27 h1iscey
2014_12_24 11:27 h1pemey
2014_12_24 11:29 h1odcy
Recovery of LSC and ISC-EY from power glitch
model restarts logged for Thu 25/Dec/2014
2014_12_25 19:27 h1fw1
one unexpected restart over the two days. Conlog frequently changing channels list attached
Yend pumpdown.
Moving the right way now.
Tested the new off-loading feature for the integrator. The attached plot shows two changes in the bias value: the first near 4.5 sec is with the offload button off, the second transition near 8.4 sec is with the offload button on. In both cases the change is reflected on the integrator output value (OUT channel). In the first case, the bias change introduces a transient in the output drive (DRIVE). In the second case, no transient is observed and the integrator value is off-loaded into the bias value smoothly and instantaneously.
Had to restart the EY laser.
Looking at the EY SUS alignment it seems to have more or less recovered from the vent. The green return beam still hits the WFS, even so it somewhat high. Using bias sliders of 93.4/-50 for pitch and yaw for ETMY (75.6/-60.5 old) and 144.2/5.8 (no change) for TMSY steers the beam back onto the center of WFS_A.
The EX SUS alignment did not recover. I did not find the return beam. I only searched around the nominal good alignment, so. Trying to steer back by looking at the plots of SUS bias and oplev positions doesn't help. The EX VEA decided to increase its ambient temperature by 1°C.
Probably from the power glitch, the DAQ EPICS shows garbage data for iscey. EPICS readbacks are fine.
I’ve done some work on IOT2R in preparation for the Schnupp asymmetry measurement:
For the time being, I’ve turned off the PD and dumped the beam immediately after the periscope.
Next steps:
With regards to turning down the aux laser power rather than using a filter in front of the PLL photodiode: the reason we didn't do this before is because we still wanted to maintain maximum aux laser power injected to the PRC (which is unaffected by the filter in front of the PLL PD). If you get plenty of signal on ISCT1 still, then maybe this isn't a problem and turning down the aux laser power is fine.
Most DTS systems were glitched by last night's power event. x1fe3, x1seibsc and x1susaux are currently offline. I restarted all the models on the running front ends, and the DAQ. I'll leave the complete recovery until we return to the site after the holidays.
Daniel, Stefan, Dave
To clear the IOP DAC error bit in the state words for h1lsc0 and h1iscey, I restarted all the models on these two front ends remotely. Procedure was: stop all user models; restart iop model; start user models. On each startup, I manually set the BURT_RESTORE channel to 1 after about 3 seconds of running. Finally I pressed DIAG_RESET on all models to clear the IPC errors raised by the restarts.
The LHO GC UPS reports going onto battery power from 19:56:21 to 19:56:26 PST Tuesday 23rd December 2014. This is the time Jeff reports the PSL developed problems.
Looking at the CDS overview this morning (attached) several systems are reporting problems. I press the DIAG_RESET button on all models and got the second MEDM screen attached.
Looks like we are having DAC errors on LSC and ISCEY. We will most probably have to restart all the models on these front ends to clear. If any commissioners are on site today, could you call my cell phone and we can walk through this.
At approximately 7:56 last night the Substation in the 400 area that feeds LHO had an event. The protective relay saw a fault on the BPA line that feeds it and tripped off. The reclosure closed immediately and the power was restored within 1 cycle. Only the most sensitive systems would have seen this. DOE is not aware of what caused the relay to trip as everything restored without incident.
model restarts logged for Tue 23/Dec/2014
2014_12_23 10:29 h1iscex
2014_12_23 11:43 h1iscex
2014_12_23 11:44 h1iscey
2014_12_23 11:47 h1broadcast0
2014_12_23 11:47 h1dc0
2014_12_23 11:47 h1fw0
2014_12_23 11:47 h1fw1
2014_12_23 11:47 h1nds0
2014_12_23 11:47 h1nds1
2014_12_23 12:15 h1iopsusquadtst
2014_12_23 12:15 h1susquadtst
2014_12_23 14:39 h1iscex
2014_12_23 14:39 h1iscey
2014_12_23 14:44 h1broadcast0
2014_12_23 14:44 h1dc0
2014_12_23 14:44 h1fw0
2014_12_23 14:44 h1fw1
2014_12_23 14:44 h1nds0
2014_12_23 14:44 h1nds1
no unexpected restarts. Maintenance day builds of ISC EX,EY with corresponding DAQ restarts. Restart of susquadtst following DAQ problems when powering it down. Beckhoff restarts shown in attachment.
Conlog frequently changing channels report attached.
At John's suggestion, I checked the turbo bypass valve and found that it was still open (from rough pumping phase) -> Closed bypass valve -> OK This is the first time that I have overlooked this in the "many" times that I have performed this procedure (there's a first time for everything!). This proved (to me) to be a surprising explanation of the symptom since the bypass circuit is ~2' of 1.5" tubing (+ two 90s) connecting the turbo exhaust to the turbo inlet. As such, I would have never guessed that a compression ration of ~500 could have been maintained with such a large conductance between exhaust-intake (who knew!) Also, isolated pump carts from BSC9 and BSC10 annulus volumes -> Both systems pumped only by ion pumps
~0635 - 0650 hrs. local -> In and Out of X-end VEA ~0700 - 0710 hrs. local -> In and Out of Y-end VEA
J. Kissel Just after the commissioning vanguard left for the evening (2014-12-24 03:57 UTC, Dec 23 2014 19:57:00 PST, 1103428636) the PMC / FSS went down, and it's been flashing and struggling to recover since. I attach a few relevant screen shots. Wish I could diagnose / fix... Merry Christmas to all, and to all a good night!
PMC back in operation. PZT HV had tripped. Would be nice to have a clear text error message...
J. Kissel I've confirmed that the ESD on H1 SUS ETMY (turned on earlier today, see LHO aLOG 15809) is functional by driving a 4 [Hz], 30000 [ct_pk] sine wave into each quadrant using awggui, and measuring the response in the ETMY optical lever. There is clear amplitude and coherence in pitch and yaw when driving each quadrant. Other less exciting notes, regarding the functionality of the whole ESD system: - Remember that the front-end code is still in correctly ordered, but Richard McCarthy assures me that instead of wiring up what had made sense in analog, he has switched the inputs at the ESD driver such that the front end drives the right channels, i.e. the front end's DC bias, which drives the ESD driver's channel 1 is connected to pin 3, all the way to the ESD pattern in chamber as per Filiberto's aLOG (see LHO aLOG 15656), and the remaining quadrants are hooked up identically to how they're hooked up at ETMX and in the two LLO quads. Once Rich Abbott solidifies a drawing of what has been implemented at H1 EY, then the changes will be propogated elsewhere and the front end models will be fixed. Maybe, eventually. But the important statement, again, is that all 4 ETMs are now wired up in the same fashion, such that the front-end drives each quadrant in the same way. Until the analog monitors are functional and I get Rich's drawing, I don't want to add more confusion by attempting to make a channel order. Thank god, when it counts, we only ever use these things in longitudinal driving all four quadrants at once. - The ESD analog monitor channels, e.g. H1:SUS-ETMY_L3_ESDAMON_LL_MON, are completely non-functional, they don't report change regardless of bias value or excitation on each quadrant. - The ESD driver's digital monitor signals also don't report anything but ADC noise, *except* for the "MCU" channel, H1:SUS-ETMY_L3_ESDDMON_MCU_MON, which is a 1 [Hz], 3460 [ct_pkpk] sine wave. DTT Templates for this measurement live here: /ligo/svncommon/SusSVN/sus/trunk/QUAD/H1/ETMY/SAGL3/Data/ 2014-12-23_H1SUSETMY_ESD_biasp9p5V_excLL.xml 2014-12-23_H1SUSETMY_ESD_biasp9p5V_excLR.xml 2014-12-23_H1SUSETMY_ESD_biasp9p5V_excUL.xml 2014-12-23_H1SUSETMY_ESD_biasp9p5V_excUR.xml
J. Kissel, R. McCarthy
At my request, after seeing that the EY BSC 10 vacuum pressure has dropped below 1e-5 [Torr] (see attached trend), Richard has turned on the H1 SUS ETMY ESD at ~2pm PST. I'm continuing to commission the chain, and will post functionality results shortly.
Also --
I've found the ESD linearization force coefficient (H1:SUS-ETMX_L3_ESDOUTF_LIN_FORCE_COEFF) to be -180000 [ct]. I don't understand from where this number came, and I couldn't find any aLOGs explaining it. I've logged into to LLO, their coefficient is -512000 [ct]. There's no aLOG describing their number either, but I know from conversations with Joe Betz in early December 2014 that he installed this number when the LLO linearization was switched from before the EUL2ESD matrix to after. When before the EUL2ESD matrix the coefficient was -128000 = - 512000/4 so we was accounting for the factor of 0.25 in EUL2ESD matrix. I suspect that -128000 [ct] came from the following simple model of longitudinal force, F_{tot} on the optic as a result of the quadrant's signal voltage, V_{S} and the bias voltage V_{B}, (which we know is incomplete now -- see LLO aLOG 14853):
F_{tot} = a ( V_{s} - V_{B} )^2
F_{tot} = a ( V_{s}^2 - 2 V_{s} V_{B} + V_{B}^2)
F_{lin} = 2 a V_{s} V_{B}
where F_{lin} is the linear term in the force model, and a< is the force coefficient that turns whatever units V_{S} and V_{B} are in ([ct^2] or [V_{DAC}^2] or [V_{ESD}^2]) into longitudinal force on the test mass in [N]. I *think* the quantity (2 a V_{B}) was mistakenly treated as simply (V_{B}) which has always been held at 128000 [ct] (or the equivalent of 390 [V] on the ESD bias pattern) and the scale factor (2 a) was ignored. Or something. But I don't know.
So I try to make sense of these numbers below.
Looking at what was intended (see T1400321) and what was eventually analytically shown (see T1400490), we want the quantity
F_{ctrl}
-------
2 k V_{B}^2
to be dimensionless, where F_{ctrl} is the force on the optic caused by the ESD. Note that comparing John / Matt / Den's notation against Brett / Joe / my notation, k = a. As written in T1400321, F_{ctrl} was assumed to have units of [N], and V_{B} to have units of [V_{esd}], such that k has units of [ N / (V_{esd}^2) ], and it's the number we all know from John's thesis, k = a = 4.2e-10 [N/V^2]. We now know the number is smaller than that because of the effects of (we think) charge (see, e.g. LHO aLOG 12220, and again LLO aLOG 14853).
In the way that the "force coefficient" has been implemented in the front end code -- as an epics variable that comes into the linearization blockas "Gain_Constant_In," (see attached) -- I think the number magically works out to be ... close. As implemented, the linearized quadrant's signal voltage is as shown in Eq. 13 of T1400490, except that the EPICs record, we'll call it G, is actually multiplied in
V_{S} = V_{C} + V_{B}(1 - sqrt{ 2 [ (F_{ctrl} / V_{B}^2) * G + 1 + (V_{C}/V_{B}) + (V_{C}/V_{B})^{2} * 1/4 ] )}
Note, that we currently have all of the effective charge voltages set to 0 [ct], so the equation just boils down to the expected
V_{S} = V_{B}(1 - sqrt{ 2 [ (F_{ctrl} / V_{B}^2) * G + 1] )}
which means that
G == 1 / (2 k) or k = 1 / (2 G)
and has fundamental dimensions of [V_{esd}^2 / N]. So let's take this "force coefficient," G = -512000 [ct], and turn into fundamental units:
G = 512000 [ct] {{LLO}}
* (20 / 2^18) [V_{dac} / ct]
* 40 [V_{esd} / V_{dac}]
* 1 / (V_{B} * a) [(1 / V_{esd}) . (V_{esd}^{2} / N)]
G = 9.5391e9 [V_{ESD}^2 / N]
==>
k = 5.37e-11 [N/V_{ESD}^2] {{LLO}}
where I've used V_{B} = 400 [V_{esd}] and the canonical a = 4.2e-10 [N/V_{esd}^2] originally from pg 7 of G0900956. That makes LLO's coefficient assume the actuation strength is a factor of 8 lower from the canonical number. For the LHO number,
G = 180000 [ct] {{LHO}}
* (20 / 2^18) [V_{dac} / ct]
* 40 [V_{esd} / V_{dac}]
* 1 / (V_{B} * a) [(1 / V_{esd}) . (V_{esd}^{2} / N)]
G = 3.2697e9 [V_{ESD}^2 / N]
==>
k = 1.53e-10 [N/V_{ESD}^2] {{LHO}}
Which is within a factor of 3 lower, and if the ESD's as weak as we've measured it to be it may be dead on. So maybe whomever stuck in 180000 is much smarter than I.
For now I leave in 180000 [ct], which corresponds to a force coefficient of a = 1.53e-10 [N/V_{ESD}^2].
B. Shapiro, J. Kissel As usual, two heads are better than one when it comes to these nasty dealings with factors of two (go figure). Brett has caught a subtlety in the front-end implementation that further makes it different from the analytical approach used in T1400321 and T1400490. In summary, we now agree that the LLO and LHO EPICs force coefficients that have been installed are closer to the measured values by a factor of 4, i.e. G = 512000 [ct] ==> k = 2.0966e-10 [N/V^2] {{LLO}} and G = 180000 [ct] ==> k = 6.1168e-10 [N/V^2] {{LHO}} which means, though they still differ from the canonical value (from pg 7 of G0900956) k = 4.2e-10 [N/V^2] {{Canonical Model}} and what we've measured (including charge) (see LHO aLOG 12220, and LLO aLOGs 14853 and 15657) k = 2e-10 +/- 1.5e-10** [N/V^2] {{Measured}} they're much closer. **I've quickly guesstimated the uncertainty based on the above mentioned measurement aLOGs. IMHO, we still don't have a systematic estimate of the uncertainty because we've measured it so view times, in so many different ways, infrequently, and with the ion pumps still valved in, and each test mass has a different charge mean, charge location, and charge variance. Here's how the aLOG 15809 math should be corrected: The F_{ctrl} and k = a in the analytic equations is assumed to be for full longitudinal force. However, as implemented in the front end, the longitudinal force F_{ctrl} has already been passed through the EUL2ESD matrix, which splits transforms into quadrant basis force F_{ii}, dividing F_{ctrl} by 4. The EPICs force coefficient, G, should therefore *also* be divided by 4, to preserve the ratio F_{ctrl} F_{ii} ------- = ------------ 2 k V_{B}^2 2 k_{ii} V_{B}^2 inside the analytical linearization algorithm. In other words, as we've physically implemented the ESD, on a quadrant-by-quadrant basis, F_{ctrl} = F_{UL} + F_{LL} + F_{UR} + F_{LR} where F_{ii} = k_{ii} (V_{ii} - V_{B})^2 and k_{ii} = k / 4 = a / 4. As such, the implemented front-end equation V_{ii} = V_{B}(1 - sqrt{ 2 [ (F_{ii} / V_{B}^2) * G + 1] )} means that G == 1 / 2 k_{ii} = 2 / k = 2 / a and still has the fundamental units of [V_{esd}^2 / N]. So nothing changes about the above conversation from G in [ct] to G in [V_{esd}^2 / N], its simply that the conversion from G to the more well-known analytical quantity k was off by a factor of 4, G = 512000 [ct] {{LLO}} * (20 / 2^18) [V_{dac} / ct] * 40 [V_{esd} / V_{dac}] * 1 / (V_{B} * a) [(1 / V_{esd}) . (V_{esd}^{2} / N)] G = 9.5391e9 [V_{ESD}^2 / N] ==> k = 2.0966e-10 [N/V_{ESD}^2] {{LLO}} where I've used V_{B} = 400 [V_{esd}] and the canonical a = 4.2e-10 [N/V_{esd}^2] originally from pg 7 of G0900956. That makes LLO's coefficient assume the actuation strength is a factor of 2 lower from the canonical number, pretty darn close to the measured value and definitely within the uncertainty. For the LHO number, G = 180000 [ct] {{LHO}} * (20 / 2^18) [V_{dac} / ct] * 40 [V_{esd} / V_{dac}] * 1 / (V_{B} * a) [(1 / V_{esd}) . (V_{esd}^{2} / N)] G = 3.2697e9 [V_{ESD}^2 / N] ==> k = 6.1168e-10 [N/V_{ESD}^2] {{LHO}} both of which are closer to the measured value as described above.
N. Smith, (transcribed by J. Kissel)
Nic called and fessed up to being the one who installed the -180000 [ct] force coefficient at LHO. Note -- this coefficient only is installed in ETMX, the ETMY coefficient is still the original dummy coefficient of 1.0 [ct].
He informs me that this number was determined *empirically* -- he drove a line at some frequency, and made sure that the requested input amplitude (driven before the linearization algorithm) was the same as the requested output amplitude (the MASTER_OUT channels) at the that frequency, with the linearization both ON and BYPASSED. He recalls measuring this with a DTT session, not just looking at the MEDM screen (good!).
Why does this work out to be roughly the right number? Take a look at the front-end equation again:
V_{ii} = V_{B}(1 - sqrt{ 2 [ (F_{ii} / V_{B}^2) * G + 1] ) } )
and let's assume Nic was driving V_{ii} at a strength equal and opposite sign to the bias voltage V_{B}. With the linearization OFF / BYPASSED,
V_{ii} = - V_{B}
Duh. With the linearization in place,
V_{ii} = - V_{B} = - V_{B} (1 - sqrt{ 2 [ (F_{ii} / V_{B}^2) * G + 1] ) } )
so we want the quantity
(1 - sqrt{ 2 [ (F_{ii} / V_{B}^2) * G + 1] )} = 1
which only happens if
(F_{ii} / V_{B}^2) * G = 1.
If Nic wants to create a force close to the maximum, it needs to be close to the maximum of
F_{ii,max} = 2 k_{ii} V_{B}^2,
which makes
2 k_{ii} * G = 1
or
G = 1 / (2 k_{ii}) = 2 / k
which is the same result as in LHO aLOG 15873. Granted, it's late and I've waved my hands a bit, but this is me trying to justify why it feels like it makes sense, at least within the "factor of two-ish" discrepancy between the canonical value and the accepted measurements of the right number.
I've summarized this exploration of Linearization Science in G1500036.