Folks have been complaining that the HAM5-ISI Rogue Excitation monitor is a pain. 
(see e.g. https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=21474)

It looks like the coil-voltage-readback monitor for the V2 coil is busted somewhere, and so the monitor is sitting around -500 counts all the time. 

Hugh gave me the GPS time for a recent earthquake, and in the attached plot you can see the watchdog trip from normal (state 1) to damp-down (state 2) at T+3 seconds. The coil voltages come down pretty quickly. Then the WD goes to state 4 (full trip) about 3 seconds later and the coil drive monitors (except V2) get quite small. The rogue excitation alarm goes off about 3 seconds after that, because the V2 monitor has not fallen to abs(Vmon) < +100 counts. 

The V2 monitor just sort of sits at ~-500 counts all the time. 
I'm pretty sure the V2 coil drive is working, otherwise the HAM5-ISI platform would act very poorly.
I'm guessing the problem is somewhere in the readback chain.

Note - the channels I use for this are all Epics channels, so the timing is a bit crude and the voltages are sort of jumpy. The channels are:
H1:ISI-HAM5_ERRMON_ROGUE_EXC_ALARM
H1:ISI-HAM5_CDMON_H1_V_INMON
H1:ISI-HAM5_CDMON_H2_V_INMON
H1:ISI-HAM5_CDMON_H3_V_INMON
H1:ISI-HAM5_CDMON_V1_V_INMON
H1:ISI-HAM5_CDMON_V2_V_INMON
H1:ISI-HAM5_CDMON_V3_V_INMON
H1:ISI-HAM5_WD_MON_STATE_INMON

I've also attached screenshots of the "coil drive voltage too big" calculation and the "rogue excitation alarm generation" calculation from the HAM-ISI master model

Title: 9/21 Day Shift 15:00-23:00 UTC (8:00-16:00 PST).  All times in UTC.

State of H1: Observing

Shift Summary:  Locked for most of the first half of my shift until another EQ in Chile took us down.  Took a few hours to ring down enough to relock.  Relocked once for ~half an hour, lost lock, and relocked again.  After performing initial alignment after the first lockloss, I noticed that the ALS-Y_DOF3 gains were once again set to 0.015 rather than 0.030.  I reverted them in SDF.

Incoming operator: Jim

Activity log:

14:43 Ryan notices a prop airplane flying over the X arm.

17:53 Lockloss due to EQ in Chile.

19:30 Start initial alignment after EQ has rung down.

19:50 Start lock acquisition.

20:28 Visitor at gate to see Bubba regarding fire inspection.

21:00 ISS diffracted power is reported as being low.  I adjust it back to ~8%.

21:55 Locked in Observing.  Some work was done by Sheila on PRMI guardian during the downtime. 

22:17 Lockloss.  ITMx saturation.

22:30 Noticed H1ASC had a red box in the CFC column.  Asked Sheila who said she had changed a filter.  She loaded the coefficients which greened it back up.  She then realized it was incorrect and changed it back.  I reloaded the coefficients once again.  Took IFO out of Observing Mode for a minute or two for this.

22:40 Lock in Observing.

22:57 DIAG_MAIN node in error.

Evan, Sheila, Travis,

Travis had a problem locking, for a reason that was at first mysterious, the OMC-ASC_MASTERGAIN was set to -0.05.  We looked back and this has happend a few times in the last month.  Evan looked into the OMC guardian and though that this could be because of a race condition between two cdsutils.ezcasteps in teh DITHER_ON state.  We've added two sleeps that should prevent this in the future.  

VerbalAlarms reports that DIAG_MAIN guardin node is in error.

I edited the ASC DHARD Yaw filter.  I added a less aggresive boost which we should be able to try engaging to see if we can ride out ground motion better (WP 5505).  We won't try engaging this filter unless there is an earthquake on the way or if we are locked when LLO is down. 

Locked in Observing Mode @ 22:40 UTC.

Lockloss @ 22:17.  ITMx saturation?

Locked in Observing Mode @ 21:55 UTC.

J. Kissel on behalf of P. Fritschel & M. Wade

Peter and Maddie have been trying to understand the discrepancies seen between the CAL-CS front-end calibraton and the (currently offline running) of the GDS pipeline -- see Maddie's comparisons in LHO aLOG 21638.

Peter put together an excellent summary on the calibration mailing list that's worth reproducing here because it motivates changing the actuation path delay in the CAL-CS model, which we intend to do tomorrow. We will change the actuation delay to it's current 4 clock cycles to Peter's suggested 7 clock cycles.

On Sep 17, 2015, at 6:39 PM, Peter Fritschel  wrote:

Maddie, et al.,

I spent some time looking into this (GDS vs CALCS) today, and I think I have a few
insights to share.

Bottom line is that I think the GDS code is doing the right thing, and that the corrections
[to the front-end calibration that are used] make sense given the way things are done. And, I think there is a simple
way to make the CAL-CS output get closer to the GDS output.

As Maddie pointed out, the amplitude corrections we are seeing from the GDS code in the
bucket (50-300 Hz or so) are caused mainly by the phase from the anti-alias (AA) and 
anti-image (AI) filters, which are accounted for in the GDS model but not in the CAL-CS one. 

Maddie already gave some numbers for 100 Hz, and pointed out that the relative phase shift
she is applying (16.4 degrees) is 8 degrees larger than the relative phase shift that
the CAL-CS model applies (8.8 degrees, from 244 usec). I’m referring to the relative 
phase shift between the DELTAL_CTRL and DELTAL_RESIDUAL signals.

The first thing to note is that this difference is going to have different effects on the
L1 and H1 GDS calibration, because they have different DARM open loop gain transfer functions. 

The simple picture for the region we are talking about is we are looking at the errors
in the sum: 1 + a*exp(i*phi), as a function of small changes in phi. Here, the ‘1’ represents
the DARM error signal, ‘a’ represents the DARM control signal, and is less than one (but 
not much smaller than 1). ‘phi’ is the relative phase between the two channels, and it is
errors in this phase (or small changes to) that we are talking about. The magnitude of the
sum is most sensitive to changes in phi for phi = 90 deg. So to bound the effect, assume
phi = 90 deg. At this point, the sensitivity is approximately:

   d|sum|/dphi = a

Sticking with 100 Hz as an example, the error in phi that GDS is correcting is 8 degrees,
or phi = 0.14 rad. ‘a’ is the DARM open loop gain at 100 Hz, which is different for L1 
and H1:

   L1, a = 0.6 —>  d|sum| = 0.084
   H1, a = 0.4  —> d|sum| = 0.056

These are the maximum possible errors, depending on ‘phi’. Maddie’s latest plots show a
correction at 100 Hz of 7% for L1, 3.5% for H1. Quite understandable.

For higher frequencies, the phase error is going to increase, but ‘a’ (open loop gain)
will decrease, so you need to look at both.

At these frequencies the phase shift/lag from the AA and AI filters (digital and analog)
is linear in frequency, so we can easily make the extrapolations. 

Maddie’s comparison plot shows that the biggest relative difference is at 250 Hz, where it
is 9%. At 250 Hz, the phase shift error is going to grow to (250/100)*8 = 20 deg = 0.35 rad.
For L1, the DARM OLG at 250 Hz is about 0.3 in magnitude (a). So the maximum error is:

  d|sum| = 0.105 = 10.5%. (vs. 9% observed)

For H1, Maddie’s plot shows a relative difference of about 8% at just below 300 Hz- say 280 Hz.
The phase shift error will be (280/100)*8 = 22.4 deg = 0.4 rad. The H1 OLG at 280 Hz is about
0.2 in magnitude. So the maximum error would be:

   d|sum| = 0.08 = 8%. (vs. 8% observed)

I think the frequencies where the differences go to very small values in Maddie’s plots, like
150 Hz for LHO, are frequencies where phi = 0 mod pi, for which |sum| is to first order
insensitive to dphi. 

OK, so now I can believe that it is realistic to see the kinds of amplitude corrections
that Maddie is seeing, in ‘the bucket’. 

However, the above picture also suggests how CAL-CS should be able to get much closer to
the GDS output. The frequencies where this is an issue is where ‘a’ (OLG magnitude) is not
too small. But at these frequencies (below ~500 Hz), the phase lags from the AA/AI filters are 
very nearly linear in frequency. Thus, they can be well approximated by a time delay.

So here’s the suggestion: Why not increase the time delay that is applied in the CAL-CS 
model to approximate the AA/AI filter effects? Adding 3 more sample delays would come close:

   3 sample delay = 183 usec; phase shift at 100 Hz = 6.6 degree

Travis pointed out another problem that comes up when we use the DRMI guardian to lock PRMI, and instead of writting more conditionals into the tDRMI states I decided to make some PRMI states to automate this.  

To do this I split the LOCK_DRMI state into two, the first part called LOCK_DRMI 1F is the main of the old LOCK_DRMI_1F state, which clears any history left over from a failed DRMI lock and sets everythign up to acquire.  Then there is a state DRMI_LOCK_WAIT in which the gaurdian just waits for DRMI to acquire, and then it moves through the rest of the graph with no change.  

The two new states are PRMI_LOCK_WAIT, which is the same as DRMI except that it turns off the input, output and offset on the SRCL filter.  OFFLOAD PRMI engages the offlaoding to the top stage of PRM.  If SRM is realigned, it wil wait 20 seconds, turn on SRCL feedback, and return DRMI_LOCK_WAIT.

This is not intended to be a normal part of acquisition, it is just meant to make the normal procedure for locking PRMI when the DRMI alignment is bad a little easier.  

To use it: 

1) You can either misalign SRM using the SUS_SRM guardian, or use a new state in ALIGN_IFO called SET_SUS_FOR_PRMI_W_ALS. 

2) Request OFFLOAD_PRMI from the DRMI guardian.

3) Once PRMI locks adjust PRM and BS alignment until you get about 80 counts on POP90 and 50 counts on POP18.  

4)Realign SRM by undoing whatever change you made in step 1.

5) Request DRMI_1F_OFFFLOADED from the ISC_DRMI guardian.

We have approval to use the following waveform for testing the hardware injection infrastructure. We want to check that the ODC bits and logging for hardware injections is correct.

The single-column ASCII file that can be injected is committed to the SVN. It can be found here: ASCII waveform file

The parameters of the waveform can be found in the SVN as well: XML parameter file

We are currently down due to EQ in Chile.  It is starting to ring down enough to start attempting to relock. 

J. Kissel, J. Betzwieser

No rocket science here, but I wanted to post to an aLOG such that we have a convenient reference to cite. Recall that the latest ESD linearization algorithm is described in G1500036, and we're using a simplified version of said algorithm because we do not compensate for charge, namely, when the ii'th linearized quadrant's output is
Vii = Vb * (1 - sqrt( 2 * ( (Fii / Vb^2) * G_ct + 1 ) ) )
where Vb is the bias voltage, Fii is the digitally requested ii'th quadrant's force input to the linearization algorithm, and G_ct is the EPICs record representing the linear force coefficient.

Since even this simplified linearization function is rather daunting to gather intuition about signs, I've made a few plots demonstrating how the algorithm plays with a 100 [Hz] frequency, 10 [ct] amplitude sine wave input request, showing both the output of the algorithm and the resulting force on the optic. The 10 [ct] was chosen because that's roughly what I see requested out of LOCK bank during low noise for both detectors. I've used LLO's numbers for the force coefficient EPICs record (G_{ct} = +/- 512000 [ct]) and bias voltage (Vb = +/- 127999 [ct]). 

As the title suggests, the sign of the output of the linearization algorithm signal gets flipped w.r.t. its input, regardless of the bias sign. 

This impacts the reconstruction of CAL-CS and therefore h(t),  in that LLO uses linearization and LHO does not, and it hasn't been included in the matlab model of the IFO to-date. It also is a piece to the missing sign puzzle (LHO aLOGs 21627 and 21703): there are therefore three polarity flips one must deal with in the ESD chain: the sign of the bias voltage, whether the linearization is on or off, and that the ESD itself is an inherently attractive actuator. It's those pieces that hadn't been considered in detail, which has result in the confusion between LLO and LHO's reconstruction of the actuation path -- at least on the TST / L3 stage.

Lockloss @ 17:53 UTC.  Likely due to 6.5 magnitude EQ in Chile.

LDAS switch has failed over the weekend, due to be replaced.  This caused several sites to be unavailable (Omega glitch plots and summary pages to name a few).

VE will need access to X-2-8 this week.  Will notify operators.

HFD on site tomorrow for annual fire inspection.

Air compressor isolators in hand.  Installation on hold until further notice.

Tuesday maintenance activities scheduled for tomorrow:

• TCS chiller forklifting in mechanical room
• Solar panel install at end stations
• supply fan 3 bearing repair
• LN2 delivery (one is also scheduled for Thurs.)
• GDS pipeline install for calibration
• SYS DIAG guardian work
• photo record of CDS racks
• sweep of LVEA for things not attached to IFO
• PEM STS2 vault maintenance
• HEPI fluid level check

There is a problem with the network switch connecting LDAS to GC in the DCS room in the onsite warehouse.

This means the cluster head nodes, ldas-grid, ldas-pcdev1, detchar, and the nds2 server at LHO are unreachable.

This may be why the summary pages are not updating.

However the computers are up and jobs are running.

I've been to the site, talked with the operator, then walked from the control to the warehouse to check on the network switch. It appears to be dead. (See my email below.)

Also, ldas-pcdev2 and the web server (located in a different room) are up and reachable and can see the data for the summary pages. I've contacted Duncan Macleod to see if there is away to get the summary pages to update again, using the working connections.

We may not be able to replace the broken switch until tomorrow.

Note that this does not affect the flow of data to LDAS at LHO or CIT.

-------- Original Message --------
Subject: Re: [DASWG] Re: Problem with the head nodes at LHO
Date: Sun, 20 Sep 2015 20:39:06 -0700
From: Gregory Mendell <gmendell@ligo-wa.caltech.edu>
To: daswg@ligo.org
CC: LDAS_ADMIN_ALL <ldas_admin_all@ligo.caltech.edu>,  "detchar@ligo.org" <detchar@ligo.org>

On 9/20/15 7:16 PM, Gregory Mendell wrote:
> The problem appears to be that nds, ldas-grid, and ldas-pcdev1 are
> unreachable from outside the internal ldas network.
>
> These computers are up and jobs are running.
>
> I will be checking on the GC switch in the ldas room at LHO once I get
> out to the site (in about 30 minutes).

The switch has no link lights on any of its copper or fiber connections
and no indication it is getting power.

Power cycling and pushing reset button several time did not work.

Moving power cord to another circuit did not work. (Other switches and
computers are all up and showing lights on the same circuits, so it was
unlikely this would have worked anyways.)

I'm in touch with Dan Moraru. If we don't have a spare we'll have to
replace the broken switch tomorrow.

Regards,
Greg
 

It was noted recently elsewhere that there are a pair of lines in DARM near 41 Hz
that may be the roll modes of triplet suspensions. In particular, there is
a prediction of 40.369 Hz for the roll mode labeled ModeR3.

Attached is a zoom of displacement spectrum in that band from 50 hours of early 
ER8 data. Since one naively expects a bounce mode at 1/sqrt(2) of the roll mode,
also attached is a zoom of that region for which the evidence of
bounce modes seems weak. The visible lines are much narrower,
and one coincides with an integer frequency.

For completeness, I also looked at various potential subharmonics  and harmonics
of these lines, in case the 41-Hz pair come from some other source with non-linear coupling. 
The only ones that appeared at all plausible were at about 2/3 of 41 Hz.

Specifically, the peaks at 40.9365 and 41.0127 Hz have potential 2/3 partners at
27.4170 and 27.5025 Hz (ratios: 0.6697 and 0.6706) -- see 3rd attachment. The 
non-equality of the ratios with 0.6667 is not necessarily inconsistent with a harmonic
relation, since we've seen that quad suspension violin modes do not follow a strict harmonic 
progression, and triplets are almost as complicated as quads. On the other hand, I do not see
any evidence at all for the 4th or 5th harmonics in the data set, despite the comparable strain
strengths seen for the putative 2nd and 3rd harmonics. 

Notes:
* The frequency ranges of the three plots are chosen so that the two peaks would
appear in the same physical locations in the graphs if the nominal sqrt(2) and 2/3 relations were exact..
* There is another, smaller peak of comparable width between the two peaks near 27 Hz,
which may be another mechanical resonance.
* The 27.5025-Hz line has a width that encompasses a 25.5000-hz line that is part of a
1-Hz comb with a 0.5-Hz offset reported previously.

Daniel, Evan

We looked at the DARM spectrum in the vicinity of the OMC dither line (at 4100 Hz) and its second harmonic (at 8200 Hz) using the OMC DCPD IOP channels.

The spectrum around the line itself appears to be bilinear, perhaps as one would expect (if the OMC is exactly on resonance, there should be no first-order modulation of the transmitted power). The second harmonic of the line is quite clean, with no obvious upconversion. Is this what we should expect?