Correction on Sep 1:

Some zero/pole numbers in the original entry was incorrect, so I made some corrections below.

Specifically, ILPD has zp=([71mHz;72mHz;2.7k],[3.1;3.4;130;2.3k]) (130 and 2.7k were swapped in the original entry). Plots were updated accordingly.

Summary:

• The outer loop OLTF was measured to be very small.
• That's because the inner loop whitening compensation is NOT implemented in the prototype board. This needs to be implemented.
• (This is implemented in the original (old) second loop board.)

What's done and found:

• Polarity of the board was found to be wrong so I and Fil pulled the chassis and flipped the sign using a jumper.
• After the polarity fix, I measured the OLTF of the outer loop servo with 2W PSL output, AC coupling servo enabled, and the maximum board gain of 45.6dB (first attachment).
  • Don't try to learn too much from this "measurement" as I was saturating the electronics, this is not meant to be a precise measurement. But the point is that this is very, very small.
• Unfortunately it turns out that this is what you get when you don't implement the zero-pole correction that I mentioned in point 2. of alog 29293 and its attachment entry.
  • Gist of it is that the intensity noise sensing gain for the inner loop PD is huge for f>3Hz because of double 70mHz zero, and the outer loop board gain needs to be even bigger in order for the outer loop to have any reasonable gain.
• After the above mentioned measurement, negative leg of the outer loop differential output was found to be grounded to the chassis on the output connector. This is not why the OLTF is very small, but we need to fix it.

These things were mostly done during Tuesday maintenance.

Some more explanation:

Since the outer loop injects its signal to the inner loop error point, OLTF of outer loop is basically the ratio of two transfer functions, one from RIN to the output of the outer loop servo board (H_OLBoard) and the other from RIN to the inner loop error point (H_ILPD):

OLTF_OL=H_OLBoard/H_ILPD.

As I noted before (alog 26879 and an entry attached later to that one), the inner loop diode is equivalent of 660 Ohm DC trans impedance and zp=([71mHz;72mHz;2.7k],[3.1;3.4;130;2.3k]) . DC output of this inner loop diode is about 1.6V differential (measured). For details you need to look at D1001998.

H_IL = 1.6[V]*zp([71mHz;72mHz;2.7k],[3.1;3.4;130;2.3k]).

OTOH, outer loop trans impedance amplifier doesn't have any coloring in the new prototype (the old one has this compensation, see D1300639), and each diode outputs about 130mV or so at 2W. There are four PDs, so the sum of all PDs at DC is about 520mV at 2W. The outer loop chassis has a flat gain of 3, a variable gain (VGA) with -12.8 to 45.6dB range, and a servo filter SERVO=zp([600;9k],[20,20,380]) with the DC gain of 1 (see alog 29293).  (There should be another factor of 2 from SE to DIFF conversion, but the negative leg of DIFF is grounded, so this is not there at the moment.). There's also a MC pole (9k) in H_OLBoard.

H_OLBoard(2W) = 0.52[V]*3*IMC*VGA*SERVO = RIN*1.6[V]*VGA*zp([600],[20,20,380]).

Combining these,

OLTF_OL @2W = H_OLBoard@2W/H_ILPD ~ zp([71mHz;72mHz;2.7k],[3.1;3.4;130;2.3k])/zp(600, [20;20;380])/VGA.

Thing is, H_IL is absolutely huge at 3Hz and higher due to double 70mHz zeros, and the outer loop board as of now doesn't have the gain necessary to keep the UGF at some kHz.

Second attachment bottom shows the above theoretical OLTF at 2W with the maximum (45.6dB) VGA and the measured data. Remember that the "measured" is no good measurement because it was saturating, but nevertheless you can see that the measurement is not totally unreasonable.

It's clear to that we need something in the outer loop to counter 70mHz double zeros.

Implementing that would also help AC coupling. As of now, the AC coupling is already sort of aggressive, but not aggressive enough. If I bring up the blue curve in the second plot bottom by 80dB or so to set the outer loop higher UGF at 2kHz, the lower UGF would become .2Hz or so, that might not be good enough of an AC coupling.

12:37 PDT DAQ restart to add following VAC channels

+[H0:VAC-LX_IBM_II122_AIP_IC_LOGMA]
+[H0:VAC-LX_IBM_II122_AIP_IC_LOGMA_ERROR]
+[H0:VAC-LX_IBM_II122_AIP_IC_MA]
+[H0:VAC-LX_IBM_II122_AIP_IC_MA_ERROR]
+[H0:VAC-LX_IBM_II122_AIP_IC_VOLTS]
+[H0:VAC-LX_IBM_II122_AIP_IC_VOLTS_ERROR]
+[H0:VAC-LX_IBM_VI122_AIP_PRESS_TORR]
+[H0:VAC-LX_IBM_VI122_AIP_PRESS_TORR_ERROR]
+[H0:VAC-LX_OBM_II136_AIP_IC_LOGMA]
+[H0:VAC-LX_OBM_II136_AIP_IC_LOGMA_ERROR]
+[H0:VAC-LX_OBM_II136_AIP_IC_MA]
+[H0:VAC-LX_OBM_II136_AIP_IC_MA_ERROR]
+[H0:VAC-LX_OBM_II136_AIP_IC_VOLTS]
+[H0:VAC-LX_OBM_II136_AIP_IC_VOLTS_ERROR]
+[H0:VAC-LX_OBM_VI136_AIP_PRESS_TORR]
+[H0:VAC-LX_OBM_VI136_AIP_PRESS_TORR_ERROR]
+[H0:VAC-LY_IBM_II192_AIP_IC_LOGMA]
+[H0:VAC-LY_IBM_II192_AIP_IC_LOGMA_ERROR]
+[H0:VAC-LY_IBM_II192_AIP_IC_MA]
+[H0:VAC-LY_IBM_II192_AIP_IC_MA_ERROR]
+[H0:VAC-LY_IBM_II192_AIP_IC_VOLTS]
+[H0:VAC-LY_IBM_II192_AIP_IC_VOLTS_ERROR]
+[H0:VAC-LY_IBM_VI192_AIP_PRESS_TORR]
+[H0:VAC-LY_IBM_VI192_AIP_PRESS_TORR_ERROR]
+[H0:VAC-LY_OBM_II196_AIP_IC_LOGMA]
+[H0:VAC-LY_OBM_II196_AIP_IC_LOGMA_ERROR]
+[H0:VAC-LY_OBM_II196_AIP_IC_MA]
+[H0:VAC-LY_OBM_II196_AIP_IC_MA_ERROR]
+[H0:VAC-LY_OBM_II196_AIP_IC_VOLTS]
+[H0:VAC-LY_OBM_II196_AIP_IC_VOLTS_ERROR]
+[H0:VAC-LY_OBM_VI196_AIP_PRESS_TORR]
+[H0:VAC-LY_OBM_VI196_AIP_PRESS_TORR_ERROR]
 

I was about to take a screen shot of the beams from the various PSL cavities when all of a
sudden the screen went black.  The laser tripped.  I haven't performed any forensics but it
looks like the same problem as last night.

On Monday, I went to EndX to retrieve data from the anemometers I had around the wind fence, and found the set up kind of trashed. The ultrasonic gust sensor had been blown over, which sounds like a common issue with that setup from the past. The fall, however, broke some of the plastic pieces that the screws holding the head of the sensor together. I think the sensor is still functional, but it will need to be glued back together.

One of the other cup anemometers may have been attacked by a bird, or possibly a vehicle. The cable had been unplugged from the sensor, drug more than 20 feet and the sheath and a couple wires had been cut. I would normally blame a bird for that, but the copper tube I had secured the anemometer to was also bent. It's possible that a gust of wind had bent the post, but neither of the other posts were similarly bent. There are tire tracks up around the fence, but it's hard to say how old they are, it's likely they are left over from the fence install, and none of them come very close to the post. Finally, the batteries in the logger for the 3 cup anemometers had worked free of their connections, so the logger only collected about 3 days worth of data, even though they were installed for a week and a half. The batteries are held in place by a plastic strap, and this strap probably got soft in the sun, which allowed the batteries to work they way free.

I don't know what we can do to keep birds (or bees, they seem to really like the fence) away from the outside instruments, but couple of other things I'll get for next time:

1. Cones or flags to mark a stay clear around the fence.

2. Weights and guy lines for the ultrasonic sensor, after I get it back together.

3. A foam shim to secure the batteries in the cup anemometer datalogger.

Kyle, Chandra

Starting step down of heating of Vertex RGA -> Closed calibration gas bottle isolation valves and reduced variacs (except one supplying turbo inlet) by 10% -> Will need access to continue this process every few hours or so throughout the day.  

[Sheila, Kiwamu, Terra, Jenne]

• ISS 2nd and 3rd loops simultaneously engaged. No dPdTheta instability tonight.
  • Bug in guardian code prevented this working last night:  wrong value sent to a switch to enable it.  Fixed.
• POP_A offsets tonight seemed to help POP sidebands as well as PRC gain, but still not so good for AS90.  Moving AS_C was delicate, resulted in lockloss.
• CoilDrivers state seems to ring up violin modes drastically.
  • Turned off violin mode damping before transitioning test mass L2 coils worked well.  Didn't turn damping back on, since the modes weren't high. 
  • Kiwamu noticed that the higher order violin mode notches for DARM / ETMs were off in frequency.  He has added notches to fix this.
• Switched to LVLN ETMY ESD. 
• Still need to figure out why CHARD Yaw engagement is so rough - it's better after re-doing the initial alignment setpoints, but it's not great.
• Took data for MICH and SRCL feedforward.
• Can't reshape MICH and SRCL loops as used to be done in Noise_Tunings.  Need to measure loops to understand what's up.

Next up:  Try dither loop to hold ETMX spot position in yaw to prevent spot position movement on BS.  Separate SOFT loops to X and Y, use offset of XSOFT yaw to hold ETMX spot position constant.  Alternatively (or simultaneously), dither BS and demodulate with several different signals at the same time, to understand better how the spot is moving.

We ended up moving PR2 (our uncontrolled recycling optic), and walking it with the POP_A offset.  This allowed us to get to a PRC gain of 31.6 by the O1 standard, but without the sideband powers tanking.  We think that this was promising, and will come back to it tomorrow.  Once we decide that we're happy, we should re-do the green initial alignment setpoints yet again (including the beatnote PDs, which weren't done earlier today) to set this as our reference alignment.  Attached is a big screenshot of where we were most happy.  Also attached is the PR2 offset screen, time-machined back to before we started moving PR2. 

TITLE: 09/01 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: Sep
SHIFT SUMMARY: Lots of commissioning, a PSL trip, and a small earthquake.
LOG:

• 01:59 PSL went out, Chiller error, flow sensor 1. Peter was awesome and drove out to the site to get it back on and going for more commissioning.
• 03:14 Dave restarting H1SUSPROCPI

Attached are trends of the various flow sensors in the PSL.  Comparison of the timestamps shows that the high power oscillator
tripped because of a flow rate problem with the AMP circuit.  The problem does not lie within the 4 laser heads, leaving either
the front end power amplifier, or the 4 crystal chambers.

    Unfortunately checking the 4 crystal chambers requires dismantling the laser.  The vortex flow sensor in the AMP circuit
is relatively new.  The other object in that circuit is the power amplifier module.  An inspection mirror might be able to
afford a view of the plumbing underneath the housing.

    One can see that the flow rate in the AMP circuit drops before the output of the front end laser drops, and that the output
power of the front end laser drops before the high power oscillator.  I think the sequence of events is the following:
 1. flow rate problem in the AMP circuit trips the flow watch dog of the front end laser
 2. the switching off of the front end laser breaks the injection locking of the high power oscillator
 3. the loss of injection locking results in a power drop in the high power oscillator which then trips the power watch dog
 4. the power watch dog switches the high power oscillator off

Unsuccessful at damping ITMX 15520 Hz PI several times tonight (previously seen here and here). We find that larger damping drive does not equal greater damping: When this mode was test driven and damped after the thermal transient at 50 W, a best gain and phase was found for damping. When the mode began to ring up later, increasing gain (by some large amound but still under saturation) or flipping sign and/or changing phase only resulted in faster ringup. This is true when still far below DAC saturation levels. It seems as if there is some gain sweet spot that must be found. 

--- --- ---

We had three occasions to damp ITMX 15520 Hz mode during the night. During the first, I successfully damped and it then rerang up (perhaps due to offset adjustments?) and lost lock. During the second and third I was not able to damp the mode and avoided lockloss only by decreasing power 50 W --> 25 W.

Below you see the mode first ring up and my gain trial and error response until I settle at 10000 and the mode is fully damped. Soon after, you see the mode ring up again right after the yaw offset step from ~ -0.02 --> -0.01. Note we started with a small negative gain so I just assumed we had actually been ringing up the mode the past few days.

During the second 50 W lock, damping was already running at the gain and phase settings that were effective at damping the first ring up above (+10000 gain, -60deg). Despite this, you see the mode slowly rising ~3 hours into the 50 W lock and my gain adjustments trying to damp. Note that here I start with some mostly successful positive gain (i.e. shallow slope) yet both raising the gain and flipping the gain sign cause the mode to ring up. I tried phase changes at this point too but existing was best. I avoided lockloss by decreasing power to 25 W, allowing mode to ring down enough to damp, then powering back up. I also rechecked my gain and phase at 50 W (post thermal transient time) and it would still drive and damp with a very steep slope. Still, an hourish later the mode began to ring up. I found similar behavior in the third attempt as the second and had to decrease power to avoid lockloss. 

Things to note: 

• After full lock aquisition, this mode takes ~ 3 hours to ring up. This morning, after dropping to 25 W then coming back to 50 W, it took only ~ 1.5 hours to ring up. Suggests something thermal.
• Q_m of ITMX ~= 18M
• Gap between ESD and ITM is larger than for ETMs ==> less actuation per drive
• Kiwamu has found similar higher gain =/= better damping in violin mode damping

Things to try:

• Retry damping immediately after decreasing power; should be able to see within ~1 min of lower power. 
• Try driving only one quadrant; test spatial overlap of ESD and mechanical mode
• Turn up ring heater (most likely ETMX RH) to decrease frequency overlap between ITMX mech mode and optical mode
• Check offset dependence

Jeff K, Chris Whittle

We used the interometer uptime to test the ITM charge measurement script by taking data for ITMX. With the previously used excitation frequency of 20.1 Hz being used for other measurements, we opted for an excitation at 11.5 Hz. At this frequency, we found an excitation amplitude of 1k counts to be sufficient for a good SNR. We have taken data at bias voltages between -10k and 10k counts, but have yet to perform post-processing.

We additionally tested whether flipping the analog voltage out switches to the quadrants had any effect on the locked interferometer. No effects were observed for flipping on/off quadrant voltages in various orders. This test was performed on just ITMX and with no actual output voltage to the quadrants.

Expecting PT120 to be 3 x 10-9 torr -> Don't see logged activity to explain -> am on road and not able to trend data etc.

The new prototype board was modified to satisfy some of the things in the requirement (T1600064) as listed in 1., 3. and 5. below. I also listed things that were not modified but would have been good if recommendations in T1600064 were implemented.

I wouldn't claim that we won't modify it further, but this is just the current state.

In this entry you'll be able to see the electronics modifications in the attached, but the photos are too small to read the original component values, so read the original drawing D1600298 as necessary.

1. Readback of the PD array signal after the servo loop switch ("SUM PE MON")  needed to be DC coupled.

The "whitening" was originally AC-coupled (second attachment) and therefore it was not compatible with digital AC coupling described in the requirement.

We bypassed the big caps in the input of the opamps with some resistors so it acts as one single pole at 2.7k with DC gain of 50 (second attachment, mod 1).

To make the OLTF measurement easier, we made the same change to the whitening downstream of the summation for the excitation DAC output.

The servo board output readback was also AC-coupled, but we want to see if everything is working fine including DC injected into the inner loop board.

We made the output whitening a true whitening (second attachment, mod 2). 0.35Hz zero might be too low, though.

2. The board doesn't implement the analog compensation for the in the inner loop filtering upstream of the error point.

The inner loop error point "whitening" works as a very steep boost seen from the outer loop, roughly an equivalent of (z, p) = ([3; 3; 130], [0.07, 0.07]). T1600064 reccomends to implement the same thing in the outer loop somewhere to cancel this effect to make the loop design simpler.

Since this is not compensated in the prototype board, the outer loop OLTF would become huge for f<3Hz. This means that the digital AC coupling servo should work REALLY hard to effectively AC-couple the entire outer loop at, say, f<1Hz, without reducing the outer loop gain at 10Hz and up.

We do not implement this by analog cut and paste job (no opeamp available for this on the board), but we'll try to make it work by an aggressive digital AC-coupling filter.

3. Integrator as a boost doesn't go well with digital AC coupling so it was converted to a usual boost with finite DC gain.

The error point of the digital AC coupling servo is kept zero at DC (i.e. a pole at zero Hz). The outer loop prototype had an integrator as a boost.

Of course they don't go well as any tiny DC difference between the AC coupling error point and the integrator input will be integrated and eventually the servo runs away. But we don't really need a pole at zero Hz.

We changed it such that the first boost acts as 50Hz LPF with DC gain of 10, and when added to a flat unity gain it becomes 500:50 boost (third attachment) (but see the next entry).

4. Boosts were implemented as three parallel integrators added to a flat unity gain.

We had a choice of unity gain, zp=50:0, 100:0 or 150:0. See the first and the third attachment.

As described in 3. above, we already changed one integrator to []:50 LPF so the boost becomes 500:50.

We didn't fix the parallel summation, it might be OK to go without any boost or using just one.

But T1600064 shows our standard practice of connecting boosts in series.

5. Outer loop servo filters on the board were also changed.

Originally the servo filtering was something like (z,p)=(100, [20^2;40]) (fourth attachment). As described in 2. above, 1st loop effectively adds ([3^2; 130], [0.07^2]). IMC adds another pole at ~8kHz.

Everything added together it's like ([3^2; 100; 130], [0.07^2; 20^2; 40; 8k]), and this might be OK.

But we made a change so the board became ([380;9k],[20^2;600]) by changing one of zero-Ohm resistors in the second stage and one C-R pair in the third stage  (fourth attachment). Everything added together it would be   ([3^2;130;380;9k], [0.07^2; 20^2;600; 8k]).

This change might not have been necessary. We just felt that letting a big 4.7uF capacitor and a zero-Ohm resistor to determine AC gain at around 10kHz in the first as well as the second stage was maybe too much of an uncertainty in the AC gain.

6. Sallen-Key at 0.1Hz for Digital AC coupling path might be too aggressive

For the moment the Sallen-Key is disabled for the digital AC coupling path (so the only analog filtering is a 10Hz LPF) to make things easier as we try to make it work.

Executive summary: 

* Good news - as expected, the 16-Hz comb due to the OMC length dither is gone (at least at this sensitivity level)
* Bad news - low-frequency 1-Hz combs remain, and some new low-frequency combs & lines have appeared 

Some details:

 I initially looked at 15 hours of 30-minute FScan DELTAL_EXTERNAL SFTs (30 SFTs) generated during ER9 and was aghast at how bad the low-frequency spectrum looked, with a pervasive 0.485308-Hz comb ranging up to its 446th harmonic at 216.4 Hz, but when I exclude the three hours of SFTs when the 2-second ALS- glitches were present, things don't look quite so bad (see figure 1 for a sample without removal and figure 4 with removal). I dewhiten according to the new 6-pole / 6-zero, 0.3 / 30 Hz algorithm.
 The infamous 16-Hz comb due to the OMC length dither tracked down and killed here is gone (at least at this sensitivity level and these SFT statistics). As a result the high-frequency band (up to 2 kHz) is remarkably smooth with only violin modes and sporadic isolated artifacts.
 The 1-Hz comb with 0.5-Hz offset remains pervasive, but other prior 1-Hz or near-1-Hz combs with different offsets (e.g., 0.25 Hz) are not strong.
 On the other hand, there is a new near-1-Hz comb (0.996798-Hz spacing) visible to its 204th harmonic at ~203.3 Hz.
 There is also a new near-2-Hz comb (1.999951-Hz spacing) visible on approximate odd-integer-Hz frequencies, starting from ~9 Hz and visible up to ~175 Hz. This is likely the same 2-Hz comb reported in May by Bryn Pearlstone (which Ansel Neunzert kindly reminded me about today). 
 There is a "new" 56.8406-Hz comb visible to its 11th harmonic at ~625.25 Hz, which in hindsight I can see was buried in the O1 spectrum (I overlooked the pattern and indicated the teeth as isolated lines). This time the pattern was strong enough to jump out at me and to jog my memory that this comb was seen in  H2 1-arm data in 2012 in both the arm feedback channel and a quiet sensor-noise-dominated OSEM channel. This seemed to indicate a DAQ system problem at the time. I can see from O1 NoEMi line lists that this comb is pervasive in ISI, SUS and PEM channels at the corner station and both end stations.
 The old "K" comb-on-comb (0.088425-Hz fine comb attached to teeth of a coarse 76.3235-Hz comb) has more 11 more fine teeth visible on the lowest-frequency coarse-tooth comb.
 The old calibration line at 35.9 Hz has been moved to 35.3 Hz. The new excitation lines at 33.7 and 34.7 Hz are easily visible, but not as strong as the primary calibration lines. I found these changes documented here.
 There are sporadic new isolated lines here and there (indicated in line list - see below)
 There is a "crab-killer" broad bump centered at about 58.6 Hz which degrades sensitivity in the Crab Pulsar band of interest (~59.3 Hz). On the other hand, the whole noise floor is elevated w.r.t. O1, anyway. So it may be premature to worry about the bump.


Figure 1 - spectrum for 50-100 Hz when the 3-hour 2-second-glitches stretch is included (~0.5 Hz lines marked with 'h' for 'half')
Figure 2 - 0-2000 Hz (removing 3-hour bad stretch from here on)
Figure 3 - 20-50 Hz sub-band 
Figure 4 - 50-100 Hz sub-band 
Figure 5 - 100-200 Hz sub-band
Figure 6 - 1300-1400 Hz sub-band (illustration of how clean the high-frequency band is)

Also attached are a larger set of zipped sub-band spectra and a lines list (excluding the bad 3-hour stretch). Note that because the statistics here are two orders of magnitude smaller than used for the full O1 run report, I am not yet removing lines seen then that may yet re-emerge with more accumulated post-O1 data. So many of the line labels in these figures are buried in the noise fuzz for now.

Line label codes in figures:

b - Bounce mode (quad suspension)
r - Roll mode (quad suspension)
Q - Quad violin mode and harmonics
B - Beam splitter violin mode and harmonics
C - Calibration lines
M - Power mains (60 HZ)
O - 1-Hz comb (0.5-Hz offset)
o - weaker 1-Hz and near-1-Hz combs (various offsets, including zero)
H - 99.9989-Hz comb
J - 31.4127 and 31.4149-Hz combs
K - 0.088425-Hz comb-on-comb
t - 1.999951-Hz, 2.07412-Hz and 2.07423-Hz combs
D - 56.8406-Hz comb
x - single line (not all singlets in the vicinity of quad violin modes are marked, given the known upconversion)

Since there seem to be some confusions about which PD has what kind of analog filtering and sent to which channel, here it is.

I'm quite certain about HPO output before AOM (which is "Power monitor PD" in D1002929) and ISS 1st loop monitors, but not that sure about "monitor PD"s in D1002164. E-travellers are incomplete (they don't say which one is installed where), so I'm just listing the nominal values for these "monitor PD"s.

the Verbal Alarms code was logging to the ops home directory. Prior to the move of this home directory (WP5658) I have modified the code to log to a new directory:

/ligo/logs/VerbalAlarms

We restarted the program at 14:04 and verified the log files are logging correctly.