It might be that we've been following the temperature drift in the LVEA. There has been a big downward temperature spike, then the FMCS responded, and totally totally overshot.

In the attached, colored vertical lines indicate Suresh's laser change, Hugh's HAM2 change and commissioners' changing input alignment. From the HAM2 oplev, Hugh's work looks like a minimal impact.

But the time the commissioners responded to whatever change by turning the mirrors corresponds to the temperature change.

according to Daniel, we could have recovered partially from this last night by going into the system manager, right clicking on the bad chassis and selecting disable.  Then a red X would appear over that chassis.  If we redo the generate mappings, check configuration, activate confguration, the system should run again. 

Attached is a screenshot of the TwinCAT System Manager. The context menu will bring up the option to disable an entire EK1100 chain. Once disabled, a red X should be visible in the overview. After disabling the offending chassis or module, one has to generate the mapping, check the configuration and activate the configuration (see second screenshot). Make sure you are in run-time mode.

Caveat: The chassis are daisy chained. Turning off a chassis will no only disable the chassis itself, but all units which are hanging off this chassis. Using an Ethernet coupler it is possible to bypass a single chassis.

Nice to see this working again. I'm confused by your sentence: We have about a factor of 7 headroom in the ESD before we would saturate, and about a factor of 30 head room in the ESD. Do you mean the UIM stage for one of these? Also, in the future 'ESD coil' should be 'ESD electrode'.

Sorry Peter, I meant to say: We have about a factor of 7 headroom in the ESD before we would saturate, and about a factor of 30 head room in the UIM. I have also attached the coil spectrum now.

J. Kissel

Sheila has tasked me into figuring out what's wrong with ETMY. Here're some additional notes that I've gathered from talking to Alexa / Sheila / Evan/ Nic that were not included in the (already amazingly and delightfully detailed) entry above.

- I found EX & EY Yaw damping loops *without* the +12 [dB] gain-only filter on. Sheila / Alexa didn't know when / why the filter was turned OFF; a dataviewer trend reveals that it was just never turned ON after install on Nov 11th. Since this bug is common to both ETMs, I don't expect it was the problem. I've now turned them ON so we have as little difference between the damping loop design as possible, so we have less things to blame.
- EX (ONLY) has a new "NicLP" low-pass filter engaged in FM2 of the L3_LOCK_L control. From what I gather, this elliptic low-pass filter was installed *after* they'd stopped using EY, so I also don't suspect this difference.
- EY (ONLY) has optical lever damping in PITCH engaged. This also was turned on right as the switch to only using EX only, when it was determined that there was a problem with EY. It's also the only QUAD damping loop that has a sufficient amount of documentation associated with it (see LHO aLOG 14878) that we trust it to be functional (thanks Evan)!

I'll now begin measuring stuff, in hopes to find problems...

another example

Still there, still a problem....

I made a few plots today. The spectrum of the CPS right before and right after the trip, and the zoom-in time series plot around the time of the trip. I notice that the time that the CPS signal goes down does not match the GPS time indicated the trip exactly (off by a few hundred milliseconds).

While making the spectrum I also notice the peak near 4Hz in the "beforetrips" spectrum plot. So I went and look at the LHO summary page and found that there's a small gaussian-looking bump around 4Hz that seems to appear everyday. Probably not very important but just wanted to point that out.

Also, please note that V2, H2, and H1 signals are almost perfectly overlapped (that's why you only see two colors). 

I'll start looking into other related channels that *should* have seen the signals. Although I believe this might be an electronics issue.

SR3 details:

Peanuts-shaped SR3 baffle was used as a reference.

The IFO beam is supposed to be in the right hole, the angle between SR3-SR2 beam and SR2-SRM is 29 mrad, the distance from the baffle to SR2 is about 460mm (just by eyeballing the drawing), so the beam separation on the baffle is about 17.5mm.

This means that the SR3-SR2 beam should be about 17.5/2 mm to the left from the center of the right baffle hole.

PR3 YAW offset when the beam just hits the right edge of the baffle was -1394.4, while the offset when the beam was on the center line of the right baffle hole was -194.4.

When looked from HAM5, the right baffle hole looks like an ellipse with the major axis vertical, and the minor radius is 67.5mm.

Therefore the PR3 YAW offset should be

SR3 Y = -194.4  + (-194.4+1394.4) * 17.5mm/2 / 67.5mm = -38.8

PR3 P offset when the beam hits the top/bottom edge of the tallest part of the hole was 2350.3/-1041.7, and the average is 654.3.

This also means that the SR3 YAW slider calibration is off.

For YAW, 1200 urad of the slider offset produced about 67.5mm.

2*angle*16m = 67.5mm -> angle = 2.1mrad.

Therefore, in reality, the SR3 slider calibration for Y is 2.1mrad/1.2mrad = 1.76. (But I'd claim that the measurement error is probably as large as 30% or so).

For SR3 PIT,  it's good.

The beam moved from the bottom to the top with -3392 urad slider step. The hole height is 114mm.

2*angle*16m=114mm -> angle = 3.6mrad.

The SR3 slider calibration for P is 3600/3392 = 1.06

SR2 details:

Before SR3 bias was set, AS_C was centered using pico.

After SR3 was done, AS_C was centered using SR2 bias. After this, SR2 bias was [1399.7, 1485.0]. This is our "initial position".

Then SR2 bias was scanned first in Y, then in P, and then Y again.

Attached left is the Y scan and the right is the P scan. Blue vertical lines indicate where I started.

As I was sort of suspecting, PIT was more off than YAW, and anyway I settled on [P,Y]=[1339.7, 1575].

One caveat is that the final position corresponds to one data point where I see a jump in the AS_C SUM (look at the green circle that goes to 1.04). This is repeatable, and I think this is where some stray beam or maybe the reflection from the AS_C goes into the AS_C. I don't think this corresponds to smaller loss. This position was chosen as the final position by eyeballing the peak of the green plot excluding that abnormal data point.

I checked on the BRS this morning, and found the code frozen. I followed Krishna's fairly straightforward document up to the point of opening the box and found the masses 90 degrees out of position (NW/SE instead of SW/NE), I was unsure of what to do, so I closed up, restarted the code and called Krishna to consult.

Afterwards, I went back to EX this afternoon, and re-aligned the masses on the damper. I did this by first modifying the code as described in Krishna's document, then opening the large insulating panel on the west side of the BRS enclosure, and just turned the turn table until the masses were approximately SW/NE. The turn table moved quite easily, I was expecting more resistance.It was dark down there, so it was hard to get a good picture, but I've attached an image taken from the west side of the BRS enclosure, showing how I left the masses. I then went back to the rack, re-commenting the appropriate code and restarting. Haven't had a chance to check on the BRS since then, but it looked okay from what I could see on the laptop readouts.

Looking back at the data, it seems that the changes made in the morning, resulted in a positive feedabck loop leading to large amplitude oscillations in the BRS. The changes made in the afternoon reversed it but it took ~5-6 hours to damp. However, the software appears to have crashed again later. Tomorrow, I'll investigate for intermittent shorts or other grounding issues.