On friday night, the wind was high(~25mph), and the arm was hard to lock. Although the angular motion of the test masses was not particularly different than on a quiet night. First attachement shows the oplev spectra from friday night, when the wind was ~25mph. second attachement shows spectra from yesterday night with a 5mph wind.
To note :
We tracked down the problems we had in the past windy nights to be related with the ALS tidal tuning , but it is still good to make these plots with the OpLEv signals.
However, I think your plots didn't turn out the way you wanted to. There is only YAW in the second attachment.
Also, it would be easier to make a comparison if you put windy/quiet curves for each DOF in the same plot. Thanks!
thanks for noticing!
Attached are some other plots :
1. wind at corner and endy stations between may 17th/may 20th. High winds references are marked in orange, low winds is in green
2. Spectra comparison of ground sensors (HEPI STS) at end and corner stations with different winds. Blue and pink curves are during high winds (blue = may 17th, pink = may 18th). Red is during low winds (may 19th). From top to bottom rows : ETMX ETMY and ITMY. From left to right : X Y Z degrees of freedom.
3. Spectra comparison of oplevs during high winds (may 18th blue curve) and low winds (may 19th red curve) for ETMx ETMy and ITMy top to bottom. Left column is pitch, right column is yaw.
(Stefan, Lisa, Chris)
Looking through the locked stretches from the weekend, we noticed that the ALS slow feedback -- intended to relieve the VCO control signal of low frequency seismic and tidal motion -- was failing in its mission. What we saw is plotted in the first attachment. The green trace shows the VCO control signal, which should be forced toward zero when the slow test-mass feedback (red trace) is enabled about halfway through the timeseries. Obviously this wasn't happening. Meanwhile the VCO frequency servo that actuates the tune slider (blue trace) was running away along with the VCO control signal. When the tune slider hits its limit at +/-5 the lock is broken.
We tracked it down to a missing integrator in the ALS-X_ARM filter bank, without which the slow loop didn't have enough oomph at DC to null the fast control signal. I think this filter was removed intentionally (but without realizing its true purpose) during some earlier automation work. We put it back and, taking advantage of the improved test mass plant inversion, we were able to increase the gain upstream (in the ALS-X_REFL_SLOW filter bank) from 1 to 30. This should help prevent the arm from being blown away by the wind. When we lose lock the integrator is switched off by the Beckhoff, and the drive bleeds away through a 0.01Hz pole. The improved behavior is shown in the second attachment (where the gain was cranked about 20 minutes in).
Here is the list of commissioning task for the next 7-14 days:
Green team (XY-arm):
Blue team (X/Y-arm):
Red team:
SEI/SUS team:
In-Vacuum preperation:
As of 12:00 PSL Status: SysStat: Green, except VB Output power: 26.7w FRONTEND WATCH: Good HPO WATCH: Red PMC: Locked: 12min Reflected power: 1.5w Power Transmitted: 9.8w Total Power: 11.3w FSS: Locked: 12min Trans PD: 0.94v ISS: Diffracted power: 3.833% Last saturation event: 12m
Jason & Betsy – SRM fine alignment at HAM5 Gerardo – Putting the OFI into storage box and fly over beam tube Hugh – ISI balancing at HAM4 Travis – ACB balance and alignment Richard & Filiberto – Chassis work in EE shop & cabling at HAM6 Peter & Olli – In H2-PSL working on ISS assembly Dave – TCS Hartman table alignment
The problems we noted over the weekend with the ETM SUS (and sometimes SEI) systems tripping on lock loss (alog 11957) can maybe most accurately be characterized as a watchdog problem. The trips are caused by large drive transients after lock loss. It's hard to imagine that there's much we can do to prevent this from happening. Even if the DARM (CARM, etc.) control signals could be shut off immediately, the residual impulses would still produce large impulse response in the LOCK filters in the suspensions. We could try to shut off the drive signals in the SUS controllers, or hold the outputs at their current value, but that's a bit more difficult to implement.
In general, though, the watchdogs should probably just not be tripping on transients. If the watchdogs were a bit smarter and only tripped on sustained saturations or oscillations, this would likely not be an issue. I vote that we solve the problem in the watchdogs, but increasing the amount of time it takes before they trip.
I closely looked at one of saturday's trip on ETMX @ 04:44:40 UTC (May 18th 2014). The sequence was :
I will post some data tomorrow
Just for a bookkeeping purpose:
I recentered the beam on the IMC WFSs on this past Saturday. I used the picomotors. Both of them had been off by 0.2-ish counts in the normalized pitch and yaw.
No restarts reported.
I made a couple of improvements to the IMC guardian:
The final point was to make it so that there is no activity in the LOCKED state, so that reaching the LOCKED state means that the IMC is fully up. In general, I think we should start making all requestable states be "idle" states in that they don't do any action other than monitoring for exit conditions. This idea here is to make them true markers of a steady state of the system.
We end up with just two requestable states now: DOWN and LOCKED. The DOWN state doesn't actually prevent the IMC from locking briefly, so we may want to change the DOWN state to something that actually prevents the IMC from locking.
Next, I want to make the IMC guardian the manager of the SUS_MC{1,2,3}, so IMC is actually in control of setting the MC suspensions to their ALIGNED states. This will also allow us to achive the point above, since we can then make the DOWN state into something where the SUS are misaligned, thereby preventing the IMC from locking.
model restarts logged for Fri 16/May/2014
2014_05_16 23:12 h1fw1
No restarts reported for Saturday.
One unexpected restart of h1fw1 over the two days.
Kiwamu, Jamie, Lisa
We figured out why we couldn't reduce the CARM offset below 1kHz while keeping the PRMI locked on REFL 45 I&Q.
The REFL 45 demod phase was tuned without the arms present; it turns out that with the arms, even with a large CARM offset (1 kHz), the demod phase changes enough to make MICH very close to instability.
So, we just tuned the REFL 45 phase by monitored the MICH OL TF and the REFL 45 I/Q decoupling by looking at a PRCL line around 50 Hz.
Not a very advanced technique, but we could recover pretty easily a reasonable shape for the MICH loop and we could reduce the CARM offset.
Kiwamu will post some plots with the MICH OL TF; here are some numbers for the REFL 45 phase:
Without the arms: 144 dg
1 kHz --- 900 Hz CARM offset: 172 dg
800 Hz --- 700 Hz CARM offset: 180 dg
We were happy and ready to do the transition to 3f, but another seismic event like the previous one made the lock of the Y arm more difficult.
We are still debating if we should go home or not...
Fig.1 MICH open loop transfer functions. The pink curve is the one after we adjusted the demod phase at a CARM offset of 1 kHz. Everything else is the ones before the adjustment. You can see that the funny bump between 20 and 50 Hz dissappered as we adjusted the demod phase.
Fig.2 PRCL open loops. The red curve is the one after the demod phase adjustment. The blue one is before the adjustment. Not a significant difference.
Fig.3 Various error signals for the length control. The references were taken when the CARM offset was at 1 kHz and the live traces are the ones with the offset at 800 Hz. REFL45I sees more noise above 40 Hz which is seemlingly from the common mode. A peak at 50 Hz is our injection in PRCL for calibrating the responses.
[Jamie, Kiwamu, Lisa]
The ETM SUS are getting kicked to the point of watchdog trips when the we lose the ALS DARM lock. The DOWN state of the ALS_DIFF guardian was shutting off the INPUT to the LSC-DARM filter bank, but not the OUTPUT, so we were suspecting that leftover filter stuff was being driven out to the suspensions. There are also no LSC triggers associated with the ALS error signals that could shut off the DARM output after lock loss.
I modified the ALS_DIFF DOWN state to turn off the DARM output as well, in the hopes that that would reduce the problem. We thought that that did help, but then we just got another lock loss that did trip the ETMY SUS and ISI ST2 watchdogs, so it didn't elliminate the problem entirely.
I wonder if we'll need to add some LSC triggering based on the ALS inputs, for when we're using ALS signals to feed back to the SUS.
While I was at it, I did a bunch of code cleanup in the ALS_DIFF module. Mostly just to keep things consistent with the guardian style guide (unpublished). I also committed the ALS_DIFF.py module to the USERAPPS svn, and added it to the GUARD_OVERVIEW medm screen.
Kiwamu, Jamie, Lisa
Today we had the illusion that we could start locking before dinner time, but that didn't happen.
While both the X and Y arms were locked on green, something happened in the corner station (visible in the PEM CS SEIS 30-100 mHz channels, especially Z) which affected only the Y arm, while the X arm remained happily locked.
I added a very-low-frequency-boost in the BS oplev damping loop which reduces the motion between 10 mHz to 100 mHz.
When seismic was high as Lisa reported, BS also moved a lot in yaw by 1.5 urad in pk-pk. This was causing a power modulation in the ALS diff beatnote. So I added this filter to reduce the motion at very low frequencies. Also, I increased the gain for this loop from -0.03 to -0.05 as there was a servo instability at 1-ish Hz with the VLF boost engaged. When seismic is at the normal level, do not forget to disengage this VLF boost.
Lisa, ChrisW, Jamie, Arnaud, Kiwamu
We locked the PRMI with the 1f signals at a CARM offset of 1-2 kHz. The goal is to hand it over to the 3f after the CARM offset passes approximately 1 kHz at which the arm cavities resonate with the 2f sidebands. However, we didn't get a real progress tonight. We will continue studying this configuration.
What we did:
"At some point, we noticed that the cavity stay locked without the slow feedback. So we stopped using the slow feedback."
There was high wind last night until around 2 am. Kiwamu's locking successes started when the wind stopped blowing. According to Robert's FOM, the wind was indeed pretty high, around 90% percentile..but still, I claim that we should be able to lock a cavity anyway.
We will try to produce meaningful plots, in the meantime here is the plot which has the times with high wind, as looking at the PEM EX SEIS BLRMS 1-3Hz.
The coupling from the BSC-ISI Stage 1 Z drive to the T240 RZ signal reported a few weeks ago has been lately our first guess as for what's limiting the RZ (Yaw) isolation performance.
This week we modified our real time model to do subtraction tests. The modification has been installed on ITMY.
The first plot attached shows:
- the coupling measurement in green
- a simple fit to be used in the subtraction block, in black
- the expected residual coupling in red
The second plot shows the residual coupling measured after installing the correction. The subtraction results look pretty good. Unfortunately, I did not observe substantial improvement in the optical lever Yaw motion. To be further investigated...
Came in this morning to find the BS ISI stage 2 GS13 watchdog tripped. Unclear why.
The strange thing is that the trip appears to have occured at GPS 1084253752, which is 10:30pm last night, whereas commissioning finished up around 4:30am. Did they just run with the BS tripped all night?
The watchdog was reset and guardian recovered everything fine.
The ISI-BS is a little peculiar: we know that the Michelson feedback could send a kick to the ISI and trip ST2 watchdogs. It's very often that the commissioners work with ST2 tripped, and I wouldn't be surprised that's what happened last night.
Given that, you should double check with the crew to see if it was really the case.
This was indeed the case. The michelson lock was kicking the BS and tripping stage 2, so they just ran with it tripped.
This is obviously an untennable solution, so we need to figure out how to prevent the BS ISI from tripping during acquisition.