Since both end stations are closed up, I increased the alarm levels for both of these monitors to 600cts (MINOR) & 800cts (MAJOR) for 0.3 & 0.5um (they had been set at 100 & 200 cts, respectively).
J. Kissel, K. Kawabe, R. McCarthy, A. Pele At ~11:30a PDT (~18:30 UTC) I began trying to drive the H1 SUS ETMX ESD in Pitch, to replicate a driven, optical lever ASD (e.g. LLO aLOG 12512). To my dismay, I found no coherence. We performed several other excitation tests, including repeating Keita's measurement from yesterday (LHO aLOG 11872), also to no avail. Finally, after obtaining Richard who was valiantly battling the ESD at EY, he told us to check out the monitor read-backs, H1:IOP-SUS_EX_MADC1_EPICS_CH0 - 5 which can be found from the A1 (the second ADC ) screen off of the H1SUSEX IOP GDS_TP screen. These channels showed values frozen at ~2050 [ct], which (though spurious) would correspond to a voltage of 2050 [ct_ADC] * 40/2^16 [V_MON/ct_ADC] * 40 [V_ESD/V_MON] = 50 [V_ESD] Richard immediately recognized these values as indicative of a failure of the driver. I started my drive at 18:32 UTC (11:32 PDT), and the bias channel falls from ~30000 [ct] (ADC), to 2050 [ct] at 18:20 UTC (11:20 PDT). There was activity in the XVEA (between 9:30a PDT - 11:30a PDT), around the racks that the ESD driver is installed, but it was with seemingly unrelated network cabling. So it appears as though the failure was a result of some closeout activity. However, when Richard and I drove down to investigate,we found that even the driver's +/-18V LEDs were off. After a quick investigation, including a full high-voltage power cycle, with limited tools, Richard concluded that the ESD Driver needs to be swapped with its spare, and is doing so now with Fil. Welp -- we got *one* promising night of ALS DIFF locking under 1 [nm] rms...
We looked at a spectrum of the Optical levers while we were locking DIFF with low bandwidth. The ETMY pitch was 2 orders of magnitude above all the others, so we switched this back to the legacy decoupling fitlers, and the excess noise went away. The dashed lines in the attached screenshot are with the new filters, solid lines with the legacy filters for ETMY UIML2P only.
Some other things that we noticed tonight:
on the ops overview screen the ETMY suspension watchdog can be green even when ETMY is tripped.
We also have had several times that the mode cleaner locks on a wrong mode, where the transmitted counts are low. In this case we need to unlock it manually, maybe we need to check some thresholds in the guardian.
While testing out new SEI/ISI/HPI guardian code on ITMY this morning (to be logged separately later), Fabrice and I encountered what may be a bug in the watchdog code associated with the T240s in stage 1.
We had applied a large pitch offset on ITMY HPI, and were attempting to restore the system to full isolation. The large pitch offset rang up the T240, causing it to saturate (expected). The new T240 monitor (H1:ISI-ITMY_ST1_T240_MONITOR_OUT) kicked in, and guardian waited for the T240 monitor to clear before it attempted to start isolating stage 1. As soon as the T240 monitor cleared, guardian started to enage the isolation loops and ramp up the gains:
20140514_17:27:05.582 ISI_ITMY_ST1 [WAIT_FOR_T240_SETTLE] 20140514_17:37:28.138 ISI_ITMY_ST1 transitioning state: WAIT_FOR_T240_SETTLE->ENGAGE_ISO_FILTERS_HIGH_RX_RY 20140514_17:37:28.139 ISI_ITMY_ST1 calculating path: ENGAGE_ISO_FILTERS_HIGH_RX_RY->HIGH_ISOLATED 20140514_17:37:28.139 ISI_ITMY_ST1 new target: RAMP_ISO_FILTERS_UP_HIGH_RX_RY 20140514_17:37:28.139 ISI_ITMY_ST1 executing state: ENGAGE_ISO_FILTERS_HIGH_RX_RY 20140514_17:37:28.139 ISI_ITMY_ST1 [ENGAGE_ISO_FILTERS_HIGH_RX_RY] 20140514_17:37:28.204 ISI_ITMY_ST1 [ENGAGE_ISO_FILTERS_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RX_SW1S => 21508 20140514_17:37:28.330 ISI_ITMY_ST1 [ENGAGE_ISO_FILTERS_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RX_SW2S => 1537 20140514_17:37:28.455 ISI_ITMY_ST1 [ENGAGE_ISO_FILTERS_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RX => ONLY ON: INPUT, OUTPUT, DECIMATE, FM4, FM5, FM6, FM7 20140514_17:37:28.456 ISI_ITMY_ST1 [ENGAGE_ISO_FILTERS_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RY_SW1S => 21508 20140514_17:37:28.582 ISI_ITMY_ST1 [ENGAGE_ISO_FILTERS_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RY_SW2S => 1537 20140514_17:37:28.708 ISI_ITMY_ST1 [ENGAGE_ISO_FILTERS_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RY => ONLY ON: INPUT, OUTPUT, DECIMATE, FM4, FM5, FM6, FM7 20140514_17:37:28.889 ISI_ITMY_ST1 transitioning state: ENGAGE_ISO_FILTERS_HIGH_RX_RY->RAMP_ISO_FILTERS_UP_HIGH_RX_RY 20140514_17:37:28.890 ISI_ITMY_ST1 calculating path: RAMP_ISO_FILTERS_UP_HIGH_RX_RY->HIGH_ISOLATED 20140514_17:37:28.890 ISI_ITMY_ST1 new target: ENGAGE_ISO_BOOST_HIGH_RX_RY 20140514_17:37:28.890 ISI_ITMY_ST1 executing state: RAMP_ISO_FILTERS_UP_HIGH_RX_RY 20140514_17:37:28.891 ISI_ITMY_ST1 [RAMP_ISO_FILTERS_UP_HIGH_RX_RY] 20140514_17:37:28.954 ISI_ITMY_ST1 [RAMP_ISO_FILTERS_UP_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RX_TRAMP => 10.000 20140514_17:37:28.954 ISI_ITMY_ST1 [RAMP_ISO_FILTERS_UP_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RX_GAIN => 0.010 20140514_17:37:28.955 ISI_ITMY_ST1 [RAMP_ISO_FILTERS_UP_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RY_TRAMP => 10.000 20140514_17:37:28.955 ISI_ITMY_ST1 [RAMP_ISO_FILTERS_UP_HIGH_RX_RY.run] ezca: H1:ISI-ITMY_ST1_ISO_RY_GAIN => 0.010 20140514_17:37:28.956 ISI_ITMY_ST1 [RAMP_ISO_FILTERS_UP_HIGH_RX_RY.run] timer['isolating'] = 10.0 20140514_17:37:29.077 ISI_ITMY_ST1 state returned jump target: WATCHDOG_TRIPPED_DEISOLATING 20140514_17:37:29.140 ISI_ITMY_ST1 transitioning state: RAMP_ISO_FILTERS_UP_HIGH_RX_RY->WATCHDOG_TRIPPED_DEISOLATING 20140514_17:37:29.140 ISI_ITMY_ST1 calculating path: WATCHDOG_TRIPPED_DEISOLATING->HIGH_ISOLATED 20140514_17:37:29.140 ISI_ITMY_ST1 executing state: WATCHDOG_TRIPPED_DEISOLATING 20140514_17:37:29.140 ISI_ITMY_ST1 [WATCHDOG_TRIPPED_DEISOLATING] 20140514_17:37:29.203 ISI_ITMY_ST1 [WATCHDOG_TRIPPED_DEISOLATING.run] USERMSG: WATCHDOG TRIP: DEISOLATING (2)
Immediately after the gains turned on, the T240 watchdog tripped. However, this appeared to not be associated with any actual activity in the T240 (see attached plot of T240 trip which shows nothing).
We suspect that this must be due to a bug in the watchdog code. A recent change activates the T240 watchdog based on the state of the stage 1 isolation loops: no T240 watchdog when the isolation loops are disengaged, and yes T240 watchdog when the loops are engaged with non-zero gain. It must be something to do with this logic. Fabrice and I are investigating.
The problem comes from the saturation counter that is not reseted:
- the T240 saturation flag turns on when we apply the HEPI offsets
- the saturation counters reach the treshold value (10 saturations)
- but the watchdog doesn't trips because the T240 are not in loop and therefore ignored by the top layer of the WD code. That's a feature we recently included so that we can turn on hepi without triping the ISI.
- as a consequence the saturation counters never gets reset
- so the WD trip when the T240 isolation is turned on, because the saturation counter flag is still on
Jamie and I have a plan to fix this. We need to modify the WD code and the simulink model. Dave says we can add those chances to work permit #4626.
I was told before that only one of the QPDs on TMSX see the IR transmission. I wanted to see how bad the problem is, so I looked at the the QPD signals for X and Y when night people were having night locking fun.
I'm disappointed to see that both of the X QPDs see nothing (attached, ch3 and ch6). QPD matrix elements and gains are correct, it's just that there's no photon. How come?
Y looks OK.
I saw an entry in the LLO log where they were able to flash the QPDs through a viewport during a time when they suspected they had a non-functional transmon QPD. If you begin to suspect the actual QPD functionality, perhaps you can try that. Otherwise, you can always give me a call and we can go over a functional check to be done chamber-side.
too far out of alignment.
Daniel told me that, after people played with X IR QPDs and saw some light, he moved picomotor of M4 (that is the harmonic separator the reflects IR but transmits green) because there was no IR on ISCTEX any more. After Daniel's treatment no IR light was seen on QPDs.
That's a long time ago.
No, Rich, I'm not suspicious about electronics.
model restarts logged for Tue 13/May/2014
2014_05_13 06:01 h1fw0
2014_05_13 10:48 h1hpietmx
2014_05_13 10:48 h1iopseiex
2014_05_13 10:50 h1isietmx
2014_05_13 10:51 h1iopsusex
2014_05_13 10:53 h1susetmx
2014_05_13 10:53 h1sustmsx
2014_05_13 11:21 h1susetmx
2014_05_13 11:24 h1susetmx
2014_05_13 11:49 h1broadcast0
2014_05_13 11:49 h1dc0
2014_05_13 11:49 h1fw0
2014_05_13 11:49 h1fw1
2014_05_13 11:49 h1nds0
2014_05_13 11:49 h1nds1
2014_05_13 13:41 h1susetmx
2014_05_13 13:51 h1susetmy
2014_05_13 13:52 h1susetmy
2014_05_13 13:57 h1iopsusey
2014_05_13 13:59 h1iopseiey
2014_05_13 13:59 h1susetmy
2014_05_13 13:59 h1sustmsy
2014_05_13 14:01 h1hpietmy
2014_05_13 14:01 h1isietmy
2014_05_13 14:06 h1broadcast0
2014_05_13 14:06 h1dc0
2014_05_13 14:06 h1fw0
2014_05_13 14:06 h1fw1
2014_05_13 14:06 h1nds0
2014_05_13 14:06 h1nds1
2014_05_13 14:09 h1asc
Tuesday Maintenance Day. Early unexpected h1fw0 restart. Many model changes with two DAQ restarts to support new configuration.
launched a TF measurement on ETMY for Arnaud at around 6:28 local.
Measurement has completed
Stefan, Kiwamu, Sheila
We have locked ALS DIFF with a bandwidth around 10Hz. The first 3 attachments are some screen shots of the configuration we had tonight when locking ALS COMM and DIFF. The ALS DIFF guardian enforces some of these settings, it doesn't control the drivealign matrices or the ESD offsets.
To get this we moved the UIM/test mass blend down to take care of the 10Hz UIM saturations, added an additional high pass to the UIM to reduce the low frequency drive, then pushed our UGF to around 10Hz. We added three low frequency boosts to DARM, after that we were approaching the VCO noise that Stefan measured earlier using a IFR in place of the DIFF PD. We added the FDD back in, which improved the noise at high frequencies. Spectra of the suspension drives are attached.
We then locked ALS COMM on the side of the fringe, and adjusted the DARM offset to bring the IR roughly half way down the fringe on the Yarm. The attached screen shot has both the ALS DIFF PLL control signal calibrated in Hz and the TRY PD roughly calibrated in Hz, with no frequency dependence (so this is not to be trusted above a few 10s of Hz). The strip tool shows the time series of this.
Then we locked both arms on IR using just ALS. The attached screenshot shows them locked and both roughly on resonance, we moved the COMM VCO set frequency by 500Hz to bring both arms off resonance, brought it back on resonance and off again.
Next we locked PRMI on the 1f signals, and adjusted the BS alignment which resulted in better IR alignment in the Y arm. PRMI power build up was 80uW in POP18 at the highest, which is pretty good.
After this we went to ISCT1 and realigned the diff beat note, we see 18.5mV rms our of the BBPD now. The attached screenshot shows the GigE cameras and the IFO align screen with this alignment.
We then locked PRMI on 3f. To do this we disconnected the power from the BBPD for Alexa's POP beat note measurement, and took the RF cable from the table to the rack back for 3f. At the rack side Kiwamu took the cable routed to POP9 back to the 3f diplexer.
We succeeded several times in locking both arms using ALS, moving them off resonance, locking PRMI on 1f. We could transition to 3f signals, but PRMI was not very stable.
The BS ISI has been tripped most of the night.
Noise plot:
black = free run (taken last week)
red = in-loop error signal
pink = VCO noise with two dividers in.
I think that the ISI trip happened before the suspension trip, this was while we were locking diff. I also thought that the T240s were green before I untripped the watchdog, but maybe they weren't because the ISI tripped again as it was isolating.
I stongly suspect that the second trip is due to the bug Fabrice and I noticed in 11888
J. Kissel for S. Ballmer Stefan noticed that the H1 SUS ETMY UIM (L1) UR and LL Coil Driver Current Monitors (specifically the FAST_IMON) H1:SUS-ETMY_L1_FASTIMON_LL_MON H1:SUS-ETMY_L1_FASTIMON_UR_MON are unresponsive to a DC offset. We should fix this.
For info : the only test that have been carried out yet (with Phil) to understand those non responsive channels was to measure the voltage between the monitor output and the anti aliasing chassis. For the 4 channels we saw a similar voltage readback increase when sending thousands counts, meaning the problem could come from anywhere from anti alias to model wiring. More details on the pb : here
I don't understand this at all, but I noticed that changing the polarity of the bias for DC channel as well as the offset of each quadrants changes the response of ESD measured by OPLEV.
I first set DC bias to 125000 counts, set the offsets of all quadrants to 51776, injected into each quadrants, one by one at 3.2Hz, and measured the transfer coefficients from the excitation to the OPLEV. No filter in the injection path, so this is the TF from ESD digital output to the oplev. Injection amplitude is either 40000 or 20000 counts.
Then I flipped the DC bias and offsets, but without flipping the signal polarity, and repeated the measurements at 3.19 Hz. The TF polarity flipped as expected, but the TF amplitude increased overall by a factor of 1.2-ish if you look at PIT, or 1.6-ish if you look at YAW.
In the attached, left two panels show PIT response, right YAW. In in each panel, to the right is non-flipped data set, to the left is flipped.
This is highly repeatable, the measurement error seems to be 5% or less. The results don't depend on the excitation amplitude (this is expected).
From the fact that the TF polarity flipped though the excitation polarity stayed the same, this cannot be just electronics coupling into OPLEV.
Keita, Do you suppose it is simply that the force is proportional to (Vsig-Vbias)^2 ? RW
My point is that it should be proportional to (vsig-vbias)^2, but it doesn't appear to be.
Expected behavior is that when you flip the sign of vbias without flipping vsig, you will have exactly the same amplitude (which is not the case in my measurement) but with exactly the opposite polarity (which is the case in my measurement).
Since I'm not measuring vbias (I just trust the DAC counts), it could be that there's an offset in ESD driver, but the offset should be huge.
Charge?
Added a ZFL-500HLN amplifier to the ALS DIFF RF path.
Alexa and I found that this amplifier was not working, so we removed it.
The amplifier on DIFF was not working because we had input/output backwards. I have installed another amplifier on DIFF. I have also put a new amplifier on COMM (HNL) that piggy-backs the power from the BBPD; this amplifier has slightly less gain (about +24dBm in comparison to +30dBm), so we expect about -7dBm for the beatnote.
