There was an ADC timeout error in the h1susb123 I/O chassis at 23:26 PDT yesterday night (May 26), which caused all of the user models (h1susbs, h1susitmx, h1susitmy) to stop running. I restarted all models without power cycling the electronics and the models restarted correctly.
Sheila, Kiwamu, Evan
Using the same method as described here, we determined that the switching of the ETMY ESD low-pass filter appears to be working fine and is compensated for correctly in the ESD output filters. With offloading turned off and DARM controlled by ETMX, the transfer function ETMY L3 LOCK L → DARM IN1 is identical in both LPF states. It very similar to the transfer function ETMX L3 LOCK L → DARM IN, up to a constant factor of −50. This is a little mysterious to us, since we expect only a factor of 20 at dc between the high-range driver and the LVLN driver.
Anyway, we partially transitioned control of DARM to ETMX and ETMY (we had them actuating with about equal strength), and then saw that the ETMY ESD drive rms was about 30 000 ct, with most of the counts coming from the 322 Hz and 538 Hz calibration lines. We turned the line amplitudes down by a factor of 10 (from 2 ct to 0.2 ct). However, h1susb123 went down right as we were about to complete the transition to ETMY.
The other piece of good news from tonight is that with tidal running the lock seems to be stable at 17 Watts of input power. We saw an oscillation in both ITMs, when we tried to increase the power, but this may be because we were unable to actuate on ITMX.
- thunderstorm early in the shift, tripped a panel alarm in the fire system
- Fire Department here to reset the pannel
- full alignment of IFO, after opening shutter at EX, then handed off to Kiwamu
- JeffK was at EY
- CP2 still in alarm after being filled
- RO water system is in alarm
Evan, Kiwamu, Sheila, Jeff
We noticed that we weren't really driving ITMX PUM. Evan and I went to the racks and saw that the DAC outputs were changing, but the LED that says DAC WD next to the DAC input was not lit and there was nothing coming out of the AI board. This is probably why Dan was unable to damp ITMX violin modes last night.
h1susb123 has stopped running, the symptoms are verry similar to what happened yesterday. There are several red block on the CDS overview screen, front end models are not synced to the IOP.
This coincided with kiwamu pulling out the AI chassis for ITMX L2, although we aren't sure if this is related or not. We don't know how to restart this without causing a problem for the dolphin network, so we're done locking for tonight.
AH-HA!
Replaced the 18-bit DAC Anti-Image interface board (possible bad optocoupler) for AI Chassis S1103818. Board removed S1103833 Board installed S1105351 Chassis was reinstalled and powered up. Verified both DAC WD LED's were on.
D. Gustafson, K. Izumi, S. Karki, J. Kissel, R. Savage, D. Tuyenbayev
Stay tuned for more details on each, but since we have so little time during these three days we're taking the "take the measurement now, look at it enough to know *you're* happy, *then* think and document it once we don't have any more time" approach, so please be patient if our aLOGging falls behind our activities. As such, I'm going to try and keep up with this daily "debriefing" so at least all are aware of our progress.
We've begun this week's calibration measurement's by doing the most invasive stuff first:
K. Izumi, D. Gustafson
- Made electronics measurements necessary to use the ALS DIFF VCO as a calibrated frequency actuator and compare line(s) generated by this VCO against ESD calibration lines in the DARM ASD to assess QUAD's actuation strength. See LHO aLOG 18634.
J. Kissel, R. Savage, S. Karki, D. Tuyenbayev
- Made SR785 measurements of all elements of the EY PCAL electronics chain, including the AA, AI, and full RX (reflection off of the test mass PD) and TX (a PD picking-off of the power transmitted into the test mass) that includes the PCAL interface chassis, similar to what has been done at LLO (see, e.g. LLO aLOGs 18106, 18199, 18309).
Goals for tomorrow:
K. Izumi (and whomever's willing to help)
- Get a free-swinging Michelson measurement of all stages of the ETMY's actuators that are involved in the hierarchy
J. Kissel, S. Karki
- Finish straight-forward electronics measurements for ETMX PCAL (i.e. the same that were done today at EY)
R. Savage, D. Tuyenbayev
- Stay at EY and compare PCAL PDs against other broad-band PDs to assess frequency response of PCAL PDs.
For an overnight test we have moved the Rx detector assembly into the Tx module and directed both output beams into the detector.
We plan to restore the nominal setup tomorrow morning.
Note that the cover on the Tx module is ajar while we are making this measurement.
I reviewed front end code which have outstanding local mods. I committed to SVN h1alsex.mdl, h1alsey.mdl, h1iopsusauxex.mdl, h1iopsusauxey.mdl, h1iopsusex.mdl, h1iopsusey.mdl. Files which still need committing: ASCIMC_MASTER.mdl, lscimc.mdl and CD_STATE_MACHINE.c
I "greened" up the IOP SDF by switching the DAC Duotone OFF on h1lsc, h1iscex, h1iscey.
I ensured filtermodule file synchronisation by pressing the LOAD_COEFF buttons on all models.
I verified all safe.snap files in the target areas are symbolic links to userapps, and checked in local mods.
I'm in the process of ensuring all filter module files are symbolic links to userapps (12 were not) and checking these into SVN. COEFF_LOAD is needed when the file is changed from local to symbolic link to clear the "modified file" message.
I copied the latest Beckhoff DAQ INI files to the DAQ. I had to remove the duplication of the H1:PSL channels as was seen before. The DAQ was restarted after this morning's model changes.
The IRIG-B excursion of h1seih45 following its power supply swap took several hours to come back down. The CDS Overview is now GREEN, any non-GREENs should be investigated.
Dave, Jim Weather station IOC's failed at EX, MX, and MY during lightning and heavy rain. Restarted IOC's, set alarm levels.
Do you often wonder if the BRS on or off? Do you stay up at night asking yourself if you should have turned it off when someone went down to EX?
Ask no more! There is now a BRS Active light on the ISI EX_GND medm screen.
Operators: if someone is heading down to EX, please make sure to check that the blue "Not in use" light is lit. If it is not and the brighter blue "ACTIVE" light is, go to the wiki: https://lhocds.ligo-wa.caltech.edu/wiki/UsefulScriptsForOperators and follow instructions to turn OFF the BRS sensor corrected signal.
An important thing to note: this medm light does not look at the sensor correction filters, only if the appropriate matrix elements are selected. This should not be a problem since the script turns on the correct filters when switching between using the BRS sensor corrected signal (ON) and not using the BRS signal (OFF). In the near future, a Guardian node will control this rather than scripts from the command line and, hopefully, eliminate all confusion.
A screenshot is attachted of the "Not in use" light.
Dick, Kiwamu,
We measured the volts-frequency relation of the ALS DIFF VCO by closing the ALS diff VCO PLL. We will upload the data and results later.
In order to measure the VCO response, we discconnected the DIFF beatnote from the phase-frequency discriminator and temporarily hooked up an IFR 2023A to simulate a clean signal. After we finished the measurement, we reconnected the beatnote cable back to the phase-frequency discrminator.
8:20 Peter and Ed to PSL enclosure. Lock loss right after.
8:22 Joe take the pesticide guy into the LVEA.
8:35 Kyle to EY. Pump RGA.
8:39 Joe out
Kris and Karen to LVEA
8:45 Elli & Nutsinee to HWS table near HAM4
8:39 Jason to PSL enclosure
9:00 Elli and Nutsinee out
9:40 Dick to ISC rack 2,3 to measure ground noise
9:48 Fil to EY and EX rack
9:52 Patrick back up database
10:32 Kyle to LVEA
10:48 Kyle out
10:53 Karen and Kris leaving EX, heading to EY
11:14 Richard & Jim swap power supplt SEI 4,5 (HAM4,5 brought to OFFLINE)
11:22 Jeff K. to EY moving binary IO cabel
11:26 Jason et al. out of PSL enclosure
11:40 EX Beckhoff restarted. Burtrestored to 06:10
11:43 Kiwamu and Dick to ISC rack. Disconnecting ALS diff PLC.
Jeff K. leaving EY. Dave restarting DAC.
11:46 Kris and Karen leaving EY
11:49 DAC back
11:53 Patrick done with the back up
11:58 Jeff K. bringing ISI EY back
11:59 Gerado to EY
12:15 Gerado and Kyle back from EY
12:53 Dick and Kiwamu back
13:15 TJ to EX restarting BRS
13:30 Richard to compression room EX
13:45 TJ back
13:50 Richard back
14:01 Dick and Kiwamu to ISC rack
14:14 Jim and TJ to EX - restarting BRS (again)
14:36 Jim & TJ back
Dick & Kiwamu back
15:35 Jason & Ed to ETMX OPLEV
15:44 Rick, Sudarshan, Darkhan, and Jeff K. to EY (Calibration work)
16:09 Jason and Ed back
Happy Maintenance Tuesday....
Just a heads up to commissioners:
I took a snapshot of the ETMy SDF diffs that popped up over the weekend of commissioning this morning. After Kissel's model fix from this morning, the diff list is smaller now. This may imply that some of the purposeful changes made to ETMy over the weekend have been undone - it is hard to tell because many of the things that were on the larger diff list from before the reboot were people likely "just testing" things that weren't working over the weekend. Or, there are some settings guardian is switching since we're not back up and lots of testing is taking place right now (meaning, I haven't caught them all and made SDF ignore them yet). Or, the reboot and sweep through the GRD SAFE-DAMp settings cleared house.
In any case, I post the before and after list in the event ETMy still is not acting properly for the evening commissioners and they are looking for "previous" settings..
Praxair delivery today resulted in CP2 overfill -> PID loop will eventually shut-off LN2 flow through transfer line as a response -> Warm transfer line will likely result in secondary under fill alarms tonight -> shimming LLCV slightly open will keep transfer line cold regardless of PID output to LLCV
Dan, Kiwamu, Evan
Tonight we worked on getting the interferometer back to its low-noise state. We are stable at 10 W, but there is some instability at higher powers.
First, at 3 W we manually steered the ITMs to a good recycling gain (38 W/W), and then updated the TMS QPD offsets. We also locked the arms in green, adjusted the green QPD offsets for maximum buildup, and then updated the ITM camera references. Then we re-enabled the ITM loops in the guardian. This allowed us to power up all the way to 21 W without significant degredation of the recycling gain.
After that, we were able to consistently engage the ASC with the guardian.
However, we found that at 21 W the interferomter suddenly unlocks in a matter of minutes. There seems to be no instability in the arm or sideband buildups before the lockloss. We looked at OMC DCPD signals for signs of PI, but we did not see anything ringing up during any of our short high-power locks. Some times to look at are 02:29:50, 02:59:50, 04:57:30, 06:55:00, all 2015-05-26 UTC. But any of the other 21 W locklosses in the past 12 hours follow this pattern.
We measured the OLTFs of PRCL, MICH, SRCL, and DARM before and after powering up, but they all look fine and did not change with power. For CARM, we start at 3 W with a UGF of 14 kHz with 47° of phase. Then during power-up, the electronic gain is automatically adjusted to compensate for the increased optical gain. The algorithm for this was shooting a little high, so after power-up the UGF was more like 27 kHz with 30° of phase. This is probably fine, but we adjusted the algorithm anyway, so that the UGF is placed at 19 kHz, with 45° of phase. Anyway, this did not solve the lockloss issue.
We also tried locking at some lower powers. At 15 W the interferometer lasted for about 15 minutes before unlocking. At 10 W, the lock time seems to be indefinite (at least 90 minutes).
Using FM9 in ETMY L1 LOCK L (zero at 2 Hz, pole at 5 Hz), we were able to push the L1 crossover from <1 Hz to 1.7 Hz by adjusting the filter gain from 0.16 to 0.31. Measurement attached, showing before and after. This is not included in the guardian. By pushing up the crossover, the rms drive to L2 decreases from >10000 ct to about 6000 ct or so.
For the record, we did not notice any kicks to the yaw of IMC REFL tonight.
Over the weekend we were able to re-commission the damping of the bounce, roll and violin modes. The bounce & violin damping settings have been propagated to the ISC_LOCK guardian, and should be stable (maybe). The roll mode settings have already changed once over the weekend, so I'll list what's been working, but your mileage may vary.
The attached spectrum (for 10W, low-noise ESD, *not calibrated*, no LSC FF, so don't study it too closely) shows the mode-damping progress. Note this was before the 2.4k and 2.8k violin harmonics were damped.
After struggling to apply very narrow band-pass filters a la Jeff's approach from alog:18483, we reverted to the method of very broad band-passes. These are loaded as FM3 in the DARM_DAMP_V filter banks. The frequencies follow those listed by Sheila in alog:18440 (we confirmed these frequencies were correct through the course of our damping exercise).
| ETMY | ETMX | ITMY | ITMX | |
| Frequency [Hz] | 9.73 | 9.77 | 9.81 | 9.85 |
| Filters | FM1 (+60deg), FM3 | FM3 | FM1 (+60deg), FM3, FM6 (+30deg) | FM2 (-60deg), FM3 |
| Gain | -0.3 | -0.5 | +1.0 | +0.3 |
The real key to squashing the bounce mode peak was to work out the damping settings for ETMX and ITMY (the optics which couple bounce --> longitudinal motion the least). The extra 30deg of phase for ITMY turned out to be important.
We were able to damp the ITMX roll mode, thus breaking the unpaired set of frequencies for roll modes and assigning each peak to an optic. The ITMX roll mode wasn't rung up this weekend, so we didn't have a chance to work out damping settings. The sign for damping the ETMY roll mode flipped between Sunday and Monday night, otherwise these damping settings were pretty stable.
For all the TMs the FM4 filter is a broad band-pass from 13.5 to 14.5Hz.
| ETMY | ETMX | ITMY | ITMX | |
| Frequency [Hz] | 13.816 | 13.889 | 13.930 | 13.978 |
| Filters | FM3 (-100dB), FM4, FM6 (+30deg) | FM3 (-100dB), FM4 | FM3 (-100dB), FM4 | ?? |
| Gain | -20 | +600 | -80 | ?? |
The roll mode is rung up after every lockloss (usually it's ETMY), so these settings need to be manually applied before the transition to DC readout. The gains listed in the table above are the "high-gain" damping state, if the mode is very rung up you need to start at a lower gain setting or you might saturate the M0 stage.
Recall that violin mode frequencies and their associated test masses were given in alogs 17365, 17502, and 17610.
All the identified modes are well-damped and have been enabled in the Guardian code, with the exception of ITMX. Despite many attempts I haven't been able to actuate on the ITMX modes at all. Before the realignment/recycling gain work the ITMX modes damped very easily, now I can't find a DOF (longitude, pitch, or yaw) or a phase setting to move the modes either up or down. It's hard to believe the L2 stage of ITMX isn't working, so we're not sure what the problem is. Maybe we just need more patience.
The complete set of violin mode damping settings is too large to list here; the various filters and gains are recorded in the guardian code. Some modes require a specific filter to get the right phase, others can be grouped together with broad band-pass filters without much trouble. In particular, ITMY requires separate filters for each mode, it's very difficult to construct a broad band-pass that catches more than one mode with the correct phase. We need to add more filter banks to the L2 DAMP section of the quad models if we want to squash the violin modes and their harmonics.
We did identify some new modes -- since we started feeding DARM back to the ETMY L2 stage we rang up the 4th, 5th, and 6th harmonics of that optic. These modes were easily damped and have been notched in the actuation path. The specific frequencies and damping settings were:
2424.38, 2427.25 Hz: Use FM6 of ETMY L2 DAMP MODE1, +60deg of phase, +100dB, gain=+20k, longitudinal direction
2878.7, 2882.5 Hz: Use FM6 of MODE2, no phase, gain=+10k, longitudinal direction
3330.6 Hz: Use FM5 of MODE3, -60deg of phase, gain=+20k, longitudinal direction
3335.7 Hz: Use FM6 of MODE3, no phase, gain=+20k, longitudinal direction
Keita, Sheila
In three of last night's 10 Watt locklosses, as well at the 15 Watt lockloss, the CARM loop dropped first, when IMC-F reached something around +/- 1440 kHz (the first screen shot attached is typical, 2015-05-26 15:23:17, 13:228:28, 12:10:29, and 7:38:16 at 15 Watts). Now that Jeff has fixed the model and we are using tidal again, this type of lockloss has not been bothering us tonight.
The slope of IMC-F is larger in the 15 Watt lockloss than the 10 Watts ones. A trend of IMC F and arm transmission from last night shows there are some inflection points in the slope of IMC F, although these don't corespond to changes in input power or changes in the state of the tidal state machine.
The other 4 locklosses that I looked at were not due to the IMC VCO, and I didn't come up with any good explanation for them. One notable feature in all of the others is the half a hertx oscillation in the ITM oplev damping loops, that starts when the power increased to 21 Watts, but it doesn't seem like this was the cause of the lockloss.
Tonight we were able to damp the roll modes with all of these settings, as well as ITMX for which we used -100 dB, bp13.9 (FM3+FM4) and a gain of 20. We also increased the gain for ETMX to 1000
Since it's the last opportunity to run a charge measurement on ETMx before the E7 run and immediate subsequent vent, I've started one now. Attached are medm snapshots of the nominal ETMx ESD screen settings, and also with the measurement in mid-sweep. I've followed the procedure Stuart outlines in his alog and TXT file at 17662.