J. Kissel Not clear if the end stations need to be rebooted this morning after the corner station front-ends seems to have crashed, but I'm reconciling the ALS EX and ALS EY front-end models with their settings definition files. It is dreadfully obvious from the time stamp that a few filters in ALS WFS control (DOFs 1 and 2) are turned on and off by guardian during the lock acquisition sequence, namely FM9, a -14 dB gain filter. As such, I'm unmonitoring FM9 in the ALS-[X/Y]_WFS_DOF_[1/2]_[P/Y] filter banks. Note that these are filter banks, so I can individual monitor / unmonitor this filter module only, so I have. I've also consistently unmonitored the inputs to these filter banks, which are also guardian controlled, and weren't consistently unmonitored in any of the 6 filter banks. There remains several diffs that are either - related to the fact that we're currently ignoring ALS WFS on the X arm because they're broken LHO aLOG 44206, which appears to be coded into the guardian, or - because (surprisingly) some of the elements of the ALS Y WFS sensing matrices have changed. I attach the remaining diffs that I'm leaving present.
Both X and Y end ALS WFS sensing matrices were changed on September 26th, as reported in 44178.
The new values should be accepted.
J. Kissel I'm not sure if ISC - end station models will die today during the recovery of CDS, but I see that transmon QPD dark offsets have been adjusted yesterday evening. Attached are the differences, and I've accepted them into SDF. (Ignore the "PWR_CIRC" differences. Richard is working on ALS fibers, and we've turned off the fiber light going to the end stations. This was *not* accepted.)
J. Kissel Nothing too terribly exciting here. Angle to Length decoupling gains are changed rather frequently as folks are searching for the right beam spot positions in the arms hoping to reduce the ASC noise couplings to DARM. A violin mode control filter (MODE 7) had it's actuation changed from P to Y on ITMY. I've accepted the MODE7 DOF change, but I leave the A2L gains to perish and capture them in the SDF screenshots below (they'll likely just be run again once we recover the IFO this afternoon).
J. Kissel Reconciling SDFs this morning, in prep for what will likely be an entire world reboot, given the apparent dolphin crash last night, I've found that someone has installed notches in the H1 SUS BS M3 ISCINF P and Y banks for 19.1 and 23.1 Hz. The dynamical model says that the beam splitter's highest vertical (a.k.a. V3, "bounce") and roll (R3) modes should be at 17.54 and 25.74 Hz, some previous aLOGs quote them at 17.7 & 25.7 Hz LHO aLOG 28470, so I'm quite surprised to see these 2 Hz away from the expected frequency, but so be it. These filters have been ON since Fri Sep 28 2018. I couldn't find a corresponding aLOG. I've accepted these filters as "ON" in FMs 2 and 3, called "not19.1" and "not23.1" with the following design strings: notch(19.1,30,40) notch(23.1,30,40) respectively.
The following models / PLCs are planned to be booted today and have differences against their settings definitions file (SDF).
OAF
ASC
CS ECAT PLC2
CS ECAT PLC3
CS ECAT PLC4
EY ECAT PLC2
EX ECAT PLC2
I've screen captured those differences below. For those whom are doing the booting (Jenne, Jeff, Dave, Patrick) try to make sure you've reconciled these changes BEFORE you get started with boots. For the record, the IFO ws in the DOWN state when these SDFs were captured. It also appears as though the entire front-end world has crashed last night, but that does not seem to be affecting the reporting of differences in settings.
We also plan to boot these processes, but they don't have any differences with there current reference file (safe)
CAL EY
CAL CS
I, finally, attach a screen cap of the SDF overview.
ECAT settings don't come back up exactly the same after PLC restart. See attached screenshots of ECAT CS PLC2 and PLC4 after Patrick finished his prep work for TwinCAT 3 this morning in which he had to restart all IFO-related PLCs. Compare these agains the above screenshots taken before the boot.
The TwinCAT system saves its values once minute. After a reboot it will restore to the last set point automatically—in most cases. If the code was changed significantly, it may choose to ignore the old settings and start fresh with all zeroes. The above screens indicate the former, where the settings were restored to the last settings, but probably the sdf was and is outdated. TwinCAT will never use an EPICS/SDF saverestore, this has to be done by a user.
The LVEA has transitioned to LASER SAFE.
This is under work permit #7854.
n.b. The TCSX table enclosure was found open with the fans going, presumably to let the table dry
out in preparation for fixing the on-table leak this morning. I have removed the key from the CO2 laser
controller, which will prevent inadvertent turning on of the CO2 laser. Please see me for return of the
key.
Sheila, Georgia, Craig Today we changed the phasing on REFL9 from 5.9 to 84.8 degrees. We've been investigating fast locklosses, probably caused by CARM. We noticed that if we moved the CARM servo board common offset around, we would win or lose arm power and see a change in the amount of DC power on REFL LF. (Pic 1) We also noticed that when powering up, we saw the offset in REFL9 I change by a lot, along with weird glitches in Q associated with servo board IN1 gain changes. (Pic 3) We went out to the racks, injected a line into CARM above the UGF of about 10 kHz, and checked the REFL9 I and Q monitors. We found that the phasing was terrible: there was far more signal in Q than I. We adjusted the phase, while measuring CARM and lowering the CARM IN1 gain from 3 dB to -9 dB to maintain our UGF. After adjusting the phase, we tested the CARM servo board common offset again and found the arm power and REFL LF to be relatively insensitive. (Pic 2) Sheila and Georgia are changing the guardian so that our digital CARM gain is lower since we have so much more optical gain now. I remeasured the CARM loop and posted the results in the pdf. The gains are in flux right now, for this plot CARM servo board IN1 gain = 1 dB, and Sum Node A IN2 gain = -15 dB at 5 watts of input power. We powered up from 5 watts to 15 watts, and I watched REFL9 I offset again (Pic 4). It still changed, but by about half as much as it did when we previously went from 2 watts to 5 watts in Pic 3. We are sitting much closer to the resonance now that we aren't dumping a factor of 5 of our optical gain.
We have tried to adjust the CARM gains correctly in the guardian to account for the increased sensor gain from our phasing. We have tested this in the adjust_power state, but not the CARM_TO_ANALOG state.
| 2W before rephasing | 5W before re-phasing | 5W after rephrasing | 2W after re-phasing (should be settings after CARM_TO_ANALOG now) | |
| SUM node A in2 gain | -3dB | -3dB | -15dB | -17dB |
| REFL IN1 gain | +11dB | +3dB | +1dB | 11dB |
| total | +8dB | 0dB | -14dB | -6dB |
The CARM_TO_ANALOG state could be re-written in a way that would be less confusing. We increased the sensing gain by 14dB with the re-phasing, so we have reduced the SUM node gain by 14 dB for each of the steps in the CARM_TO_ANALOG guardian.
After this we were stayed locked with 15W of input power and a PR gain of 45, for 20 minutes, then powered up to 20.5W of input power (PR gain dropped to 41) and lost lock after 5 minutes.
Georgia is now turning off CO2X and its chiller to prevent overflowing the bucket that is catching the leak.
It looks like the front-end world has crashed. Here's a screenshot of what I think are the relevant SDF differences for this aLOG such that we don't lose this new goodness.
Since nobody wrote, I summarize the story.
1. Some remember that, at some point in the (distant?) past, Q and I cables were swapped, i.e. Q output was used as I output instead of setting almost 90 degrees of phase in delay line. Everything was good except for cross-cabling.
2. Recently people found on the floor that the demod phase was off by almost 90 degrees.
3. At that time, Q and I were found to be NOT cross-cabled (I output went to I channel). Somebody should have disconnected them and put them back in a "correct" order.
4. The cables were swapped back again (Q output went to I channel) to make IFO happy.
Cross-cabling needs to be fixed before O3 to prevent this from happening again, but this is not a priority for now (it works).
Gabriele, Jenne, Sheila, Hang
We fixed the right-hand zero in CS_P OLTF (LHO:44216) by using a blended error signal for CSOFT, as illustrated in LHO:44237.
Specifically, we used a combination of TR QPDs that are insensitive to TMS drifts (LHO:44240) at low freq, and then blend in a different combination of TR QPDs that are insensitive to CHARD_P for the high freq input. The blending freq is set to 0.06 Hz for now.
The input matrix we use is now
| TR_X_A | TR_X_B | TR_Y_A | TR_Y_B | |
| -0.619 | 0.313 | -0.271 | 0.133 | CSOFT_P_A (< 0.06 Hz) |
| -0.140 | 0.135 | 0. | 0.039 | CSOFT_P_B (> 0.06 Hz) |
The HF input is consistent with those measured in LHO:44237 with the error in the overall sign fixed. We set the overall scale by dithering CS at 13.2 Hz and match the two sets of signals' responses.
With this setup we measured the CSOFT OLTF as shown in the attached plot. The red trace was the measurement result for the new CS loop with blended error signal, and the blue was the O2 reference. The brown trace was the one showed the right-hand-zero at 0.85 Hz due to cross-coupling with CHARD, and presumably with DH/DS as well (as subtracting CH alone did not fix the problem; LHO:44246).
After the measurement, we further turned off the -20 dB filter (FM1 in CSOFT_P filter bank) so the current CSOFT loop should have a high BW of 0.8 Hz or so, matching the O2 reference.
In addition to switching off the -20db filter, we could also increase the gain from 3 to 30.
J Kissel, reporting for S. Dwyer, J. Hanks, G. Vajente While making a routine parameter change in FM1 of the SRCL1 filter bank (a cdsFiltCtrl that can receive front-end input from a PD trigger; the change was to change the filter ramp time from 0.3 to 0.1 to 0.25, loaded by hitting the "individual" bank coefficient load), we found the filter started to do bonkers things. This filter, in FM1, is intended to be a +106dB (x200000) gain-only filter. However, with the input disabled, and an input offset of 0.1 and having only this filter bank on (under both front-end control and manual control) would result in an output of millions of counts -- and would still be varying as though there was an input. Also, the status lights underneath the filter request button were behaving confusingly as well -- the "requested" light could be manipulated to be on, but the "it's on" light would never flip green, OR (after trying to load several over different configurations) we could get it to hang with the "it's on" light to stay on, and the "requested" light was off / red. And those statuses did not correlate with the output. We found no other choice but to restart the h1lsc front-end process (with the help of Jonathan), which appears to have fixed the problem. Note that this a new computer (recently upgraded two weeks ago), and we're running a new RCG (two weeks ago).
Opened corresponding FRS Ticket 11570. The problem appears to be solved, but I left it open as PENDING for the CDS team review for closure.
I'm getting an error I have never seen before when trying to compile the ASC model with the new smooth limiter. I think it's because I was trying to use ANDs and relational_operators (greater than, less thans), and those don't actually exist in CDS, despite that they're in CDS_PARTS. I tried compiling with the ANDs replaced by multiplication, and I got fewer errors. So, I think if I were to get rid of the relationals also, it'd be okay. But, I really want those there. I'll back out my ASC model modifications, and check in with Team CDS in the morning.
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16645: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16648: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16648: error: expected ';' before 'noperator6'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16648: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16651: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16651: error: expected ';' before 'noperator7'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16660: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16660: error: expected ';' before 'noperator'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16663: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16663: error: expected ';' before 'noperator1'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16663: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16666: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16666: error: expected ';' before 'noperator4'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16666: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16669: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16669: error: expected ';' before 'noperator6'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16669: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16672: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16672: error: expected ';' before 'noperator7'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16681: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16681: error: expected ';' before 'noperator'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16684: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16684: error: expected ';' before 'noperator1'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16684: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16687: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16687: error: expected ';' before 'noperator4'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16687: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16690: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16690: error: expected ';' before 'noperator6'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16690: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16693: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16693: error: expected ';' before 'noperator7'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16702: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16702: error: expected ';' before 'noperator'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16705: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16705: error: expected ';' before 'noperator1'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16705: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16708: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16708: error: expected ';' before 'noperator4'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16708: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16711: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16711: error: expected ';' before 'noperator6'
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16711: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16714: error: stray '' in program
/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.c:16714: error: expected ';' before 'noperator7'
make[3]: *** [/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc/h1asc.o] Error 1
make[2]: *** [_module_/opt/rtcds/lho/h1/rtbuild/rt-3.4.3/src/fe/h1asc] Error 2
make[1]: *** [default] Error 2
make: *** [h1asc] Error 1
J. Kissel WP 7847 ECR E1500386 IIET Ticket 11551 I've installed a copy-and-paste of the X end's hardware injection library part and its associated connection the DAC. This is in the event that we control the O3 interferometer with ETMX instead of ETMY. In that event, we'll need to switch to using PCALX as our calibration line producer, which means that we'll have to switch to using PCALY as our hardware injection producer. This comes with the following new channels stored in the frames at 16 kHz: H1:CAL-PINJY_HARDWARE_OUT_DQ H1:CAL-PINJY_CW_OUT_DQ H1:CAL-PINJY_TRANSIENT_OUT_DQ and an upgraded, uint32 ODC channel sampled at 256, H1:CAL-PINJY_ODC_CHANNEL_OUT_DQ I'll call the corresponding MEDM screen from the site map once these changes are installed tomorrow. Attached are screenshots of the Simulink model, - [...]_Before.png shows the top level h1caley.mdl before the change, having only the ODC channel below the main PCAL system block, - [...]_After.png shows the top level after the change - [...]_LibraryPart.png reminds us what's in the innards of the library part
TITLE: 10/01 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: None
SHIFT SUMMARY:
15:00 (8:00) Start of shift
15:34 (8:24) Karen leaving EY
16:20 (9:20) Karen to EX
16:45 (9:45) Hugh to Optics Lab
16:51 (9:51) Karen leaving EX
16:56 (9:56) Hugh out of Optics Lab
17:00 (10:00) TJ to LVEA -- Investigate TCSX table leak
17:03 (10:03) Karen to LVEA
17:36 (10:36) Ed to LVEA -- re-centering oplevs
17:50 (10:50) Ed out of LVEA
18:04 (11:04) Karen out of LVEA
18:07 (11:07) Sheila, Nutsinee, Haocun to SQZ table
Fil to EX -- Looking at ESD install for tomorrow
18:35 (11:35) Betsy to LVEA -- test out new card, check out TCSX leak
18:38 (11:38) Sheila, Nutsinee, Haocun back from LVEA
18:53 (11:53) Jason to LVEA -- investigate TCSX water leak
18:55 (11:55) TJ, Tvo, Betsy out of LVEA
18:56 (11:56) Fil back from EX
19:58 (12:58) Jason back from LVEA
20:50 (13:50) TJ, Jason, Jeff to TCSX table
21:13 (14:13) Sheila, Nutsinee, Haocun, Terry to SQZ racks
21:28 (14:28) TJ, Jason, Jeff back
22:41 (15:41) Sheila back from LVEA
23:00 (16:00) End of shift
In conversation with Stefan, I have created a "smooth limiter" that allows you to set an EPICS limit, but the signal that is limited doesn't have a discontinuity in the derivative.
This will be particularly useful for ASC loops that have a large error signal when they are first engaged. We want to implement a limit (as Stefan did in alog 44133), but we don't want a sharp derivative in the limited error signal, since that can cause trouble with our control filters (see the outmons in Stefan's attached plots, where there is a lot of hash when the error signal comes off the limit). Having an error signal limiter should enable us to just turn on our ASC loops at full gain immediately, rather than constantly waiting for convergence before increasing gains, and not worry about large overshoots. The smooth limiter lets us have those benefits, and should also prevent the transients in the control signals that are seen in Stefan's plots.
The transition between the y=x (no limit) region and the hard limit is done with a 2nd order polynomial, fit such that the derivative is matched at each of the transition regions (i.e. the derivative is 1 when it's on the y=x side, and the derivative is zero when it is on the y=limit side). The size of the transition region is variable, and can be chosen by the user for each limiter via a smoothing parameter. The attached plots show examples of a signal that has limits at +/-1, with various values of the smoothing parameter (0.1, 0.5, 1, 2). The smoothing factor must be between (0,2]. Neither the smoothing factor nor the limit can be zero or negative - there is an abs() and lower limit saturation to enforce this, so that we don't divide by zero.
Also attached is the simulink code for the library part. We should be able to stick this at the error point of all of our ASC loops. (Screens will need to be modified to reflect these new limiters....)
In addition to the limiter itself, I add an offset just in front of the limiter. Note that this offset does not have an on/off switch (just use zero if you don't want an offset), nor does it have a ramp ability.
I'm backing out this change for today. See alog 44265 for my compiling error.
J. Kissel
One of the design considerations for A+ larger beam splitter suspension is whether to add actuation on to the final (optic) stage of suspension. The primary motivator for doing so is to improve DRMI lock acquisition speed. In order to model whether that will be useful, and/or whether the change in force allocation will affect things like whether we'll be able to constantly run broadband feedback on the second stage of the supporting isolation platform (whether we can run isolation loops on stage 2 on the BS ISI), we need some representative times in which DRMI is attempting to lock as a part of a normally functional full IFO acquisition sequence on the aLIGO beam splitter suspension, which only has actuation at the penultimate (M2) and top (M1) stages. The following are a list of times gathered today:
Sep 27 2018 17:39:00 - 17:41:45 UTC
Sep 27 2018 19:25:00 - 19:40:00 UTC
Sep 27 2018 21:15:00 - 21:35:00 UTC
Sep 27 2018 23:18:10 - 23:21:40 UTC
As per normal, the BS ISI ST2 feedback loops are off, but the platform (and HEPI) otherwise fully functional as designed
- ground to ISI sensor correction is on for X and Y, with CML_BB_SC (Conor Mow-Lowry Broad Band Sensor Correction filters)
- ground to HEPI sensor correction is on for Z, with Mitt_SC (Rich Mittleman broadband Sensor Correction filters)
- all ST1 DOFs besides Z have "Quite 250" blends, Z has 45 mHz blends.
As for the suspension,
- top-mass OSEM damping is ON,
- optic (M3) to penultimate (M2) stage optical lever damping is ON.
- penultimate M2 coil drivers are in their highest range stage (state 2),
- top mass (M1) coil drivers are in the low-noise state (also state 2).
- The top mass has DC alignment offsets that are eating up only about 10% of the DAC range at the top mass (M1).
- during lock acquisition, there is no global alignment control present, only longitudinal. However, there is some frequency dependent
If anyone needs a refresher on more about each stage of aLIGO beam splitter's actuation strength, check out T1100602.
2018-09-28 20:32:45 - 20:38:00 UTC
2018-09-28 21:02:00 - 21:11:00 UTC
~15 Mph Wind, 0.1 useism
2018-10-01 22:40 - 22:55 UTC
2018-10-01 23:05 - 23:08 UTC
2018-10-01 23:22 - 23:33 UTC
10 mph level wind, 0.1 um/s level useism.