Oli, Georgia, Elenna
While we were taking care of several LSC items, Georgia pointed out that we are seeing the 0.8 Hz buzzing in INP1 yaw again, and to a lesser degree CHARD Y. While we did several useless things to debug, I noticed that the DC centering loops were also buzzing with the same frequency oscillation. I decided to try changing the DC centering loop gain (since I had raised the loop gain significantly last week), and noticed that NONE of the DC6 pitch or yaw filters were engaged, but a gain of 1 was engaged, essentially feeding the input signal right back to PM1. Bad!!
Searchingh DC6 in the guardian shows that in PREP_ASC_FOR_FULL_IFO, I found these lines of code:
ezca.get_LIGOFilter('ASC-DC6_P').only_on('INPUT','OUTPUT','DECIMATION')ezca.get_LIGOFilter('ASC-DC6_Y').only_on('INPUT','OUTPUT','DECIMATION')Not good! Since the gain of DC6 is set on, this effectively keeps the gain on but turns off all the filters and just feeds back the pure error signal to the suspension.
I commented those lines out of the guardian and loaded, and then reengaged all the proper filters. A large 28 Hz line that had been present in DARM disappeared at this point, likely because the DC centering loops require a 28 Hz notch to avoid some suspension mode.
After this, I then tried turning down the DC1 and DC2 pitch and yaw gains to -1. This seemed to help the buzzing in CHARD and INP1. Georgia and I were able to engage all the low noise ASC settings without any issues (CHARD P had been held in high bandwidth to avoid a ring up and the soft loops had high gain for the same reason).
We need to monitor for a while longer and also try relocking, but these changes may fix many of our ASC issues.
I have SDFed these gain changes.
While locking this arvo, we stepped through lownoise_asc slowly by hand trying to catch which step was causing the 1Hz ring up that caused locklosses yesterday.
One time we reduced the DSOFT_Y gain down, thought we saw something ring up, but when we gradually decreased it the ring up did not come back.
The CHARD_P filter and gain changes in this state seemed to cause some oscillations in the PRG, so we left CHARD_P in a high noise state for a while. This left a lot of noise in DARM (first screenshot). We later ran the lownoise_asc steps for CHARD_P, after Craig and pals phased the LSC-POP diode, and this seemed to fine (second screenshot).
Hopefully the reduced gain in DC1 and 2 and the existance of filters in the DC6 loops will improve the ASC. But if we still see ringups in lownoise_asc the next best thing to try is undoing the steps for CHARD_P.
We now have confirmation that the DC centering loops have been having a significant effect on the ASC and causing many of our problems.
The DC loops appeared to have a large amount of gain peaking at low power around 0.8 Hz (I turned up the gain too high and didn't check the gain peaking). After we powered up I measured CHARD Y, which is a loop whose gain we have not been able to raise until reaching high power during the past few days. Attached is a comparison of the CHARD Y OLG both with the DC centering loop gain high and low (cut by half).
We were able to turn up and turn down the DC centering gain to confirm that it is causing the right half plane zero in CHARD Y. We only measured CHARD Y so we don't know the effect on other loops, but it appears that other loops are also more stable with less gain in the DC centering.
I have turned up the CHARD Y gain at engage ASC with no problems, this is now saved in the guardian.