During all lock stretches, the sensitivity steadily degrades over time. The plot below shows the range trend over one of the last locks.
I picked a few times during the lock, when there were likely no actions on the IFO, and computed the DARM spectrum. The evolution is shown below
Clearly, what degrades the sensitivity is very well approximated by a 1/f displacement noise.
Here's a comparison of degrading sensitivity over another long lock stretch. Similar 1/f noise
This seems to be the same noise that we saw when the CARM noise gain was accidentally low. As the high frequency noise get worse (presumably because the frequency noise coupling get worse) the noise at 100 Hz gets worse.
BruCo is running again on ldas-pcdev12. Here are two scans comparing the "low" and "high" noise times:
Low noise: https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_1223676058/
High noise: https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_1223680111/
It's CM settings that were somehow screwed up on Sunday evening. (Craig, Keita)
As noted in alog 44533 CM gain used to be low (that's what Sheila was talking about when she wrote "CM gain was accidentally low") before Craig changed in on Oct/11, and after the change the frequency noise (in kHz range) as well as 100Hz noise got better.
It stayed that way for 3.5 days or so, but the bad behavior came back on 2018/Oct/14 23:30 UTC and it stayed bad since then (first attachment.) That's Sunday 16:30 Pacific or so.
It turns out that CM settings were somehow changed (second attachment) at that time though people were not working on CM at all (second attachment).
I and Craig are trying to figure out what happened.
Note that when it was really bad the excess noise was not smooth, it had a bump (e.g. look at this plot from alog 44533).
Somehow, between the time Keita pointed out as "Something Happened" and the previous lock, there was an error added to step 13 of CARM_TO_ANALOG:
if self.counter == 13 and self.timer['wait']:
if ezca['IMC-REFL_SERVO_IN2GAIN'] < 0:
ezca['IMC-REFL_SERVO_IN2GAIN'] += 1 # from -12 to 0 dB
ezca['LSC-MCL_GAIN'] *= round(10**(1./20),4) # from 34.6 to 47.6 dB
else:
self.timer['wait'] = 0.5
self.counter += 1
self.counter += 1
This error basically caused CARM_TO_ANALOG to skip this step without adding an additional 11 dB of gain to CARM.
The line highlighted in red has been removed. Our frequency noise suppression should return.
As a test of 1/f noise, we can get to NOMINAL LOW NOISE and reduce the CARM gain to see if this low frequency coupling changes somehow.
