Temperature sensors for the HVAC system tend to be on the walls. So when it is really cold outside, I think the temperature at the chambers tends to be higher than usual because the system is trying to maintain temperature near the walls, which, because of the higher gradient, tend to be colder relative to the room than usual. Figures 1 and 2 show a moderately good correlation between range/inability to lock and BSC3 temperature (BSC1 is not that different). The trouble really seems to start when BSC3 warms up above 9810 counts. It seems like a long shot, but it might be worth turning the zone 1A temperature set point down, and perhaps others as well, when it is cold, so that the temperature is not so hot away from the walls. Figure 3 shows that BSC3 tends to be warm when the outside temperature is cold. Eventually it might be good to control the HVAC with sensors near the chambers and not on the walls.
Indeed, it does look like the temp of the LVEA has been wandering more the last few days than it was last week when we at least had ~60MPC locking and is seen by vertical drifts in the ITM and BS suspension which are translating to pitch motion (see attached). Not sure if this is the only culprit for why we're down on our lock acquistion, but plots attached are complement to Robert's plots.
IMs don't look too far off from where they were during last 8Mpc locks... IM4 is a few clicks off but people have been doing various adjustments of it during alignments over the last few days.
End station temps look to have stabilized over the last ~10 days, although the ETMy and TMSY have been drifting in vertical since then for some reason, maybe following other pointing issues around.
FYI. We have seen similar behaviour of QUADs at LLO during a recent cold snap, vertical heights drifted by as much as -50 um, approaching close to -100 um, which is our threshold where we begin to get concerned about bottoming out on earthquake stops. The sagging is indicative of heating of the blades, despite the cold weather and stability exhibited by the HVAC system maintaining VEA set-point temperatures, therefore, I suspect additional HVAC heating could be responsible for the drift observed in the suspensions (see LLO aLOG entry 30776).
Control Room Screens are working for me, I did have to press the browser's refresh button the first time to get the thumbnails to show up.
Quick Summary: ALSY spot have fringes. ALSX can't be maximized. COMM beatnote wouldn't go beyond 5.6. TMS optics have been moved a lot before I got here. I'm putting the optics back to where they were 19 hours ago when we had th ebest range last night. Stay tuned.
Stopping at DC readout to damp Bounce, Roll, and 2nd harmonics violin modes
Something went unstable during Lownoise ASC. Then lockloss when Guardian moved up to Coil Drivers.
ITMY went all the way to -10 um after the lockloss. I had to bring it back by hand.
Since then I have been repeatedly losing lock at ALS_DIFF. I'm going through Jenne's troubleshooting document.
That StripTool image is pretty identical to the only semi-successful locking attempt that I had during my shift and also at the same stage. Comment to aLog 33292
So far with the ALS Diff checklist:
- ETMX ESD setting is correct
- H1:ALS-X_REFL_CTRL_OUTMON is between +/-10
- There's no way to tell if Bounce and Roll mode is not too high, it's burried in DARM spectrum. I damped them before the lockloss.
- ALS-[X,Y]_REFL_CTRL_OUT_DQ don't look glitchy while sitting at Locking Arms green.
I'm doing an initial alignment AGAIN since Patrick seems to make it work with that...
22:30 Bounce mode came back high. Stopped at DC readout transition to damp it.
22:39 Lockloss at LOWNOISE_ASC AGAIN. Similar behavior as last time.
I'm putting slide bars back to where they were 22 hours ago. I already did IM3 when I took this screenshot. Then I will trend oplevs and fine tune the alignment.
TITLE: 01/16 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC STATE of H1: Aligning INCOMING OPERATOR: Nutsinee SHIFT SUMMARY: I made it as far as COIL_DRIVERS and then things started getting worse again. At the end I could no longer lock DRMI on 1f. I started another initial alignment that I am handing over to Nutsinee. Sheila has been helping over the phone. LOG: See previous alogs.
I have run the dither alignment twice for both TMSY and ITMY. This improved things to the point that I can now get flashes and lock at around .28. However I am still struggling to improve on this.
19:54 UTC Got ALSY locked for initial alignment.
21:28 UTC Initial alignment done. Struggled with locking SRCL. Had to move SR2 and SRM mostly in YAW I believe.
21:53 UTC Lock loss from ENGAGE_SOFT_LOOPS.
22:29 UTC Lock loss from COIL_DRIVERS. The power recycling gain began oscillating prior.
23:14 UTC Now struggling with ENGAGE_DRMI_ASC.
The green power in the arms now appears too low to move from LOCKING_ARMS_GREEN to LOCKING_ALS. The Y arm is steady at around .87 and the X arm is steady at around .91. The diag main message is 'Waiting for arms to settle'. Does this mean another initial alignment? I am going to try and adjust TMS first.
23:47 UTC Now no longer locking DRMI 1f. Starting another initial alignment.
TITLE: 01/15 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC STATE of H1: Aligning OUTGOING OPERATOR: Ed CURRENT ENVIRONMENT: Wind: 3mph Gusts, 1mph 5min avg Primary useism: 0.03 μm/s Secondary useism: 0.27 μm/s QUICK SUMMARY: I could not log in to the alog earlier. This seems to be resolved now (see Dave's alog). Ed was having trouble with initial alignment when I arrived and was on the phone with Keita. Ed told me that Keita sent a message to Sheila, Jenne and Kiwamu to ask for assistance. Sheila called me and I told her about the diag main message I was seeing: 'IM_ALIGNED: IM3 P is out of its nominal range of 1961'. I sent her a plot of the IM 1-4 witness sensor positions over the last 7 days. She suggested that I move IM3 P back to 1961 and retry initial alignment. I did so and am now on the initial alignment of the arms on green. The X arm locked easily, but I am not getting any signal for the Y arm. There is a spot on the camera, but it does not seem to move with the optics. I trended back and moved the positions of the ITM, TMS and ETM to no avail. I have just started looking for the dither align script.
Moved IM3P alignment offset from 29542 to 29864 to move the WIT P from 1945 to 1961.
We are seeing glitches that come in sets and look like stacks of harmonics in the most recent lock. It looks like they can be explained as the non-linear upconversion of some 1 kHz violin modes, based on the spacing between glitches. We think that these glitches happen when the violin modes get high enough to run into some non-linearity in the sensing. I used the derivative of OMC-DCPD_SUM, since it seems like these glitches should not care about the DC value and would most likely be some kind of slew-rate limit. The 1 kHz violin modes dominate the RMS, in particular a pair at 1009.44 and 1009.487 Hz. The beat frequency between these gives a period of about 21 seconds, which is the spacing between bursts of glitches. The glitches occur when the amplitude of the DCPD derivative is highest. The amplitude has a period of 21 seconds because of the two 1 kHz violin modes. Comparing to the previous lock, when these glitches were not present, the amplitude of these two modes is no higher. But there are a number of other modes near 1 kHz and several of those are substantially higher. So they may have pushed the amplitude into the nonlinear region. Attached is a PDF showing the glitches as they appear on the summary page (they are most clearly seen at 2 kHz in Omicron), and the comparison of the 1 kHz spectrum with the previous lock which did not have these glitches. The second page shows the bursts of glitches compared to the amplitude of the DCPD derivative.
If the interferometer is up I will spend some time damping them tonight.
It is important to notice that this 2kHz glitch line has been appearing and dissapearing quite irregularly in the past, but when it is present the associated Omicron glitches are of high SNR. In fact the last time this line showed up was all the way back to 29-30th November:
* 2kHz glitch line started to show on first 29th Nov lock
* 2kHz glitch line disappears on the 30th Nov
Originally I thought that the 2kHz glitch line could have been related to PCALX roaming calibration lines, based on Evan's alog on PCALX roaming calibration line frequency changes. The 2kHz glitch line seem to start as soon as the detector locked after the PCALX calibration line at 1001.3Hz was activated on UTC 2016-11-30 17:16:00, and then the glitch line disappeared around the time the cal line at 2001.3Hz was moved to 2501Hz at 2016-11-30 22:07:00. The fact that the time coincidence was not precise made me believe that the time coincidence may have been casual. It can now be confirmed that it must be unrelated because not such PCALX roaming line was not set at 2001.3Hz during the time of the current appearence of the 2kHz glitch line.
The obvious question is if the November 2kHz high SNR glitch line shows a similar 21 second spacing between bursts of glitches. The answer is yes ,as seen next during the dissapearance of the 2kHz glitch line on the 30th Nov (attached are the original images from which this image was made):
A zoom around the beginning of the above spectrum shows the ~21secs periodicity of the features:
A closer look to the 2nd harmonic violin modes for 30 mins during the time of the 2kHz glitch line (in blue) and 30 mins after the glitch line dissapears (in red) shows that only few violin modes were higher during the time of the 2kHz glitch line:
There are two cases when the blue lines are higher than the red:
* At about 1003.7Hz:
* At about 1009_4Hz:
It is clear that only the pair at around 1009.45Hz would beat with a periodicity of about 20 seconds. And in this case while the lower frequency violin mode of the pair does not change much in amplitude however it is the higher frequency violin mode of the pair which increases by 30.
I have also compared the 1009.45Hz pair peaks amplitude for 4 different cases, two of them correspond to times when the 2kHz glitch line was present, and 2 other (dashed lines) to cases when the glitch line was not present. It shows how the higher frequency line has to be high enough to cause the nonlinearity for the 2kHz glitch to be present:
Nutsinee has just now created a damping filter for this pair of violin modes, so hopefully this will be enough to avoid growth of this peaks to the point of causing appearance of the 2kHz glitch line but only time will tell.
A small note for the operation purpose: Borja's 2kHz glitches thresholds on 1009.44Hz and 1009.49Hz correspond to
8e-13 and 6.5e-13 m/sqrt(Hz) in the (dtt calibrated) CAL-DELTAL_EXTERNAL_DQ channel in November
and
7e-13 and 4.4e-13 m/sqrt(Hz) in January.
Now that the guardian is turning the damping on, these two modes should be well controlled. But if something bad happened and the modes ring up I would suggest operators to take some time to damp the mode when they get close to 1e-12 on DARM FOM.