The ETMy has been slowly sagging over the last few months. Recently, the sag rate has increased. While the temperature control at EY appears to be noisier during this December's cold spell, the overall temp has not changed by a large (greater than a deg C) amount, so I doubt that is the full cause of the drift. However, there have been a few pressure changes that the ETMy chamber has witnessed, according to the VE pressure channels I picked off and plotted (attached). I can't correlate all of the vertical motion humps exactly to these pressure steps, but the direction is correct: lower pressure means a a lower QUAD - that said, the sag usually happens much faster than weeks when the pressure or temp is changed, so wt-heck?. The most recent step in pressure shown on the plot was Dec 11th when the VE crew was doing some work on an IP bakeout at Y2-8. I'll poke them to see what the nature of the pressure step is. I worry a bit that there is something wrong with the channel after the step feature since it goes very quiet past that point.
The corresponding ETMy pitch motion change is much more subtle, which is good from an alignment standpoint. The BOSEMs still have plenty of signal, so we shouldn't have any issues with drive due to this drift.
Another day, another mystery. Keep H1 weird.
This pressure change is too small to account for the vetical drift. In log 15887, ETMY is measured to sag by 120 microns during pumpdown, a pressure change of 760 torr. For a pressure change on the order of 1e-8 torr, as seen here, you would therefore expect a vertical change of
(1e-8 / 760) * 120 microns = 1.6e-9 mincrons or 1.6e-15 meters. This is well into the noise of the OSEMs.
I think it is more likely the in-vacuum temperature is different from the VEA temperature sensor. Are the any in-chamber temperature sensors on the ISI table?
A consistency check you can do is see if it is just the suspension, or really a change in the whole chamber is to see if the ISI is sagging as well; or rather trying to sag since it has active position control. So, if the vertical ISI actuators are pushing up harder proportional to how much the quad is sagging, then there is at least some consistency throughout the chamber.
Since the suspension pitch isn't changing (which is great) that suggests that either the suspension is super well balanced, or it is just the top springs at the ISI table that are sagging. The temperature of these springs likely follows the temperature of the ISI much more closely than the ones lower down.
Just because...
While watching the IFO come back up, it was noticed that the live trend of ISS Diffracted Power was wiggling alot like the live trends of POP_A and IMC_TRANS. Attached is a trend for the curious.
J. Kissel, B. Weaver I'm the curious. I did a bunch more starring at these channels, trying to come up with a theory for all of this ETMX oscillation business. My best theory at this point: this wiggling is a mixing of tidal + length to angle coupling of the QUADs against power fluctuations arising from the ISS 2nd loop's feedback to the 1st loop, which is then fed back to the AOM's signal that we stare at for drift. Check out my attachment. All of the channels in the attachment, representing DHARD, Tidal, the IMC, the ISS... are all wiggling at roughly the same frequency. Here're some things I've notice while staring: (1) Notice that the Tidal Error signal (as shown by H1:IMC-F_OUT16 in DARK Green) is perfectly anti-correlated with the DHARD Pitch control signal (H1:ASC-DHARD_P_OUT16 in Purple). This is no surprise to me because I know that Tidal is fed to the UIM stages of both QUADs, which are known to have plenty of UIM-length-to-TST-angle imperfections. You could also claim some correlation between the DHARD yaw control signal (H1:ASC-DHARD_Y_OUT16) as well, but it's not so obvious, so I segregated it. (2) Along the bottom are the DC component of 1st / Inner loop ISS PDs (H1:PSL-ISS_PD[A/B]_CALI_DC_OUT16), these lag just a smidge (5-10 [sec]), but perfectly behind the IMC TRANS PD (H1:IMC-TRANS_OUT16). This PD is watching the same light the 2nd ISS array is watching, whose control signal is fed back to the -- you guessed it -- the error point of 1st loop of the ISS, whose actuator -- you guessed it -- is the AOM. The same AOM who's diffracted power we monitor for drift (which is why Betsy had it up). When you actuate on the diffracted power, you're directly changing the input power into the IFO. Not so stable at low frequency, this intensity stabilization loop... (3) The cavity powers (PRC and ARM, as measured by H1:LSC-POP_A_LF_OUTPUT and H1:LSC-TR_[X/Y]_NORM_INMON, respectively), also show the same "oscillation" as the ISS Inner Loop PDs, but they're look like a mix between the IMC TRANS / ISS Inner Loop PDs's fluctuations (i.e. input power fluctions into the PRC) and the Tidal / DHARD Pitch fluctuations (i.e. arm power fluctuations because of inadvertent pitch from tidal control). Some times they follow one, other times they follow the other. I suspect this is why the POP_LF trace (shown up on the front wall in bright yellow, so very obvious) seems to show slow oscillation, with and even slower envelope. I have a feeling that when you add a gain peaky-very-low-frequency blend filter into the equation (ETMX, when in the 45 [mHz] blends), it exacerbates the whole problem. (Note, I attach the correct configuration of the BSC ISIs's blend filters) Hurumph! It seems like the source of all of these symptoms appears to be the IMC -- both tidal control and PRC input power fluctuations. Maybe we should take a serious look at the low-frequency performance of the IMC? My deepest darkest fear is that it all comes from the HAM3 0.6 [Hz] problems, but I think that's too fast this issue. Anyways as I've claimed all along, I think at least these sets of traces convince me that this is a particularly complex interaction of *many* plants and anima*ahem* control loops. I think we might be at the point in LIGO's life time that we can't keep getting away with just assuming that everything below 100 [mHz] won't interact with each other, and I expect this will only get worse with higher power. Since we'll shortly have the oppurtunity to play again, I suggest we try a few things and see if it helps: (1) Change / improve the isolation configuration on HAM2/HAM3 (2) Try improving the UIM L to P (3) Change the UGF of the DHARD loops (4) Rejigger the ISS loop (or at least characterize it better) such that the low-frequency loop shape is different. P.S. The template for this StripTool lives in /ligo/home/jeffrey.kissel/2016-01-03/PowerFluctionations.stp Betsy's pulling up DTT on this, check for coherence, but just by looking at the time-series, I'd have a tough time mathematically proving that this is a linear coherence. We'll post if we get any clues there.
Note the coherence of many things with the HAM3 0.6Hz... (spectra taken from this lock stretch).
Dear DETCHAR - Since we can't seem to get the data from 10 days ago in our own controlroom, can you please look into coherence with IFO channels during that time when the 0.6Hz SEI HAM3 peak wasn't around?
Thanks-
Betsy, Kissel
Sad that I have to tag SEI on this... Perhaps in the O1/O2 interim we need a concerted campaign of replacing electronics in HAM3.
The low frequency gain of the ISS should be large enough: in the normal configuration there is a low frequency boost and an integrator. However, in the past it was clear that the low frequency (below ~20 Hz) error signal was spolied by some additional noise, most likely scatetred light. Indeed, when the beam on the external table was not properly dumped, we saw a lot of increased noise due to back scattering.
So, it's also possible that the ISS loop has enough gain at low frequency, but the error signal is polluted by some scattered light, which might be correlated with the motion of some suspended object.
Once the EQ band calmed back to normal levels, I did an initial alignment and brought us back to Observing at 2:50 UTC. I adjusted the ALS fiber polarization for both arms while we were waiting for the EQ to ring down.
After notifying the operator I drove past the CS @ 01:31 and arrived at Y-Mid station @ 01:35.
Started filling CP3 @ 01:36, opened the exhaust check valve bypass valve, then opened LLCV bypass valve 1/2 turn, started to notice flow 29 seconds later. however I closed the LLCV bypass valve after allowing flow for a total of 2 minutes.
6 minutes later I closed the exhaust check valve bypass valve.
Started driving back from Y-Mid @ 01:47, arrived at the CS @ 01:52.
Activity Log: All Times in UTC (PT) 00:00 (16:00) Take over from TJ 18:02 (10:02) High temp alarm on CS Zone 4 duct. Temperature hovering around 100F. Will monitor 23:24 (15:24) Lockloss – Mag 6.8 EQ in India. 23:38 (15:38) Put IFO in DOWN until EQ ring up passes 23:47 (15:47) Reset timing error on H1SUSETMY 00:00 (16:00) Turn over to Travis End of Shift Summary: Title: 01/03/2015, Day Shift 16:00 – 00:00 (08:00 – 16:00) All times in UTC (PT) Support: Vinny, TVo Incoming Operator: Travis Shift Detail Summary: Mag 6.8 EQ in India, R-Wave 9.6um/s, seismic a little over 2.0um/s. Put IFO in DOWN until seismic settles down. Otherwise, it was a good observing shift.
23:24 (15:24) Lockloss due to mag 6.8 EQ in India. Seismic rung up to over 2.0um/s.
Reset H1SUSETMY timing error.
IFO locked and observing for past 9 hours. Seismic and microseism are flat and mostly within normal range bounds. Microseism is near the top of the range bounds. Winds are up to a light breeze (up to 7mph), outside temps are 18F for all reporting stations. There have been several ETM-Y saturations, however the remains around 80Mpc.
It is currently snowing at the site, a dry light snow with about .25" accumulation. The roads and parking lot do not appear to be slippery. However visibility is somewhat limited.
Our cdsfs1 backup machine once again crashed, was not accessible via the network. I was able to remotely power cycle it via IPMI.
O1 day 107
model restarts logged for Sat 02/Jan/2016 No restarts reported
Transition Summary: Title: 01/3/2016, Day Shift 16:00 – 00:00 (08:00 – 16:00) All times in UTC (PT) State of H1: 16:00 (08:00), The IOF has been in Observing mode for last 5.5 hours. Range and power are normal. Environmental conditions are good. There have been several saturations during the last shift. Will clear the timing error on H1SUSETMY (CPU max at 63uS) at the next out of Observing opportunity. Outgoing Operator: TJ
Title: 1/3 OWL Shift: 08:00-16:00UTC (00:00-8:00PDT), all times posted in UTC
State of H1: Observing at 78Mpc for 5hrs
Shift Summary: One lockloss with some odd happenings just before. The POP power was degrading over the prior few hours and the control signals didn't look good (see previous alogs from me and Travis for more info). Had to play with green a bit and then the OMC to get things back up.
Incoming Operator: Jeff B
Activity Log:
Observing at 77Mpc for 3hrs.
Environment is calm, CW inj running, Timing error on H1SUSETMY
I had to fiddle with ALS for a bit, run through an initial alignment, change the PZT sweep starting voltage in omcparams.py again to -30 (see alog24609 for the last time),and then lost it at DC_READOUT. But it came right back up after and we are back to observing!
Looking at the spectrograms and glitchgrams on the summary pages for January 2nd there seems to be some interesting behaviour from about 10 - 35 Hz at LHO. The noise drops in this band over the course of the day and glitches in this band become less frequent as well. It does not seem to correlate to wind speed or ground motion, at least not in an obvious way. Looking at the ground motion BLRMS page of the summary pages, there do seem to be some spikes between 4 - 6 UTC, in multiple bands. However these spikes are relatively short-lived and the noise in the spectrograms starts before that. I've attached a few images showing noise and glitches from the band of interest.. The plots are also visible on the summary pages.
The last lock loss on 1 Jan was due to PSL issues (see alog). The low frequency noise reported here seems to be correlated with unusual noise in PSL ISS and also shows up in IMC frequency noise. The attached pdf shows apparently normal behavior on 1 Jan for PSL ISS, IMC frequency noise, and strain followed by excess noise in all three on 2 Jan. While things appear to have calmed down on 3 Jan (so far), there appears to be a PSL glitch that shows up in the strain channel.
Happy New Year all!
H1 has historically not has as bad of RF beat / whistle problems as L1 has. In fact, the last alog report is for data on October 30th. But dec 31st shows a high density of glitches above 100Hz and densest above 500Hz, which have the signature shape of RF beats we've seen before and are correlated with PRCL and SRCL, similar to one of the manifestations of whistles seen at LLO.
Note 1: we produce auto omega scans for the loudest glitches that hveto finds, and these whistles are all very quiet. If anyone wants to follow these up in more detail, you can get GPS times for the high-frequency low-snr triggers for Dec 31 correlations with PRCL and SRCL at those links.
Note 2: Dec 31 looks like the worst day, but there might have been some weak whistles occuring on other days too, but we'd have to follow up some low SNR triggers on those days (e.g. today Jan 1 and the past few days).
Quick update: RF beat / whistles are still happening today, Jan 3. The hveto page shows whistles in rounds 4 and 8 coincident this time with H1:ASC-REFL_B_RF45_Q_PIT_OUT_DQ (and not PRCL and SRCL as above, so a different line/VCO freq must be at play). They are still low SNR, but lightly visible in omega scans. Some example omega scans are attached and linked here. Text files of glitches coincident with ASC REFL B RF45 are here for round 4 and round 8.