Beam Manifold Ion Pump looks like it is railed at 10 at the beginning of the shift.
After the 20:20 lockloss, checked to see if I could get any where in locking, but H1 could not get past FIND_IR (never saw any X-arm locks). Why are X-Arm locks suddenly harder? Is it really due to all this seismic?
The EY BSCISI Location monitor had over 10000nm oscillations. When it died, I tried switching Y-axis to 90mHz (this brought oscillations down, but did not improve anything with locking, so I returned 45mHz Blend).
Started an Initial Alignment. ALS was sort of fine (Y-arm would break lock every few seconds/minutes). INPUT_ALIGN would not lock the X-ARM. At this point, I took locking DOWN to wait out the wind (will watch the End Station drifts on the 0.03-0.1Hz DMT Viewer to see when winds die down).
Ambient emperatures are up and it appears that the LVEA is warming quickly so I have dialed down one heater;
HC 2B has been lowered to 9ma (one stage) from 13ma (2 stages).
John feared an electrical glitch could have knocked H1 here, but his temperature change was around 20:13, a few min before H1 went down.
Winds have started to avg up around 20mph site-wide (gusts on X-arm up to 40mph). Might be dying down?
Range dropped down to 74Mpc with winds.
EY BSCISI Location monitor has had some big oscillations of +/-10,000nm over the last few minutes. SUS EY on the Tidal FOM (nuc1) has some clear sinusoids recently, too.
From 11-11:30amPST, Air Handler MAJOR alarm for LVEA 2B ("as usual"?)
Similarly to yesterday, H1 drops out around lunch time (20:20UTC). Winds were picking up & adding to our noisy useism.
Attached are trends of useism & wind from August to present for E8/O1.
MICROSEISM: Clearly, useism has been growing with the season. The worst storm was around 11/12 (but current one looks to be approaching that level, too).
WIND: What made 11/12 storm worse was that it ALSO had winds over 40mph! (For this current storm, winds have generally been under 10mph, BUT over the last few hours we had rain & now the winds are picking up to around 20mph (with an EX gust near 40mph).
Will continue to monitor (the H1 range is running great at 78Mpc and no ETMy saturations thus far *knock on wood*). And End Station ISIs look good with no ring-ups on the StripTool.
TITLE: 12/6 DAY Shift: 16:00-00:00UTC (08:00-16:00PDT), all times posted in UTC
STATE of H1: NLN at 78Mpc
Outgoing Operator: Jim
Quick Summary:
It's been 90min (chatted with Jim for first hour). Great work on helping H1 ride through these extremely rough seas of ground motion! Last night after reading Cheryl's alog about rung up ISI, I wanted to ask Jim:
What is a rung up ISI?
Watch End Station Location Monitors: I have a striptool up and I am watching H1:ISI-ETM[x & y]_ST1_CPS_[x & y]_LOCATIONMON. The y-axis is set to +/- 15,000nm. (signals currently about +/- 1000nm with not a very strong/visible sinusoid).
useism continues to be bad. Looks like it reached its worse during Cheryl's EVE and continues to stay elevated (LVEA has had some data points above 2um/s; easily averaging 1.1um/s.)
If we drop out of lock, Jim says I should NOT have to do anything with the ISIs (let Guardian do its thing). IF, I get a rung up EX, he says I could watch the EX Location Monitor, and wait for it to calm and then try enabling the X-direction 90mHz BLEND. (but that's only an iffy proposition)
Basically, we are in one of those unique periods when seismic is noisy enough to push us into this out-of-norm SEI state.
O1 day 80
model restarts logged for Sat 05/Dec/2015 No restarts reported
Title: 12/5 owl Shift 8:00-16:00 UTC
State of H1: NLN
Shift Summary:Rough night, EX ISI was rung up (microseism? IDK), took several hours to get locked
Activity log:
Cheryl had no luck relocking earlier because the EX ISI was misbehaving. I spent several hours poking at it, I'm not sure the things I tried had any effect, but it's possible we want to start engaging the last couple of control loops (St2 RX/RY) on this ISI. ISS 2nd loop engagement was also an issue.
It's been a rough one so far. ETMX ISI was rung up when I came in. I've tried a bunch of things, I'm not sure what or IF I've fixed it. Currently, the Z sensor correction is running on the ISI and the RY loop is running on St2. I've switched everything back to nominal, I'll keep watching the ISI, but for now everything looks OK. Shifting the Z sensor correction was what I thought earlier was the fix, but when I got almost to NLN, EX ISI rang up and blew the lock. I just gotten to NLN I've now turned on the St2 RY isolation loop, and the ISI may have settled down, but I've been here before. The current St2 RY blend is a 750mhz blend that injects a lot of GS13 noise at low frequency and CPS noise at high frequency, so it's not ideal, and I have a hard time believing that this configuration is really necessary for the ISI's stability but it's difficult to argue with qualified success.
Attached are a few spectra of the EX ISI St2 sensors. First plot is the RX (blue), RY(brown) GS13s, with GS13 noise curve(black). You can see turning on the loop makes the microseism better by a little, but worse at the blend cross-over and at a couple hz. Maybe the high microseism is whats giving us problems? Next plot are the CPSs red is RX, green is RY, I kept GS13 noise in for scale. Again, microseism is suppressed, but GS13 noise is making the low frequency worse by almost an order of magnitude, below 70mhz.
Turning on the St2 RY loop looked like it settled the ISI down, but I've backed everything out, and I think I can go back to observing. The EX ISI still needs monitoring.
It's not clear to me that trying to turn on the St2 RY loop was the fix here. It's entirely possible that I tried that just as the microseism receded below some threshold, or something. I don't recommend other operators try this if the ISI is rung up. Not yet, anyway.
Ops Eve Summary: 00:01-08:00UTC (16:00-23:59PT)
State of H1: down, ETMX ISI is oscillating
Help: Keita and Vern
Incoming Operator: JimW, who's looking into the ISI now
Shift Summary:
Currently:
As I was leaving I remembered an issue with ETMX guardian during my shift.
I ran INIT on the guardian for ETMX seismic, thinking it wouldn't do anything to the ISI, just refresh all the settings, so I would know that I was worknig with a good ISI guardian state.
What I got was guardian flashed, and then recovered, but in the process kicked up the ETMX oscillations.
I thought all INIT states were safe to run and not disturb hardware, but that's not how it turned out today.
According to alog 23420, "When useism is high, however, we have to use a 45mhz blends..." so I've changed all of them on ETMY and ETMX to 45mHz.
Status is that all BSC ISI's are on all 45mHz blend filters.
It appears that changing the blend filters rung up ETMX. I'm trying to get it back. I have the ISI in Isolated Damped, and manually changed GS13 fitlers and ST2 ISO filters and gains to engage ST2 ISO, which has worked, however, I'm at 0.01 gain for the loops when they are typically at 1 for ISO_X, Y, Z, and RZ. I did not engage ISO RX or RY.
6:44UTC - oscillations come back.
DRMI has locked for the second time. The first time was from 3:42-3:59UTC. Will attempt to progress the lock toward Low Noise.
two plots attached: Dec. 1st and Dec. 5th - all while H1 is locked in Low Noise
Dec. 1st plot shows BS optical lever sum glitching by 200+ counts.
Dec. 5th plot of the same signals shows ETMY optical lever sum glitching 50-70 counts, and ETMX optical lever sum gltiching 200+ counts, but BS sum is now quiet and +/-25 counts from the mean sum value.
Is this glitching, or an artifact of something else, and could this be effecting the IFO's locking?
Red herring... Variation in signal is a small percentage of the sum.
