Checked water levels and filters for the PSL Chillers. The water levels and filters are good. Closing FAMIS task.
For the past 45 minutes or so the RF45 noise has been elevated above the reference. The range is currently down into the upper 40s Mpc.
Recovered from earlier lockloss and back into Observing. The wind had been gusting up into the mid 30s, but has since laid down bringing down the microseism as well. After relocking the range has moved up to over 71Mpc. So far, after the relock the PIs have been behaving.
01:24UTC (18:24) Received GRB alert. Spoke with LLO. Both sites good. Stand-down for one hour
Upon entering the control room, I noticed that we have a low frequency oscillation in the PRC gain, POP DC, etc. on our front wall striptool. Turns out, this is the same as what Jim documented last night in alog 34999, and likely comes from Sheila's changing of the oplev damping cutoff in alog 34988.
I would like to go back to the old 10 Hz cutoff that we used to have in the oplevs, to see if that fixes this. However, Sheila noted to me that the old cutoff is zero history, no ramp, so we can't just go out of Observe and switch filters. I'll need to change the filters to always on, few sec ramp, load the filters, then we can switch back.
[Jenne, Sheila]
Hmmm. Reverting the cutoff frequency filters did not fix the problem. We're leaving them with the old 10Hz filters for now.
We also tried lowering the oplev gains by a factor of 8 on all test masses at the same time (in factor of 2 steps), but that caused CSOFT to start being unhappy, so we put them back to nominal before we lost lock.
We're going to go back to Observe since we don't have another idea right now of how to fix this.
As a note, it looks like this is upconverting with the 36Hz Cal lines to broaden them significantly, which is probably part of the reason we're at a reduced range right now.
Still no clever ideas, but it certainly seems like ITMY's oplev or something in the suspension is suspicious, and perhaps imposing noise through the ASC on the other optics. All 4 test masses see the 0.43 Hz oscillation in pitch, but ITMY seems to have many harmonics and other "hair" in the oplev spectrum. Not plotted, but ITMY yaw also sees the 0.43 Hz motion, but the other optics don't seem to see yaw problems.
EDIT to add spot position plot - spots on the test masses aren't moving very much over the last month, so it doesn't seem to be related to that. (Except ETMX yaw - still no idea why that's so scattered)
Beverly, Andy
On 22 Mar, a line of glitches at about 3.3 kHz appeared in apparent coincidence with these ASC problems. A known, always present line at the same frequency shows glitching at about 0.43 Hz (see attached spectrogram). A previous alog (27488) referred to this line as an SRM mode at 3335 Hz. In fact, the SRM spectrum (see attached spectrum) developed a feature at 0.43 Hz on 22 Mar that was not present on 21 Mar.
We haven't checked out anything related to Beverly's suggestion yet since the 0.43 Hz hasn't seemed to be a problem in the last ~day, and we've had other things to worry about. But, Sheila's 8Hz oplev cutoffs were in the guardian, so other than the lock that I had changed them back to 10Hz, they've been on the 8Hz cutoffs.
The DARM spectrum is noticeably better below about 25Hz, so it's good that Sheila's new oplev cutoffs aren't causing any problems, and are just helping.
22:08 Sudarshan, Suresh and Rick to EX
22:09 BRSX sensor correction turned off
22:21 Rick and company out
22:20 Robert out to LVEA to wrap up his testing so H1 can go back to Obeserving.
22:26 Spoke to Livingston. They Are back to Observing.
22:29 Robert out of LVEA. BRSX Sensor Correction turned back on.
22:32 Accepted some IM4 M1 P&Y diffs as per Sheila.
22:33 H1 back to Observing
16:00 Handing of to Jeff B
Nothing outstanding to report. Everything looks normal.
Agreed, everything looks normal.
22:33 65.3Mpc
Everything looks normal with these trends.
TITLE: 03/22 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Ed (for 1hour)
SHIFT SUMMARY: Locked this morning till commissioning at 11PST (18UTC) and at 20:30UTC we lost lock and have struggled to get it going again. Started out with the same issue I had yesterday with finding IR in ALS_DIFF. Running that state by hand very slowly seemed to work again, but not sure why that was happening. Now we are having some trouble with RF45. I'm passing off to Ed for an hour for the rest of my shift.
LOG:
Tragically, about 20 min ago the RF 45 started glitching. We were losing lock on Switch_to_QPDs twice before it started, so I think this isn't the #1 reason why we hadn't been locked for the first bit of time since I accidentally broke the lock.
But, while trying to look at why we might be struggling, I noticed that the RF45 is glitching and jumping around. It looks from the dataview plot attached like it started doing this right around 21:30:00 UTC.
Richard is in the CER trying to investigate, to see if there is anything obvious.
The old plan of waiting around for a while seems to have worked yet again. Mystery remains.
It would be interesting to see if there are any glitches in the observed LO drive monitors for this chassis coincident in time. There have been the occasional RF problem with the LO on these devices which would be specific to the demodulator card if it was an LO amplifier problem.
Tagging DetChar for further investigation.
Ran the full script for EX but had to cut EY short after about 35min. I did not get the chance to process the data yet, but I can do it tomorrow.
I had to tag Jeff on this one to run the long_trend.m because I had nds2 troubles. Jeff ran it with no problems...
He also mentioned that we may need to flip the bias soon on ETMX because (let's see if I understood this correctly) the angular actuation is off by ~8% in some quadrants, but the longitudinal is only off by ~3% (See summary screenshot).
Sorry abandoning TJ quickly on this one. Here's what I said in regards to these plots: Last week, I'd threatened that we needed to flip the ESD bias sign on ETMY because the traditionally posted angular relative actuation strength plots continue to show changes at the level of 8-10% because appreciable effective bias voltage from charge (negative 20-30 V) is accumulating in some quadrants. Note that this is relative to zero effective bias voltage. However, if I look at the summary pages, to show me the longitudinal relative actuation strength (the fifth plot that TJ posts) as measured by the PCAL and SUS calibration lines -- the actuation strength direction that actually matters for calibration -- it shows a level of 3-4%. Note that *this* is relative to the latest calibration reference model, created Jan 4th, when the effective bias voltage was already in the negative 10 V range. In summary -- I don't think we're in dire need of a bias flip on ETMY any longer. But we remain dilligent!
In the morning, let's replace the harmonic generator with a spare. If it still glitches it's probably not a glitchy harmonic generator.
In the past the harmonic generator was checked in the lab, but the "check" was for stationary noise and not the glitches (how can we "check" sporadic glitch that might or might not happen for a year?), and the same unit was put back in place.
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=21094