TITLE: 03/23 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 0Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY: Rough shift, RF45 was bad, alignment looks bad now.
LOG:
RF45 was terrible all night. Richard came in this morning and replaced the chassis (~14:00 UTC), but I couldn't get DRMI after the swap. I started IA at about 14:30, but DARK MICH FRINGES wouldn't work. I tried to move on, but DRMI still looks hopeless.
TITLE: 03/23 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 0Mpc
OUTGOING OPERATOR: Jim
CURRENT ENVIRONMENT:
Wind: 5mph Gusts, 3mph 5min avg
Primary useism: 0.01 μm/s
Secondary useism: 0.19 μm/s
QUICK SUMMARY: Jim had issues with RF45 all night, but Richard swapped out the chassis this morning and we are trying to lock DRMI now after and initial alignment.
After a night of wasted science, (45Mhz Glitches) the Harmonic Frequency generator was swapped. Hopefully this was the culprit causing the glitches in the RF45. Serial number S1000796 was replaced with S1000798 Work Permit 6537
Checked water levels and filters for the PSL Chillers. The water levels and filters are good. Closing FAMIS task.
For the past 45 minutes or so the RF45 noise has been elevated above the reference. The range is currently down into the upper 40s Mpc.
Mike L. called to inquire about the RF45 issue. Per his suggestion, I've informed Keita of the situation. As there is nothing ops can do at this time, will continue to monitor.
In the morning, let's replace the harmonic generator with a spare. If it still glitches it's probably not a glitchy harmonic generator.
In the past the harmonic generator was checked in the lab, but the "check" was for stationary noise and not the glitches (how can we "check" sporadic glitch that might or might not happen for a year?), and the same unit was put back in place.
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=21094
Recovered from earlier lockloss and back into Observing. The wind had been gusting up into the mid 30s, but has since laid down bringing down the microseism as well. After relocking the range has moved up to over 71Mpc. So far, after the relock the PIs have been behaving.
01:24UTC (18:24) Received GRB alert. Spoke with LLO. Both sites good. Stand-down for one hour
Upon entering the control room, I noticed that we have a low frequency oscillation in the PRC gain, POP DC, etc. on our front wall striptool. Turns out, this is the same as what Jim documented last night in alog 34999, and likely comes from Sheila's changing of the oplev damping cutoff in alog 34988.
I would like to go back to the old 10 Hz cutoff that we used to have in the oplevs, to see if that fixes this. However, Sheila noted to me that the old cutoff is zero history, no ramp, so we can't just go out of Observe and switch filters. I'll need to change the filters to always on, few sec ramp, load the filters, then we can switch back.
[Jenne, Sheila]
Hmmm. Reverting the cutoff frequency filters did not fix the problem. We're leaving them with the old 10Hz filters for now.
We also tried lowering the oplev gains by a factor of 8 on all test masses at the same time (in factor of 2 steps), but that caused CSOFT to start being unhappy, so we put them back to nominal before we lost lock.
We're going to go back to Observe since we don't have another idea right now of how to fix this.
As a note, it looks like this is upconverting with the 36Hz Cal lines to broaden them significantly, which is probably part of the reason we're at a reduced range right now.
Still no clever ideas, but it certainly seems like ITMY's oplev or something in the suspension is suspicious, and perhaps imposing noise through the ASC on the other optics. All 4 test masses see the 0.43 Hz oscillation in pitch, but ITMY seems to have many harmonics and other "hair" in the oplev spectrum. Not plotted, but ITMY yaw also sees the 0.43 Hz motion, but the other optics don't seem to see yaw problems.
EDIT to add spot position plot - spots on the test masses aren't moving very much over the last month, so it doesn't seem to be related to that. (Except ETMX yaw - still no idea why that's so scattered)
Beverly, Andy
On 22 Mar, a line of glitches at about 3.3 kHz appeared in apparent coincidence with these ASC problems. A known, always present line at the same frequency shows glitching at about 0.43 Hz (see attached spectrogram). A previous alog (27488) referred to this line as an SRM mode at 3335 Hz. In fact, the SRM spectrum (see attached spectrum) developed a feature at 0.43 Hz on 22 Mar that was not present on 21 Mar.
We haven't checked out anything related to Beverly's suggestion yet since the 0.43 Hz hasn't seemed to be a problem in the last ~day, and we've had other things to worry about. But, Sheila's 8Hz oplev cutoffs were in the guardian, so other than the lock that I had changed them back to 10Hz, they've been on the 8Hz cutoffs.
The DARM spectrum is noticeably better below about 25Hz, so it's good that Sheila's new oplev cutoffs aren't causing any problems, and are just helping.
22:08 Sudarshan, Suresh and Rick to EX
22:09 BRSX sensor correction turned off
22:21 Rick and company out
22:20 Robert out to LVEA to wrap up his testing so H1 can go back to Obeserving.
22:26 Spoke to Livingston. They Are back to Observing.
22:29 Robert out of LVEA. BRSX Sensor Correction turned back on.
22:32 Accepted some IM4 M1 P&Y diffs as per Sheila.
22:33 H1 back to Observing
16:00 Handing of to Jeff B
Nothing outstanding to report. Everything looks normal.
Agreed, everything looks normal.
22:33 65.3Mpc
Everything looks normal with these trends.
Tragically, about 20 min ago the RF 45 started glitching. We were losing lock on Switch_to_QPDs twice before it started, so I think this isn't the #1 reason why we hadn't been locked for the first bit of time since I accidentally broke the lock.
But, while trying to look at why we might be struggling, I noticed that the RF45 is glitching and jumping around. It looks from the dataview plot attached like it started doing this right around 21:30:00 UTC.
Richard is in the CER trying to investigate, to see if there is anything obvious.
The old plan of waiting around for a while seems to have worked yet again. Mystery remains.
It would be interesting to see if there are any glitches in the observed LO drive monitors for this chassis coincident in time. There have been the occasional RF problem with the LO on these devices which would be specific to the demodulator card if it was an LO amplifier problem.
Tagging DetChar for further investigation.
RF demodulation phases for 2f, 3f, 4f, 10f and 15f might change with this swap.