Over the last few days there have been strange glitch patterns that appear to be harmonics of some sort showing up on the summary pages. I've included a few plots showing this from today and yesterday. It's not particularly easy to read off the glitch frequencies sometimes from these plots so I analyzed spectra to try and find the actual frequencies of these glitches. The numbers below came from looking at peaks in a Strain ASD plot. These are estimates, and for a few of them it's hard to tell exactly which peak corresponds to the glitches.
Frequencies [Hz]:
46.1 56.8 70.1 80 113.75 144 (170.5 or 176) 208 256 (302.2 or 303.2)
I'd asked Vinny to check this out, and am to blame for calling them "harmonics." However, based on the frequencies he's quoted, they're not so harmonic. Attached is a plot of the frequencies and their frequency difference. No two omicron glitch "lines" are exactly harmonic as Vinny suggests. The first few are only roughly 10 [Hz] apart, the next few are 30 [Hz] apart, etc. My hope was that, if they were harmonic, it would point us to a fundamental frequency which might correspond to a known feature (e.g. the ~10 [Hz] QUAD highest vertical modes) that was modulating other noise at low frequency. Looks like no cigar. However, we've been having these "harmonic"-like omicron triggers for the past several days as Vinny shows, and they can reach several hundreds of Hertz. I'll caution folks to just dump this in the "RF45 noise" bucket: while we have seen the EOM driver's control servo show coherence with DARM during / surrounding / between when harmonic-like omicron triggers are happening (such episodes usually correspond with a much more gruesome, pattern-free, omicron glitchgram, and a noticeable decay in sensitivity / sensemon range), we've also seen the harmonic-like omicron triggers when there is no coherence between the EOM driver control servo and DARM. Since this (and the EOM RF frequency control servo) appear to come and go entirely randomly -- has anyone made a comparison with something equally random and unexpected, like ... the space weather?
03:20UTC - Not a test. Confirmed alarm with Tom Evans at Livingston. Jeff is going to be leaving the 4501.3Hz PCal line running. He says that it won't hinder data regarding this alert due to the frequency range difference. He has advised that Livingston maintain their's as well to achieve 6 hrs of data necessary to complete the test.
Though there is no active alarm, HVE-MY:CP3_LT200 is in alarm state. I didn't notice th alarm handler reporting this on my last shift despit the ongoing CP3 woes. Maybe it was?
Ed, the bar graph is displaying -7.9 and therefore not visible. I suspect this is why you don't see RED.
This is probably worth fixing.
john
at 00:39UTC i noticed that OMC REFL and TRANS cameras are producing NO images. Also, Refl centroid is at 0:0 and Trans is off scale -1 : -1.
an FRS has been filed.
TITLE: Jan 10 EVE Shift 00:00-08:00UTC (16:00-00:00 PDT), all times posted in UTC
STATE Of H1: Observing
OUTGOING OPERATOR: Corey
QUICK SUMMARY: IFO locked and Observing; Wind Calm: µSei trending below 90%ile;EQ bands nominal. A little funny business with periscope resonances and SRx bounce/roll eating at the range a little bit. Dr Kissel on site doing some work. Everything is good at this point.
TITLE: 1/10 DAY Shift: 16:00-00:00UTC (08:00-04:00PDT), all times posted in UTC
STATE of H1:
Incoming Operator: Ed
Support: Hugh (phone)
Quick Summary:
Other than the short lock at the beginning of the the shift; got back up fairly quick & have had it in Observing for 6+ hrs (Kissel would like 12hrs with the PCal lines where they are).
Shift Activities:
Not much to pass along here. Current lock is 5+hrs. The range has been showing a slight trend downward as of late (i.e. from 80.5 down to 77.5Mpc). Not seeing a reason for the drop (LSC POP A looks flat, and useism looks like it's actually dropping).
What a difference a lock makes.
After 54min, H1 dropped out of lock this morning. That lock had the TIdal DIAG_MAIN error messages noted earlier (but Tidal looked fine according to Hugh, whom I had on the phone). The one issue noted was H1:IMC-F_OUT16 diverging.
Since there were no obvious environmental/seismic reasons for the lockloss, I am blaming the IMC drift here. And with that, I am submitting my very first FRS Report (FRS#4199) with 0.8hrs Observation Time Lost.
NOTE: Terramon forecasts a 4.9 EQ from Turkey arriving here in about 20min (18:28UTC/10:28amPST), with motion of a YELLOW 0.15um/s....we'll see how we ride through it.
O1 days 111-114
model restarts logged for Sat 09/Jan/2016
No restarts reported
model restarts logged for Fri 08/Jan/2016
2016_01_08 08:29 h1nds0
Unexpected crash of h1nds0, required a restart
model restarts logged for Thu 07/Jan/2016
2016_01_07 14:40 h1nds0
Unexpected restart of h1nds0
model restarts logged for Wed 06/Jan/2016
No restarts reported
During the last lock segment, we had the Tidal Error messages (X& Y COMM CTRL within 10% of limit). Additionally, noticed on the Striptool on nuc1 that IMC-F_OUT16 was diverging & drifting off-screen (see attached). Ultimately there was a lockloss (could it be related?).
Talked to Hugh while we were locked, and he pointed me to the End Station HEPIs & also to the ISC signals they get. He mentioned they have a limit of 700,000counts. ETMy was moving below -36,000 (this was OK). EX was flatlined at 14,660 (this was ODD). But since both of these were well away from 700,000, we ruled out this being a tidal issue and figured it was something upstream (ISC? PSL?). We'll see how the next lock looks.
The attachment shows how the integrated ALS offsets remain as error point offsets for the IMC-F → UIM offloading.
We should bleed these ALS offsets away once we have transitioned off ALS, and preferrably before increasing the laser power (as this will load CARM by a few microns).
I thought that with EX tidal flatlined, the IMC frequency changes to try and keep H1 locked, but then runs out of range causing lock loss?
Wouldn't that mean the EX flatlined is the issue, IMC_F is just responding?
It seems that during this lock acquisition, there were large offsets remaining on the ALS → UIM offloading filter modules, and correspondingly large offsets on the IMC-F → UIM offloading filter modules (which are used during full lock).
The IMC-F → UIM offloading hit the limit around 15:50:00 Z, causing the tidal offloading to halt. As Cheryl said, this means IMC-F starts accumulating a dc offset to keep the laser on resonance.
TITLE: 01/10 [OWL Shift]: 08:00-16:00 UTC (00:00-08:00 PDT), all times posted in UTC STATE Of H1: Observing (at the last minute) SHIFT SUMMARY: Changed end X PCAL freq. Lost lock coincident with jump in earthquake band. Ran an initial alignment after seeing no flashes in DRMI. All the digital cameras froze for ~ 5 min. Had issues with OMC SUS (see previous alog). Engaged ISS second loop by hand. Getting diagnostic messages for X and Y tidal limits (see attached). A trend of the TIDAL_CTRL channels for the last 30 days does not show anything unusual (see attached). INCOMING OPERATOR: Corey ACTIVITY LOG: 14:41 UTC diode room dust monitor invalid 14:43 UTC all digital cameras frozen for ~ 5 min 15:55 UTC back to observing
TITLE: 1/10 DAY Shift: 16:00-00:00UTC (08:00-04:00PDT), all times posted in UTC
STATE of H1: Recently taken to OBSERVING by Patrick
Outgoing Operator: Patrick
Quick Summary:
Foggy morning on the site. Patrick restored H1 to Observing after the EQ (from 5.0 Antartica?). No winds and high useism (LVEA ~0.5um/s.
Have X & Y COMM CTRL within 10% of limit messages on DIAG_MAIN.
I'm stuck at DC readout transition with a message saying the OMC is not ready. I have tried rerequesting the OMC guardian state and changing the starting PZT voltage in omcparams.py. Something strange is going on. The SUS OMC WD is tripping and untripping itself. On the OMC SUS overview screen the IOP DACKILL is oscillating between red and green, but the numbers continue to go to the DAC regardless. LF and RT are stuck railed at -131060. ... Now it has stopped and LF and RT are no longer railed.
After rerequesting it one more time it worked. Moving on.
Coincident with jump in earthquake band (see attached). USGS: 2.8 37km SW of Ferndale, California 2016-01-10 13:05:09 UTC 20.4 km deep
Not sure if this is the correct earthquake. Corey says there may be a larger one in Antarctica scheduled to arrive around this time: 5.0 Pacific-Antarctic Ridge 2016-01-10 12:12:03 UTC 10.0 km deep
Have remained locked. Took out of observing briefly to change the end X PCAL frequency. H1:LSC-POP_A_LF_OUTPUT has been slowly falling and is now around 15,750. There is a slight indication that it may be leveling out (see attached), so I'll give it some more time to see.
Per Rick's request I changed the end X PCAL frequency from 3501.3 to 4501.3. I also changed the sine amplitude from 35,000 to 40,000. I went out of observing from 12:00:23 - 12:11:29 UTC to do this. I accepted the SDF differences from doing this (see attached). Per Rick's schedule this should be left to run for 12 hours.