Jamie, Dan, Dave:

Prior to the online guardian log lookback time being shortened to 2 months (https://dcc.ligo.org/LIGO-E1900337), yesterday evening I did a full rsync of the entire guardian log data set to LDAS (1.1TB) and today started a cronjob on h1guardian1 to perform an incremental rsync every 6 hours. This rsync runs at 7 minutes past the hour at 00, 06, 12 and 18 hours.

During the full backup yesterday I monitored h1guardian1's process loading and did not see any impact.

TITLE: 11/23 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 114Mpc
OUTGOING OPERATOR: Niko
CURRENT ENVIRONMENT:
    SEI_CONF state: WINDY
    Wind: 6mph Gusts, 5mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.59 μm/s 
QUICK SUMMARY: No issues.

TITLE: 11/23 Owl Shift 08:00 – 16:00 (00:00-08:00), all times posted in UTC

STATE of H1: Observing

INCOMING OPERATOR: Patrick

SHIFT SUMMARY: Sudden lockloss early in shift, re-locked and rode through small EQ. Lock and Observing ~5 hours.

LOG:

09:47 (01:47) Sudden lockloss, no warning

11:19 (03:19) At NLN, going into Observing

13:01 (05:01) Going to EQ mode for 6.1 near Indonesia

13:42 (05:42) Switching back to WINDY

Lost lock at 09:47 (01:47). Back in Observing, here are my re-locking notes:

Attempt #1

Intervening to lock green arms

• X: 0.2 flashes initially. Tried ETM’s, slight improvement, moving to TMS

  • ETMX Pitch sliders: 63.2 → 63.5

  • ETMX Yaw sliders: -131.5 → -131.725

  • TMSX Pitch sliders: -84.67 → -82.05

  • TMSX Yaw sliders: -50.57 → -46.22

• Y: No light initially. Tried ETM’s, up to 0.4 flashes. Moving to TMS

  • ETMY Pitch sliders: 118.6 → 128.75

  • ETMY Yaw sliders: -135.52 → -136.45

  • TMSY Pitch sliders: 86.847 → 91.06

  • TMSY Yaw sliders: -279.77 → -277.89

DRMI/PRMI ran automatically

CARM reduction states ran automatically

ENGAGE_ASC_FOR_FULL_IFO/ENGAGE_SOFT_LOOPS ran automatically

Lockloss at beginning of PREP_DC_READOUT_TRANSITION

Attempt #2

NLN. No intervention required

Ops Shift Transition: 11/23/2019, Owl Shift 00:08–16:00 (00:00-08:00) - UTC (PT)

State of H1: Locked

Intent Bit: Observing

Weather: 0-5 mph wind

Primary 0.03 – 0.1Hz: 0.01 um/s

Secondary 0.1 – 0.3Hz: 0.3 um/s

Outgoing Operator: Jeff

Quick Summary: Locked for 54 hours, Observing for 9 hours

   All quite on the observing front.

   The range is hovering around 116Mpc. 

   The evening thus far has been punctuated by several small to medium sized earthquakes in the Southwestern Pacific. None have caused any real issues. HAM5 remains tranquil. Microseism is elevated, but not trending upward. 

   No other concerns at this time.      

Today I checked whether REFL SERVO ERR is a trustworthy readback by comparing it to an analog spectrum from OUT2 while the IFO was in a single-sensor state (so that OUT2 == REFL SERVO ERR).

What matters is OUT2, since the CARM loop is fully analog for fast feedback.  For our projections, we use REFL SERVO ERR.  I was wondering if that is a valid thing to do.
Below is a comparison spectrum of the two.  Seems fine.

OUT2 is worse at low frequencies, this is likely because the SR785 noise floor was too high during the time of the meas.
REFL SERVO ERR has some peaks at 2 to 3 kHz.  Robert pointed out these are probably from aliasing down from the sampling frequency (16 kHz) and this huge peak forest at 14 kHz.  
In any case, they are not present in the actual loop.

Frequency noise projections using REFL SERVO ERR should be valid for all frequencies.

Chiara, Jenne.

Today we took some measurements of PRCL with LSC_DIFF on and off for PRM M1. The CPS_DIFF was always ON. A couple of things to take note of:

• To notice that, for PRM, the gain of LSC_DIFF must be with minus sign.
• The LSC filter (see block diagram in 53160), is built in this way:

LSCdiffFiltzpk = zpkgf(0,[0.05,0.05],1,0.005);

LSCboost=zpkgf([0.0878904+i.*0.0817023,0.0878904-i.*0.0817023],[0.0440496+i.*0.111623,0.0440496-i.*0.111623],1,1000);

LSCdiffFiltzpk = 2*LSCdiffFiltzpk*LSCboost;

The plot in attachment 1 shows the ratio between ON and OFF measured conditions (light blue) and expected conditions (black): probably we need a higher gain in the measurements.

Attachment 2 and 3 shows the crossovers between M1 and ISI, and M1 and M3, and the distribution filters. If you can notice something unusual in these plots around 0.7 Hz, the reason is that we did not change the M1 and M3 filters before applying  LSC_DIFF: probbaly, in future, we will change those filters to roll off the M1 filter more steeply.

Last attachment shows the difference between PRCL open loop gain when we apply ISI offloading (red trace) with respect to the PRCL open loop gain withount ISI offloading (blue).

FUTURE WORK:

• Investigate why DARM got worse (alog 53433)
• Increase the LSC offloading gain to try to match the simulations
• Try also SRCL

Cody Barschaw, Julian Carlin, Evan Goetz, Jack Heinzel, Sandra Hughey, Shivaraj Kandhasamy, Andrew Matas, Hannah Middleton, Lucas Mueller, Ansel Neunzert, Jay Tasson, Andres Vargas

Summary:
We have flagged and identified as many narrow spectral artifacts in the O3a averaged spectrum using ASDs of coherence length with 7200 seconds. From this flagged list, we have identified non-astrophysical lines and combs in a "vetted" list and maintain a list of unidentified lines. Overall, as reported in other aLOGs, the H1 data is much cleaner of narrow spectral artifacts than in O1 or O2. As always, nice work to commissioners and stay vigilant!

We highlight a few issues worth further investigation that impact both H1 and L1 spectra: 1) calibration line non-linearities; 2) apparent calibration line mixing with loud violin resonances; 3) weird non-physical "anti-line" features that dip below the shot noise sensitivity around certain violin resonances; 4) ASC dither lines are loud with significant sidebands. Item 3 may be due to mismatch in violin notch filtering or because of some issues damping the resonance, and item 4 is only an issue at H1.

See the linked files: Vetted list and Unidentifed list

Details:
Using standard CW tools in LALSuite, we generated 7200 s long SFTs from the C00 calibrated h(t) frames, only selecting observing times using H1:DMT-ANALYSIS_READY data from April 1 - Sept. 30. The SFTs were then averaged using noise weighting (a running median of the noise variance on a per-SFT basis) in order to generate an O3a average ASD. This provides rejection of high noise SFTs, fine frequency resolution, and a large number of averages in order to reveal narrow spectral artifacts that may remain hidden in a normal ASD.

Our team then divided the data for both H1 and L1 into broad frequency bands to flag all narrow spectral artifacts, attempting to identify combs and lines where possible. Those identified as non-astrophysical go into the vetted list, and everything else remains in the unidentified list.

Highlights:
- Calibration lines: 15.1, 15.6, 16.4, 16.7, 17.1, 17.6, 35.9, 36.7, 410.3, 1001.3, 1083.7, 1153.1, 1501.3 Hz
These are fairly obvious since they are on tenth Hz frequencies. Simple aLOG search was all that was needed. Calibration lines have been moved, but the run-average spectra see all of them. Also worth noting is that the 1153.1 Hz "cancelling" line is the same frequency as a roaming L1 line. Was this chosen on purpose? Generally we don't like to make coherent lines between detectors, so I'd advise moving this to a different frequency.

- Calibration line non-linearities occuring at 2f, f_1+f_2, or f_1-f_2: (27 lines identified in H1 so far)
Not as obvious, but has been identified at H1 as a problem (see LHO aLOG 48161) and also shows up at L1.

- Calibration line mixing with loud violin harmonics (at least 278 lines, see linked files above)
Identified at H1 (see see LHO aLOG 48161 and LHO aLOG 49281) and seen also at L1 by their spacing in concordance with the calibration line spacings (see, for example figure 1). It appears this problem is more of an issue with the violins are rung up, but they may also be present at a lower level even if the violins are not excited. Damping the violins and/or reducing calibration line amplitudes may help, but neither may be a good solution for a variety of reasons.

- Non-physical "anti-line" features in the spectra (13 features identified in H1)
These can be seen in spectra like in Figure 2 and are clearly dipping below the shot noise sensitivity of the detector around certain violin harmonics. It is not caused by the noise weighting of the SFTs; changing to a simple arithmetic average shows the same features. I speculate that this could be due to some kind of notch filter mismatch around the violin resonances or it could be due to a problem in the violin mode damping. Maybe there is a better idea for the cause. This may have escaped notice before because it's not seen in the summary page spectra or control room spectra. Other ideas?

- ASC dither lines: 18.37, 19.653, 20.131, 20.789, 21.9, 22.347 Hz
Only an issue in H1 since the L1 dither lines are all below 10 Hz and we don't yet see any issue with noise above 10 Hz caused by them. These lines are loud (some are larger in the h(t) ASD than the calibration lines) and have fairly substantial sidebands, see figure 3. It would be great if the SNR can be turned down on these lines and/or moved. I'm sure there is good reason for being where they are and as loud as they are. Maybe the commissioners could have a look at this.

Other details can be found on the main wiki page for organizing this effort, and recent presentation slides in the DCC.

We will have to check for any new lines in cleaned data. Additions or revisions can always be made in the git repo containing these lists.

TITLE: 11/23 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 115Mpc
INCOMING OPERATOR: Jeff
SHIFT SUMMARY: Remained locked all day, some commissioning and maintenance work done. We are still concerned about the HAM5 1Hz resonance.   
LOG:

• 16:39 Wind fence crew arrived. Picking up vehicles from EY then EX.
• 17:27 Timesh and Robert out to EX
• 17:39 Planned Commissioning started
  • SEI_CONF turned to WINDY_NO_BRSX as Timesh and Robert and entering EX
• 18:29 ISC_LOCK reloaded
• 18:37 Kyle out to visit MX and MY (back 20:20)
• 19:17 Timesh finshed in EX (finished NCAL noise investigations)
• 19:23 Robert finished in EX
  • SEI_CONF turned back to nominal WINDY
• 19.30 Observing
• 21:32 Kyle out to MY (back 21:45)
• 22:48 Commissioning / Maintenance 
  • Jim and Robert out to LVEA HAM5 to check HEPI for source of 1Hz resonance ringing up: see alog 53436
  • Craig using opportunity to go out  to LVEA PSL Racks for measurements
• 23:17 All back from LVEA, lights off, WAP off. Back to Observing

I noticed recently that there is a 1ish-hz peak on the HAM5. Digging through the summary pages it looks like it may have appeared around the 17th of Oct. It's not visible on the asds on the summary pages  that day, but if you look at the spectrogram, it shows up about 16:00 UTC. It's clearly there on the asds for the next day. 

Digging a bit deeper, it's on the HEPI and ST0 L4Cs as well, first attached plot. Worrying this might be like the HAM3 peak, I check the HAM2,3,4 & 5 HEPI L4Cs, and HAM3 HEPI doesn't show the ISIs 1 hz peak (second plot) , so I think it's a problem with the HEPI on HAM5, not the ISI. The HAM5 peak doesn't seem stationary, when I looked at midnight last night the peak was small on the HEPI L4Cs,  it was bigger a few hours ago, when Jenne and Chiara were trying PRCL to ISI offload, and now it's smaller again. 

I worry that something has changed the HEPI plant some. The 3rd attached plot shows the design plot for the X loop, the orange is the calculated open loop gain, and there is a 180 degree phase crossing at about 1hz, where the HEPI has gain of about 3. If something has reduced the gain of the plant, that could make this loop close to unstable. Most of the other loops are like this. 

If loop stability is the problem, it would be relatively straightforward to modify the boosts to get more phase margin at 1hz. I played around a bit in foton, the boost is just a pair of poles at .01hz and a pair of zeros at 1 hz, and some small tweaks were enough to get back 20 degrees of phase at 1 hz. Fourth attached plot, the dark blue line is the current filter, the pink(?) line is with at tweak to the boost. I haven't installed this.

If a chance comes up, or during a maintenance day, I'd like to get some open loop measurements on HAM5 HEPI. It would also be good to look at doing some other tests like linearity, range of motion or...

For the operators:

If HAM5 HEPI trips, please give me a call. Likely whatever has happened to HEPI has gotten worse, and it won't be stable. I can either come in and try to patch it up, or talk through the filter modifications over the phone. 

[Chiara, Jenne, Jim]

We did a first round of PRCL offloading to the ISI.  Chiara will post plots later of our filter designs and stability, but just quickly here I post a plot showing the effect on the IFO.

Following Chiara's convention, Pink is with offloading on and Green is off.  We offload some of the low frequency (but not the DC) of the PRCL control to the ISI rather than the M1 suspension stage.  We see clear improvement in the motion of the PRM (as witnessed by the OSEMs), and no effect in the IMC (as witnessed by the MC2 OSEMs).  Seeing no effect in the IMC is expected since we showed on Tuesday that when we push HAM2 and 3 together they really move together. 

However, we see a pretty significant degredation of the DARM low frequency signal.  We'll need to think more on this. 

We had the offloading on during commissioning time, but we turned it off before going to Observe.

UPDATE: Second attachment has a few more plots with same pink/green on/off definitions.  The left side plots are the same as the original first attachment.  The left side plots are PRCL, SRCL, and MICH controls.  It seems that turning on the PRCL offloading to the ISI impresses the 1.1 Hz line from PRCL onto SRCL and DARM, which is not so great.  Perhaps this line could be fixed by (Jim's suggestion of) swapping our CPS offloading between HAM2 and HAM3.  Right now, we're pushing on HAM3 to follow HAM2.  But, since HAM3 has the big 1.1 Hz line, perhaps we should push on HAM2 to follow HAM3.  This would make HAM2 move at 1.1 Hz, but would hopefully reduce the line's appearance in PRCL, and then offloading PRCL to the ISIs wouldn't put the line onto SRCL and DARM. 

In both sets of traces, the CPS_DIFF was on.  So, the only difference should have been the LSC PRCL offloading.  We do not plot any angular signals since they were corrupted due to end station sensor correction being off for other parasitic commissioning.

• H1's been locked over 41hrs!
• Went out observing this morning for planned commissioning (out for ~ 1hr50min)
• Jim noticed a ~1Hz HAM5 HEPI resonance has been ringing up.  There's a worry this could trip the HEPI/break H1 lock.
• It's a balmy 28degF at noon!

posting mostly because I've always wanted this.
click on traces in the legend to remove.
Good for removing all those LF traces to focus on the ones we care about.
If it's too big you can resize your browser.