Adam, Alex U., Chris, Mike L., Eric

With a bit of effort we have successfully scheduled and injected a coherent CBC hardware injection. We have scheduled three coherent hardware injections for later tonight at 1127179817, 1127181617, and 1127183417. Those injections are the last of the night.

Here I will summarize what has happened.

All attempted/successful injections used the H1L1 coherent waveform from aLog 21838.

The are a couple differences between the sites. LLO uses monit to run tinj, whereas LHO does not. LLO uses the new hardware injection SVN, whereas LHO uses the DAC SVN still. The directory to run tinj at both sites is /home/hinj/Details/tinj.

Note that the times below are not the end time of the inspiral but rather this is the start time of injecting the ASCII file. The end time of the inspiral is listed on the gracedb page for the hardware injection and is ~6.986 seconds after the injection was scheduled.

Adam and Chris set out to do a H1L1 coherent hardware injection. See 21901.

We initially tried to scheduled a hardware injection at 1127175848. However the first step, i.e. to upload a GraceDB entry did not work. Chris was given a permissions warning and denied to upload to the CBC group.

The second attempt was scheduled for 1127168901. This was unsuccessful.
  * A gracedb page was generated for this hardware injection H1 and L1. No explanation for why Chris had permission this time.
  * At LLO the wrong schedule file was updated so there was no injection at LLO.
  * At LHO tinj had stopped before the time of the injection so there was no injection at LHO.

We then stopped the tests and went to the hardware injection telecon.

On the hardware injection telecon we had a group debug session.

We had to get tinj running at LHO again. To do this we did:
cd /home/hinj/Details/tinj
./run_tinj &

After running this command tinj was running again with process ID 11229. We tailed the log file (/home/hinj/Details/tinj/tinj.log) and saw that it was updating again.


Even though tinj is running in the background it still prints output to stdout. So Chris logged out of h1hwinj1 and logged back into h1hwnj1. tinj was still running.

The thought for the tinj failing was that when Chris edited the schedule file (/home/hinj/Details/tinj/schedule) the MCR environment that was used for testing monit received an error. Eric had checked the run_tinj log (/home/hinj/Details/tinj/run_tinj.log); note run_tinj is a wrapper to run tinj in monit. The log said:
ERROR: Unexpected error in tinj.
GPS=1127167412

So we did another test with tinj. The tests were logged in 21911.

The third attempt was scheduled for 1127172657. This injection was unsuccessful.
  * tinj stopped. Debugging by Eric showed that we need to change the filename of the waveform single-column ASCII files. Inspiral waveform files need have _H1 or _L1 at the end. So we renamed coherenttest1_*.out to coherenttest1_*_H1.out and coherenttest1_*_H1.out.
  * We also had to include a _ at the end of the filename in the schedule. So the line should have read "1127172657 1 0.5 coherenttest1_1126257410".

We made these corrections. But LLO went out of lock.

The fourth attempt was scheduled for 1127173421 and it was H1 only. It was successful.
  * The waveform was injected and observed going through the hardware injection filterbanks.
  * It had a scale factor of 0.5 applied.
  * A gracedb page was generated for this hardware injection H1 and L1.

Now that we had tinj running and a successful injection at LHO we waited for LLO to come back.

The fifth attempt was scheduled for 1127175848 and was a H1L1 coherent injection into both detectors. It was successful.
  * This was the first scheduled coherent CBC injection in aLIGO.
  * It was observed in both detectors. There was a CWB trigger for this event in gracedb and it was flagged with the INJ tag.
  * A gracedb page was generated for this hardware injection H1 and L1.

We then scheduled three more injections for the night at 1127179817, 1127181617, and 1127183417.

The schedule file now reads:
1127168901 1 0.5 coherenttest1_1126257410
1127172657 1 0.5 coherenttest1_1126257410
1127173421 1 0.5 coherenttest1_1126257410_
1127175848 1 1.0 coherenttest1_1126257410_
1127179817 1 1.0 coherenttest1_1126257410_
1127181617 1 1.0 coherenttest1_1126257410_
1127183417 1 1.0 coherenttest1_1126257410_

More plots and followup to come later.

An action item is to update the hardware injection documentation with the correct filename format and the commands to restart tinj on the command line.

And I will add as another note, to get to the monit web browser interface. On a controls machine you can type for the URL "http://h1hwinj1/" and it will take you to the monit web browser interface. At the moment I see no way to manage tinj this way but once we switch to monit this will be a good thing to remember.

Chris Biwer is running tests of the hardware injection pipeline. When the injection went in, both the VerbalAlarms and GraceDB saw the injection properly. It wasn't clear from the verbal alarm that this was different from a GRB, but Grace made it clear pretty quickly that this was an injection.

This is a repetition of Valera's projection for how we expect the DARM spectrum to behave as we increase the laser power. The current (23 W) quantum noise estimate from GWINC was subtracted from the DARM spectrum and replaced with a set of higher-power quantum noise curves. For reference, I've also plotted the null stream and the sum/null residual. This is from 2015-09-12 06:30:00 Z.

This projection assumes, of course, that nothing else about the ISC changes. Also, I suspect that the range estimates are slightly higher than what we're used to because these spectra are median-averaged. Dan has already showed that our bucket noise is not Rayleigh distributed, so we expect the median spectral estimate to be lower than the mean estimate at these frequencies. I briefly compared a mean-estimated spectrum (from the frontend calibrated DARM channel, and from the same time) and got an inpsiral range of 75 Mpc for the 23 W spectrum.

Anyway, the message is that with our current correlated noises, we only win about 30 % more range with a factor of 5 more laser power.

We have tinj running again at LHO. We are going to test scheduling injections again.

I think it's possible that we're closer to the Newtonian gravitational noise limit than I had thought.  This is on the list of "things we knew were coming, but are perhaps here sooner than I thought they would be". 

The punch line is that we may be limited by Newtonian noise between 16-20 Hz.  Not a wide band, but reasonably consistent with the expectations from papers such as P1200017.

In the attached plot, the blue trace is the calibrated DARM spectrum (CAL-DELTAL_EXTERNAL_DQ) that we show on the wall, taken yesterday.  The green trace is my estimate of the Newtonian noise. 

For the Newtonian noise, I have taken the Z-axis STS-2 seismometer data from the sensors on the ground near each test mass.  (There is one seismometer at each end station, and one in the vertex near the ITMs - I use the same seismometer data for each ITM). The seismometers are in velocity units (I believe Jim said it's nm/s), so I pwelch to get velocity/rtHz, then apply the calibration zpk([],0, 1.6e-10) to get to meters/rtHz.  I then translate to acceleration due to Newtonian noise using eq 1 from T1100237.  Finaly, I add the 4 acceleration contributions (one from each test mass) incoherently and get to displacement by dividing the spectrum by (2*pi*f)^2. 

The Newtonian noise is touching the DARM spectrum between about 16 - 20 Hz. We're within about an order of magnitude in the band 10 - 30 Hz.  Evan will shortly re-run his noise budget code using this "measured" Newtonian noise to see if it helps explain some of the discrepancy between the measured and expected DARM spectra (this spectra is higher than the GWINC curve that is currently used in the noise budget). 

Notably, this estimate of seismically-induced Newtonian noise is somewhat larger than what we've quoted in P1200017 and T1100237.  If I use only the ETMY spectrum as an estimate for all 4 test masses, I get an answer more consistent with our past estimates.  However, using the actual seismic signals from each test mass, I'm getting this slightly higher estimate. 

The script to generate this plot is attached, as is the exported-from-DTT text file of the calibrated DARM spectrum.

JeffreyK, SudarshanK, DarkhanT,

We compared DARM model for ER8, O1 with canonical DARM loop TF measurement and DARM loop measurement from Sep 23 (LHO alog 21868). The purpose of this analysis is to see if DARM OLG TF changed from Sep 10 (canonical parameter set for O1) till Sep 23, and to see how DARM time-varying parameters were estimated by kappa correction factors. The comparison show that the overall DARM OLG TF difference between two measurements is ~ 1% in magnitude and < 1 deg.

κ_tst and κ_C corrected O1 models* resulted in:

• sensing function magnitude residual to be mostly flat at ~ 2% offset (that looks like mostly systematic); systematic phase discrepancy did not change;
• reduction of actuation function meas/model magnitude residual from 4% (Sep 10) and 5% (Sep 23) to mostly under 2%; systematic phase discrepancy did not change;

• total DARM OLG TF meas / model magnitude residual reduced from 4-5% (in range [100 .. 200] Hz) to < 2%; systematic phase discrepancy did not change, notice that phase discrepancy of the O1 model at the reference time is ~ 10 deg at high frequencies (1 kHz).

*kappas were informed by cal. lines within 2 hours from both of the TF measurements (Sep 10 and Sep 23)

On calibration telecon we discussed that we could possibly accout for systemtic phase discrepancy in O1 model by applying time advance in actuation and sensing functions.

Earlier, after resolving some of the O1 DARM model issues we did a similar analysis that did not include Sep 23 measurement results, for details on analysis see LHO alog 21827.

The script that compares the models was committed to calibration SVN:

CalSVN/aligocalibration/trunk/Runs/O1/H1/Scripts/DARMOLGTFs/CompareDARMOLGTFs_O1.m (r1533)

TITLE: Sept 24, 15:00-23:00UTC, 8:00-16:00PT

STATE Of H1: Observe, Range = 77Mpc, locked 3 hours, Observe 1 hour

SUPPORT: Hugh, Sheila

SHIFT SUMMARY: 

IFO locked as soon as ground motion decreased, however, another four earthquakes of magnitude 4+ arrived, and full lock in Low Noise took the first 4 hours of the shift.

Currently IFO is in Observe and injections are planned.

INCOMING OPERATOR: JimW

ACTIVITY LOG:

Earlier activity details logged in alogs 21900, 21894, 21893, and 21888.

15:00UTC - start of shift, IFO was aligned by Corey, locked with his alignment, only tweak I made was to ETMY TMS

15:47:12UTC - ground motion just coming down enough to relock, first DRMI lock since the earthquake in Canada

16:09:36UTC - IFO made it to Bounce Violin Mode Damping

16:19:37UTC - IFO unlocked

18:04:31UTC - IFO ready to lock, and this starts the locking sequence that resulted in the IFO going to Low Noise

18:55:04UTC - IFO in Low Noise

18:58:57UTC - IFO in Observe

19:26UTC - 12:26PT, truck at receiving

19:37:35UTC - Commissioning , Praxair on site, other activities

19:44UTC - Richard, Filiberto, Mid-Y

19:45:55UTC - engaged FM2 on DHARD_Y, Sheila's filter

21:43:53UTC - IFO back to Observe, lock has been continuous since 18:58:57UTC

23:00:00UTC - end of shift, plans being made for injections 

CURRENT ToDo List, when not locked:

- reload Guardian on SR2.  It is showing a filter change on it's Guardian, which does NOT prevent Observe mode, but indicates that Guardian needs to be reloaded, which should clear this up.

BTL logged some details about the Rogue Excitation Problem we suffer with HAM5 ISI.  The solution to this problem is fixing the coil driver monitor or unwiring this signal from the model.  We might get approve to fix this in one way or the other but until then...  We have to recover.

If you look at the model image Brian includes, you can see or not and I'll tell you anyway, if the Masterswitch is opened or the WD is fully (state4) tripped for more than 3 seconds, the Rogue Exc alarm will go off (because of the perceived excess voltage.)  This is why it always goes into Rogue alarm everytime the WD trips.  Also why it is a problem when we have opened the master switch.

So to untrip the platform, you have to untrip the watchdog, and untrip the rogue trip within 3 seconds.  If you manage to do this fast enough which isn't that bad, and the guardian wasn't the one to close the masterswitch, you could end up with an SPM diff showing on the guardian for HAM5 SEI.  Not that this is a big deal as we aren't keying on SPM and yes we could unmonitor the SEI but I like the monitoring.

So if the ISI is tripped:

1) check that the MASTER SWITCH is closed,

2) On the Watchdog page Click RESET ALL Rogue Exc WD RESET, and RESET ALL again, all within a couple seconds.

If it trips again, check the MASTERSWITCH, if it is open, it may be the Guardian, make sure the Guardian is set to Isolated and try the RESETs again.

Until we do the main fixes in line one, if these steps don't work, call me anytime.

If you end up with SPM diffs, and you care, you may be in for a platform cycle to OFFLINE (includes HEPI) so that the GUARDIAN closes the MASTERSWITCH.

Once OFFLINE (MASTERSWITCHs open,) set the guardina to isolated and do the above RESET dance.  I'm crossing my fingers so be patient.

Adam, Chris

We are beginning to test coherent hardware injection with tinj. Will update this aLog with schedule as they happen.