TITLE: 04/28 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: Travis
CURRENT ENVIRONMENT:
    Wind: 17mph Gusts, 7mph 5min avg
    Primary useism: 0.39 μm/s
    Secondary useism: 0.25 μm/s
QUICK SUMMARY: Earthquake hit little over 3 hours ago and we have been ringing down since. Some rotation stage work is going on in the mean time.

TITLE: 04/28 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: TJ
SHIFT SUMMARY: Relocked the IFO to DC Readout by 15:45 UTC.  At ~20:00 UTC, we got nailed by a 7.0 EQ in Vanuatu and have been down since.  Jim and Rich have been using the downtime to work on EY ISI, and various commissioners have been doing offline tasks as well.
LOG:

16:00 Jeff B in and out of cleaning bay all day

16:50 Fil to MY

16:30 Jim and Rich starting EY ISI work

17:36 Fil back

18:00 Chandra to GV7

18:00 Jeff B to both ends mech. rooms

18:33 Jeff B back

19:59 EQ hits, trips all ISI platforms and a few SUSes
 

I added 150 mL of H2O to the Crystal Chiller.

C. Cahillane

I have attached remade systematic error and uncertainty spectrograms for all of O1, as well as some a specific calibration at GPSTime = 1135136350.

These plots include uncertainty from detrended time-dependent kappas.

I have also included the uncertainty components plots for ease of viewing what contributes to uncertainty.

For LLO, see LLO aLOG 25914

Here are trends of the BRS raw balance position: a 12 day (right) and 30 days, left.  The driftmon has a range of +-16000 so at -13000 and still trending we don't have much room.  At the current rate, we'll reach the limit in about three days.

A test version of MEDM is currently installed in the control room to address CDS Bugzilla 789.  Although this wasn't really intended, the simple act of compiling the MEDM software installs it.  If it causes no harm, I'll leave it in place, if there's problems the original can be reinstalled in a few minutes.

This is to support testing of the historical MEDM screen software.

I restarted the HWSX sensor with a new template file that excludes the very high variance centroids from the measurement. This should result in a much less noisy measurement of the wavefront error.

The new Python version of the HWS code does this automatically by weighting each data point in the HWS image by the inverse of its variance when calculating beam properties.

Restarted web screen capture for h0 to pick up revised FMCS overview and fan sensor detail screens.  These now show all values since the new vacuum controls were installed.

One of the lockloss classes from tonight has been some kind of angular instability that we see very strongly on the AS camera right when we start to power up.  Not sure what it is, but it's the kind of thing that could be a BS angular instability, since the Michelson fringe separates, and the 2 bright spots orbit around each other for a second or so before we lose lock.  It looks primarily like pitch according to the oplevs, but there are certainly some yaw components in there.

Attached is a plot showing the lockloss, including the power at the AS port and the oplev (or wit) signals from each of the IFO optics.  If we call the frequency of the AS DC power fluctuations 2f, then it looks like perhaps BS and the ITMs are moving in pitch at 1f.

We tried once turning off the CHARD, MICH and SRC1 loops just before powering up.  We were able to make it to 3W and sit for a few moments, but then lost lock on the way to 5W, although this was a fast lockloss, probably just because we didn't have enough loops on.  We'd like to try powering up with all the loops on to 3W, to confirm that we can't even do that (we had always been going straight for 5W when we saw the instability). We'd also like to try with only 1 loop off at a time - MICH seems the most suspicious loop, but CHARD does seem to get noisy when the rotation stage is moving. 

As a side note, I've added tried to add a convergence checker to the Engage_SRC_ASC state, so that you no longer have to wait by hand before going to Part1.  You should be able to safely select Part1, and the guardian log will tell you what loop(s) aren't converged yet. The wait seems to not be working consistently yet, so maybe we can't depend on it yet, unfortunately.

Today we have seen at least three examples that might make us want to return to trying 90 MHz centering.  The first two screenshots attached show locklosses where we lost sideband buildups, which is usually a sign of misalignment of the BS or SRC.  (We are currently using a combination of AS36I A+B for SRM control 26785 and AS36BQ  for BS control).   You can see that the BS yaw control signal drifts along with the sideband powers, and that there is clearly also a signal in the 90MHz Yaw signals, which might indicate we could use these sensors to prevent this kind of run away and lockloss.  

The third one shows the lock that CHris and TJ left overnight last night, and you can also see sideband powers drifting a bit over the first  couple hours of the lock, which is also tracked by some signal in 90 MHz yaw.  

Jenne, Sheila, Chris, TJ, Evan

It seems that tonight we have been sabotaged by some code that we have been using for a long time (this haas only happened once that we caught it, although we have a lot of unexplained locklosses tonight).

In the attached screenshot you can see that the DRMI guardian was sitting at DRMI_3F_LOCKED (130) when it decided to go to LOCK_DRMI_1F (30).  There is a decorator in DRMI_3F_LOCKED that apparently returned LOCK_DRMI_1F, because it though DRMI was unlocked (it was fine as you can see from the power build ups in the top row). 

The code that checks for DRMI lock is: 

Made measurements of the oscillator power noise with a few photodiodes.

PowerNoise.png shows the free-running oscillator relative power noise
measured before the acousto-optic modulator.  This is more than 10 times
noisier than when the laser was installed in the H1 enclosure.  The
other trace in the plot is the out of loop of the relative power noise.
It is also about a factor of 10 higher than it should be.

    Whilst the power stabilisation was locked, I looked at the AC coupled
output of the photodiode and did not observe any oscillations.  The maximum
peak-to-peak variations were ~40 mVpp.

The X1 PT140 Pirani gauge is reading above the software interlock threshold to turn on the Cold Cathode gauge. Per Chandra's request I have bypassed the software interlock by forcing the variables in Beckhoff on h0velx (see attached screenshot).

Chris B., Jamie R.

The hardware injection guardian node has been setup at LHO.  The node should be ready to perform injections for the engineering run. Many thanks to Jamie.

The node is called INJ_TRANS. I have paused it.

Code is in: /opt/rtcds/userapps/release/cal/common/guardian

States that can be requested

A graph of the guardian states is attached. There are two states that can be requested:
  * INJECT_SUCCESS: Request this when you want to do injections
  * INJECT_KILL: Request this to cancel an injection

You should request INJECT_SUCCESS to perform an injection. The node will move to the WAIT_FOR_NEXT_INJECT will continuously check for an injection that are going to happen in the next five minutes (so if there are no injections for a long time, the node will spend a long time in this state). Once an injection is soon, it uploads an event to gracedb, reads the waveform data, and waits to inject. Eventually it will move into the injection state and inject the waveform. It will move back to the WAIT_FOR_NEXT_INJECT state and begin waiting for the next injection.

While the node is preparing to do an injection, eg. gracedb upload, etc., there will be a USERMSG letting the operator know an injection is about to occur. See MEDM screen below.

How to schedule an injection

This is just some short hand notes for how to schedule an injection with the guardian node until a document is in the DCC.

There are three steps:
  (1) Update the schedule file and validate it
  (2) Reload the guardian node
  (3) Request INJECT_SUCCESS if its not already

The current schedule file at the time of writing is located here: https://redoubt.ligo-wa.caltech.edu/svn/cds_user_apps/trunk/cal/common/guardian/schedule/schedule_1148558052.txt

The location of the schedule file is defined in https://redoubt.ligo-wa.caltech.edu/svn/cds_user_apps/trunk/cal/common/guardian/INJ_TRANS.py, search for the variable schedule_path.

An example line is:
1145685602 INJECT_DETCHAR_ACTIVE 0 1.0 /ligo/home/christopher.biwer/projects/guardian_hwinj/test_waveforms/box_test.txt None

Where:
  * First column is GPS start time of the injection.
  * Second column is the name of the guardian state that will perform the injection. Choices are INJECT_CBC_ACTIVE, INJECTION_BURST_ACTIVE, INJECT_DETCHAR_ACTIVE, and INJECT_STOCHASTIC_ACTIVE.
  * Third column says whither you want to do the injection in observing mode. If this is 1, then do the injection only if the IFO is in observing mode. Otherwise set this to 0.
  * The fourth column is the scale factor. This is a float that is multiplied with the timeseries. For example, 2.0 makes the waveform's amplitude twice as large and 0.5 makes the waveform's amplitude twice as small.
  * The fifth column is the path to the waveform file. Please use full paths.
  * The sixth column is the path to the meta-data file. Please use full paths. If there is no meta-data file, then type None.

Do not schedule injections closer than 300 seconds apart. If you want to do schedule injections closer than 300 seconds, then you will want to tune imminent_seconds in INJ_TRANS.py.

You should validate the schedule file. To run the script on a LHO work stations do:
PYTHONPATH=/opt/rtcds/userapps/release/cal/common/guardian/:${PYTHONPATH}
python /opt/rtcds/userapps/release/cal/common/scripts/guardian_inj_schedule_validation.py --schedule /opt/rtcds/userapps/release/cal/common/guardian/schedule/schedule_1148558052.txt --min-cadence 300

Note you need the glue and gracedb python packages to run this script - currently an FRS to get this installed.

Failure states

There are a number of failure states, eg. waveform file cannot be read, etc. If you validate the schedule the node shouldn't run into any failures. If a failure state is entered, the node will not leave it on its own. To leave a failure state identify the problem, resolve the problem, request INJECT_SUCCESS, and reload the node. Places where a failure could occur will print a traceback in the guardian log.

GraceDB authentication

I write this for anyone not familiar with the process.

Running this guardian node will require a robot certificate because the node will upload events to GraceDB automatically. To get a robot certificate follow the instructions at https://wiki.ligo.org/viewauth/AuthProject/LIGOCARobotCertificate.

We created a robot certificate for the controls account at LHO for the h1guardian0 machine.

We had to ask the GraceDB admins (Alex P.) to add the subject line from the cert to the grid-map file.

In the hardware injection guardian node env we set X509_USER_CERT to the file path of the cert and X509_USER_KEY to the file path of the public key.

Tested gracedb API with: gracedb ping.

Successful injections on GraceDB

Injections on GraceDB are given the INJ label if they are successful. There is a success message also printed in the GraceDB event page, with the line from the schedule file. For example H236068.

Test injections

At the end of the night I did a 2 hour series of CBC injections separated by 400 seconds, I've attached plots of those injections as sanity checks that everything looks alright.

Chris, Keita, Evan

Today we were able to lock the outer ISS loop with the modecleaner at 20 W (and no interferometer). We looked at several PSL/IOO PD signals (the FSS transmission PD, the ISS inner-loop PDs, the IM4 transmission PD, and the ISS outer-loop PDs) and tried to understand their behavior in different ISS configurations.

Naively one would expect all these signals (except the in-loop ISS PDs) to agree with each other, since they should all be out-of-loop sensors for the RIN leaving the PMC. Together, these signals monitor three of the four PMC ports: the FSS transmission sees the RIN of one port, the out-of-loop inner-loop ISS PD sees the RIN of another port, and IM4 trans and the out-of-loop outer-loop ISS PD sees the RIN of yet another port.

These are the behaviors we observed (see attached pdf):

• When neither ISS loop is closed, both the FSS and IM4 transmission PDs see the noise of the HPO (about 1×10⁻³/Hz^1/2 at 10 Hz). From 10 to 100 Hz, these PDs are coherent with each other and with the HPO PD. All four of the ISS PDs are swinging rail-to-rail in this configuration, so they are not useful. [Apricot traces.]
• When the inner ISS loop is closed, the FSS PD, IM4 PD, and outer-loop ISS PDs see a RIN that is a few times 10⁻⁵/Hz^1/2 at 10 Hz, with a 1/f slope. These three signals are coherent with each other between 10 and 100 Hz. However, the inner-loop PDs are in some parallel universe in which the inner-loop appears to be happily stabilizing the laser RIN to better than 10⁻⁶/Hz^1/2 at 10 Hz. The inner-loop PD signals have low coherence with the FSS, IM4, or outer-loop PDs between 10 and 100 Hz. [Red and blue traces.]
• When the inner and outer ISS loops are closed, the outer-loop PDs seem to behave sensibly. The out-of-loop RIN is slightly better than 10⁻⁷/Hz^1/2 at 10 Hz. The IM4 PD more or less agrees. However, the FSS RIN seems to show no change from the previous measurement (inner ISS loop only), and no longer has very much coherence with the IM4 and outer-loop PDs. The FSS has some coherence above 100 Hz with the outer-loop PDs. The inner-loop PDs now see the 1/f RIN of the FSS PD, and they are coherent with this PD. [Purple and green traces.]

We think that a possible explanation for these effects is that both ISS PDs are seeing some correlated noise that is not seen by either the FSS PD or the post-IMC PDs. In this scenario, the inner-loop ISS would suppress the HPO noise but impress this correlated noise on the light entering the PMC.

Briefly we entertained the idea that the light circulating in the PMC could be multimoded (either from the NPRO or the HPO), but judging from the RIN before and after the IMC, this seems to not be the case (png attachment).

One other idea is that some of the 808 nm light is getting through the PMC and onto the ISS.