Dave Barker created a shared 'hinj' account for running hardware injections (alog 19057), and we restarted tinj (the transient injection process) under that account yesterday.  Unfortunately, the injections failed overnight due to awgstream errors.  That was puzzling because executing awgstream on the command line (with zero amplitude or extremely small amplitude) worked fine.  It turns out that different versions of the gds, awgstream and root packages are installed on the LHO injection machine compared to the LLO injection machine, so the environment setup that was copied over from LLO caused awgstream to fail when executed by tinj.  I made a separate environment setup script for LHO and restarted tinj under that, and now it seems to be working fine.

After doing a zero-amplitude test injection at 1118063333 (which should have had no effect on anything), I modified the schedule to more promptly do som burst injections (coincident at both sites), at GPS times 1118063933, 1118067123, and 1118067543.  (The actual signal comes a second or two later than those waveform file start times.  As of this writing, the first of those was picked up by the burst pipelines: see https://gracedb.ligo.org/events/view/G159516 .  We'll see about the others.

Related:

Den's alog 16624

Den's alog 16727

Summary:

Many things are dubious.

1. EX LL ESD is broken as the ESD to length response is 3 orders of magnitude smaller than the others. Probably shoddy connection somewhere between the driver and the ESD electrode, as the voltage readback looks normal.
2. It looks as if either the sign of EX ESD output is flipped (positive digital out induces negative voltage) or the sign of EY ESD output is flipped but not both.
3. I'm just assuming that the sign convention for CAL-CS_DARM_EXTERNAL_DQ is length(X)-length(Y), and that it's correct, without any confirmation.

Despite these things, it seems as if the charges on the back are on the same order as reported in Den's alog 16624.

If we assume that the sign of EY ESD is wrong and we still take 1. into account, the charges are calculated as:

This looks semi-reasonable.

If we assume that the sign of EX ESD is wrong and we still take 1. into account, the charges are:

I don't like that the signs are all over the place.

If we assume that everything is correct except that the EX LL is broken (i.e. we ignore the 2. above but take 1. into account), the charges based on are:

Again the signs are all over the place.

These are based on the same calculation as Den's alog 16624.

I'm assuming that the sign convention of CAL-CS_DARM_EXTERNAL_DQ is length(X)-length(Y) (i.e. positive when X stretches and Y shrinks).

Anyway, no matter how you look at the data, the back surface charges are quite similar to what was reported in Den's alog (except for the signs that don't make much sense for the latter two tables).

We tried similar measurements as described in Den's alog 16727 but the angle data for X was unusable (no coherence at all). If you're interested in Y data, all measurements were saved in Betsy's template directory.

The gist of the measurements:

• Lock the IFO in DC using EX ESD for EY injections (or using EY ESD for EX injections). Use a correct-ish CAL-CS configuration.
• Ring heater upper and lower sections were bundled together and connected to one of GDS DACs via SR560 while the drive ground was connected to the ESD ground.
• Set the EY ESD bias to zero for EY injections (or EX bias for EX injections).
• Inject 5 lines (each one corresponding to LL, LR, UL, UR and ring heater), all at around 80Hz, and measure the transfer coefficients from injections to CAL-CS_DARM_EXTERNAL_DQ and IR QPDs on the TMS. CAL_CS thing was dewhitened using the calibration in the dtt template used for the control room FOM.

Differences between EX and EY measurements:

• Low noise driver, no low pass for EY. This gives a DC gain of 2, and a pole at 152Hz from PI summing node. For EX, high voltage driver with DC gain of 40 and a pole at 2kHz.
• High power for EY injections. Low power (3W) for EX injections because we needed to disable EX ring heater and I was worried if the PI becomes the problem if the measurement takes too long.
• In EY a large DC analog offset was observed in all GDS testpoints. To block this, SR560 was set to AC coupling which should have zero effect at 80Hz. (Later it turned out that the offset was already in the digital, so probably it's a digital offset problem but that doesn't change the conclusion either.)
• No meaningful coherence for the QPDs in EX. Don't know why, maybe we really need high power.

Fishy sign of ESDs (Go to the floor and figure out):

EY ESD length drivealign matrix has a negative DC gain while the corresponding matrix for EX is positive even though the LSC DARM output matrix already takes care of the sign difference necessary for DARM control for EX and EY.

It looks as if either the bias line has a wrong sign for one ETM but not the other, or LL/LR/UL/UR lines have a wrong sign for one ETM but not the other.

Raw-ish data and calculations:

Measured the zero-bias transfer coefficients from ESD segments and the ring heater (top and bottom combined) to the calibrated DARM channel in meters/volts at around 80Hz. After taking the TF of the drivers and the DAC V/cts into account, they are:

Positive data is actually about 24 deg (Y arm) or 30 deg (X arm) delayed, while negative data is about 210 deg (X arm) delayed.

EX LL is not working. Coherence is very large, the voltage readback looks OK, but it has 3 orders of magnitude smaller response than the others. EX LL did not change much when the nominal EX ESD bias was put back on.

I multiplied the ESD combined data by 4/3 only for EX to take into account that the EX LL driver is not working.

Force to length transfer function at 80Hz is -1/M/(2*pi*80Hz)^2 = -1E-7[m/N] (negative as the phase is 180 degrees relative to DC).

Also, the above is the TF to DARM, which is supposed to be X length - Y length. In order to move to a sign convention where positive means that the ETM moves closer to ITM, the sign of the X data should be flipped.

Combining these, the above raw-ish data is converted to N/V as:

The signs of this table don't really make sense (positive ESD electrode potential should move ETMX and ETMY in the same direction if the charge has the same sign).

Anyway, from here, you solve Den's rough formula:

F_RH / V_RH = A_front Q_front + A_back Q_back

F_ESD / V_ESD = B_front Q_front + B_back Q_back

A_front = 1 / 0.20 [1/m] ; A_back = -1 / 0.04 [1/m] 

B_front = 1 / 0.20 [1/m] ; B_back = 1 / 0.04 [1/m]

~16:00 Locked IFO

~16:45 Winds pick up, lock loss

Many frustrating hours of lock losses at REFL_TRANS or there abouts

23:15 Lock IFO again.

When I took over from Jeff this afternoon, he had successfully damped down bounce and roll modes on various optics, so I was able to proceed with locking the IFO at 17:15. About 45 minutes after that, the DARM got glitchy and we noticed that the wind traces on the wall were all trending up. After 45 minutes we lost lock and I haven't been able to relock since. Winds seem to be slowly winding down, but it's not totally calm yet. Pretty consistenly the IFO will get o CARM reduction, then lose lock on REFL_TRANS. Attached plots show trends for the last 6 hours at EX and EY. We lost lock pretty much right as winds topped out at EY.

I've created 1 hour injection files intended for simultaneous injection at LHO/LLO. There is a single 3600 second text file on h1hwinj1 at this location:
h1hwinj1:/ligo/home/edward.daw/research/hardware_injections/dependencies/sources/virgo/NAPNEW/SCRIPTS/IsotropicSbGenerator/first_injframes/SB_H1_ER7_1hr.txt
This contains a single set of channel data for injection at the Hanford site. There is a corresponding one hour ascii text file at Livingston at this location:
l1hwinj1:/home/edward.daw/injections/SB_L1_ER7_1hr.txt
Both files were created using a single call to IsotropicSbGenerator.py on the l1hwinj machine as follows:
cd /ligo/home/edward.daw/research/hardware_injections/dependencies/sources/virgo/NAPNEW/SWIG/
./IsotropicSbGenerator.py -i IsotropicSbGenerator3.ini
The results were 2 frames, one containing the Hanford data, the other containing the Livingston data. The LLO file was copied by ftp
to l1hwinj1.ligo-la.caltech.edu. At each site, libframe and frgetvect were used to convert the frame data to ascii text.

The amplitudes of these injections are omega_GW=0.001 at 100Hz in each detector. This should be far more subtle than the previous injection, so no need (hopefully) for Jeff/Adam to rescale the amplitude this time.

Hope there is some simultaneous up time at the two sites to try this test; I will be delighted if this is possible, but understand of course if there isn't an opportunity.

Thanks for any help. I've left a copy of this entry on the LLO aLOG.

Following what was done recently at LLO, I have created a common hinj account on the LHO hardware injection machine h1hwinj1. To conform to security requirements, users cannot ssh into this machine using the common account, they must still use their own LIGO.ORG account. Once on the machine, they can set-user to access the hinj account. The plan is to run all injections (contuous and transient) using the hinj account.

Dave Barker. WP5269

I have put together a quick EPICS alarm handler for Gamma Ray Bursts and Supernovae (GRB/SN) which runs on the operator alarm machine. It is called grbsn.alhConfig (in svn under cds/h1/alarmfiles).

The GRB alert system (which runs on h1fescript0) polls the GraceDB database every 10 seconds looking for External events (Gamma Ray Bursts and Supernovae). If an event is detected, its information is written to EPICS records which are hosted on the FE model h1calcs.

To put together a quick alarm system for ER7, I am using the alarm fields of the H1:CAL-INJ_EXTTRIG_ALERT_TIME record which records the GPS time of the trigger. By setting the HIGH alarm field to the current GPS time plus one second, when the next event is recorded this PV goes into the MAJOR alarm state.

The grbsn.alhConfig file provides the operator with two buttons:

[G] button opens the guidance text. It refers to the alog entry  https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=19037

[P] button runs a script which resets the alarm level to the current event, turning off the current alarm

When a GRB/SN alarm is raised the operator should:

1. acknowledge the alarm to stop it beeping/flashing

2. press the [P] button to turn off the current alarm and prime the system for the next event

3. read the guidance to see if any further action is needed

Information on the current event can be obtained by running the script grb_latest_event_info.py. This gives you the event time as local time, so you can easily determine when the one hour stand-down time will expire.

(baffled control room crew)

After the most recent lock loss the IMC wasn't relocking and looked completely misaligned. The screenshot shows about 20 minutes of the previous lock, 30 minutes of confusion and 10 minutes on a new lock. The only dof which shows a signifcant change is the input pointing. Strange.

Observation Bit: Commissioning 

08:00 – Working on relocking
08:05 – Add 150ml water to PSL crystal chiller
08:11 – IFO Locked at LSC_FF
08:13 – Intent bit set to Undisturbed 
08:30 – Bean tube cleaning team working on X-Arm
09:11 – Set intent bit to Commissioning – Kieta running charging measurements at both end stations
09:33 – LLO ops reported a power hit this AM. Working on recovery
09:49 – Lockloss – Commissioning activities 
11:52 – Jim Blunt on site to meet with Richard
11:55 Dan – Making delivery to VPW with PU truck
11:59 – IFO Locked at LSC_FF
12:00 Keita – Running charging measurements – Intent bit Commissioning 
12:21 – Beam tube cleaning crew breaking for lunch
13:08 Daniel – Moving charge test rig from End-Y to End-X
13:45 – Beam tube cleaning crew working on X-Arm
14:52 Daniel – Back from End stations
15:00 Keita – Finished with charge measurements
15:03 - IFO locked at LSC_FF
15:05 - Set Intent bit to Undisturbed 
15:15 Bubba – Going to Mid-X to get some tubing
15:17 - Lockloss – Possible due to beam tube cleaning
15:42 - Beam tube cleaning crew finished for the day
16:29 - Turn over to Jim W. – Still working on relocking

We attempted to set up the IFO for some end station ESD measurements around 10am.  During some of the switch settings, the IFO lost lock.  Relocking is ongoing with help from commiss.  A suite of ESD/charge measurements have become increasingly more important, hence time has been allotted today.

00:00 IFO not locked. Jim said it lost lock twice at BOUNCE_VIOLIN_MODE_DAMPING. And lost lock again at ANALOG_CARM when I came.

          Knowing that it fails to grab lock repeatedly, I approach all the Guardian state with caution (wait until the spectrum becomes settle before requesting the next state)

01:13 Lock lost at REFL_TRANS. I brought everything down, readjust SR2, PR2, PR3, BS, and IM4. Ended up took me a while because I messed up the alignment.

         Couldn't get ASC-C_COMM_A_DEMOD_RFMON to reach 5 counts. I touched PR3 all I could. Any other optics I could touch to improve this?

2:18 4.7 Earthquake in Japan - no WD tripped.

4:00 Lock loss at LOCK_DRMI_1F

4:10 Trouble locking ALS-X

5:00 Kristina to the wood shop.

        The ifo still tries to lock DRMI.

5:10 Kristina back.

6:16 Still not grabing DRMI lock. Evan suggested bad BS alignment as AS air momentary locked at 10 mode. I couldn't get it to lock just by adjusting the BS so I redid the initial alignment. The earthquake earlier might have thrown the optics off.

6:50 Lock loss at REFL_TRANS.

7:19 Lock loss at REFL_TRANS AGAIN. I waited until AS90 and POP18 became stable before requesting the state.

7:31 Stopping at DRMI_LOCK. Waiting for the 6.0 earthquake in Chile to pass (it didn't affect us - no WD tripped at least).

8:06 Hugh came to find ITMY Optlev damping has been turned off. This could have been the cause of repeated lock losses through out the night. Next time I'll be sure to check SDF before even touching the interferometer....

       Evan touched PRM to bring up POP18 (and AS90) before requesting RESONANCE. Maybe helped with the lock loss at REFL_TRANS?

       The ifo is now locking. Handling the ifo off to Jeff B.

Add 150ml water to the PSL Crystal chiller. 

Operators and commissioners were reminded to check the SDF file for changes. There were problems with locking last night due in part to the ITMY OpLev. This error was apparent when looking at the SDF overview.

All were reminded to aLOG, aLOG, and aLOG. The aLOG is a valuable repository for documenting the many things that happen during a day. When in doubt aLOG it.
 
The kitchen sink has been fixed. The Facilities folks asked that food scraps not be put into the sink, as they cause plugged pipes. Food scraps should be put into the compost bins or at the least put in the garbage, but not down the sink drain.

Repairs to the RO system are complete. There is no potable water in the staging building due to a water leak. 

Beam tube cleaning continues.

No safety meeting for today.

Charge measurements will be taken today at both end stations.  

We are seeing two issues with the autocal of the 18bit DAC cards (used almost exclusively by suspension models). The first is a failure of the autocal; the second is the autocal succeeding but taking longer than normal.

There are three calibration completion times: ~5.1S for unmodified DAC, ~5.3S for modified DAC, ~6.5S for modified long-running DAC

h1sus2b, h1sush34, h1susex, h1susey all have no DAC issues

h1sush2a: the third DAC is taking 6.5S to complete. This DAC is shared between PRM and PR3 models. First two channels are PRM M1 Right and Side. Last six channels are PR3 M1 T1,T2,T3,Left,Right,Side.

h1sush56: this has unmodified cards. 4th DAC is failing autocal

h1susb123: This one is strange:

On first autocal run after a computer reboot: 7th DAC failed autocal

On first restart of all models: 1st DAC failed autocal, 7th DAC succeeded

On second restart of all models: 1st DAC failed autocal, all others good (consistent restart behavior)

In all cases, 5th DAC card is running long for autocal (6.57S).

In the current situation with the 7th card now succeeding and the 1st DAC failing, the 1st DAC is driving ITMY M0 (F1,F2,F3,LF,RT,SD) and M0 (LF,RT)