When I took over from Jeff this afternoon, he had successfully damped down bounce and roll modes on various optics, so I was able to proceed with locking the IFO at 17:15. About 45 minutes after that, the DARM got glitchy and we noticed that the wind traces on the wall were all trending up. After 45 minutes we lost lock and I haven't been able to relock since. Winds seem to be slowly winding down, but it's not totally calm yet. Pretty consistenly the IFO will get o CARM reduction, then lose lock on REFL_TRANS. Attached plots show trends for the last 6 hours at EX and EY. We lost lock pretty much right as winds topped out at EY.

I've created 1 hour injection files intended for simultaneous injection at LHO/LLO. There is a single 3600 second text file on h1hwinj1 at this location:
h1hwinj1:/ligo/home/edward.daw/research/hardware_injections/dependencies/sources/virgo/NAPNEW/SCRIPTS/IsotropicSbGenerator/first_injframes/SB_H1_ER7_1hr.txt
This contains a single set of channel data for injection at the Hanford site. There is a corresponding one hour ascii text file at Livingston at this location:
l1hwinj1:/home/edward.daw/injections/SB_L1_ER7_1hr.txt
Both files were created using a single call to IsotropicSbGenerator.py on the l1hwinj machine as follows:
cd /ligo/home/edward.daw/research/hardware_injections/dependencies/sources/virgo/NAPNEW/SWIG/
./IsotropicSbGenerator.py -i IsotropicSbGenerator3.ini
The results were 2 frames, one containing the Hanford data, the other containing the Livingston data. The LLO file was copied by ftp
to l1hwinj1.ligo-la.caltech.edu. At each site, libframe and frgetvect were used to convert the frame data to ascii text.

The amplitudes of these injections are omega_GW=0.001 at 100Hz in each detector. This should be far more subtle than the previous injection, so no need (hopefully) for Jeff/Adam to rescale the amplitude this time.

Hope there is some simultaneous up time at the two sites to try this test; I will be delighted if this is possible, but understand of course if there isn't an opportunity.

Thanks for any help. I've left a copy of this entry on the LLO aLOG.

Following what was done recently at LLO, I have created a common hinj account on the LHO hardware injection machine h1hwinj1. To conform to security requirements, users cannot ssh into this machine using the common account, they must still use their own LIGO.ORG account. Once on the machine, they can set-user to access the hinj account. The plan is to run all injections (contuous and transient) using the hinj account.

Dave Barker. WP5269

I have put together a quick EPICS alarm handler for Gamma Ray Bursts and Supernovae (GRB/SN) which runs on the operator alarm machine. It is called grbsn.alhConfig (in svn under cds/h1/alarmfiles).

The GRB alert system (which runs on h1fescript0) polls the GraceDB database every 10 seconds looking for External events (Gamma Ray Bursts and Supernovae). If an event is detected, its information is written to EPICS records which are hosted on the FE model h1calcs.

To put together a quick alarm system for ER7, I am using the alarm fields of the H1:CAL-INJ_EXTTRIG_ALERT_TIME record which records the GPS time of the trigger. By setting the HIGH alarm field to the current GPS time plus one second, when the next event is recorded this PV goes into the MAJOR alarm state.

The grbsn.alhConfig file provides the operator with two buttons:

[G] button opens the guidance text. It refers to the alog entry  https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=19037

[P] button runs a script which resets the alarm level to the current event, turning off the current alarm

When a GRB/SN alarm is raised the operator should:

1. acknowledge the alarm to stop it beeping/flashing

2. press the [P] button to turn off the current alarm and prime the system for the next event

3. read the guidance to see if any further action is needed

Information on the current event can be obtained by running the script grb_latest_event_info.py. This gives you the event time as local time, so you can easily determine when the one hour stand-down time will expire.

(baffled control room crew)

After the most recent lock loss the IMC wasn't relocking and looked completely misaligned. The screenshot shows about 20 minutes of the previous lock, 30 minutes of confusion and 10 minutes on a new lock. The only dof which shows a signifcant change is the input pointing. Strange.

Observation Bit: Commissioning 

08:00 – Working on relocking
08:05 – Add 150ml water to PSL crystal chiller
08:11 – IFO Locked at LSC_FF
08:13 – Intent bit set to Undisturbed 
08:30 – Bean tube cleaning team working on X-Arm
09:11 – Set intent bit to Commissioning – Kieta running charging measurements at both end stations
09:33 – LLO ops reported a power hit this AM. Working on recovery
09:49 – Lockloss – Commissioning activities 
11:52 – Jim Blunt on site to meet with Richard
11:55 Dan – Making delivery to VPW with PU truck
11:59 – IFO Locked at LSC_FF
12:00 Keita – Running charging measurements – Intent bit Commissioning 
12:21 – Beam tube cleaning crew breaking for lunch
13:08 Daniel – Moving charge test rig from End-Y to End-X
13:45 – Beam tube cleaning crew working on X-Arm
14:52 Daniel – Back from End stations
15:00 Keita – Finished with charge measurements
15:03 - IFO locked at LSC_FF
15:05 - Set Intent bit to Undisturbed 
15:15 Bubba – Going to Mid-X to get some tubing
15:17 - Lockloss – Possible due to beam tube cleaning
15:42 - Beam tube cleaning crew finished for the day
16:29 - Turn over to Jim W. – Still working on relocking

We attempted to set up the IFO for some end station ESD measurements around 10am.  During some of the switch settings, the IFO lost lock.  Relocking is ongoing with help from commiss.  A suite of ESD/charge measurements have become increasingly more important, hence time has been allotted today.

00:00 IFO not locked. Jim said it lost lock twice at BOUNCE_VIOLIN_MODE_DAMPING. And lost lock again at ANALOG_CARM when I came.

          Knowing that it fails to grab lock repeatedly, I approach all the Guardian state with caution (wait until the spectrum becomes settle before requesting the next state)

01:13 Lock lost at REFL_TRANS. I brought everything down, readjust SR2, PR2, PR3, BS, and IM4. Ended up took me a while because I messed up the alignment.

         Couldn't get ASC-C_COMM_A_DEMOD_RFMON to reach 5 counts. I touched PR3 all I could. Any other optics I could touch to improve this?

2:18 4.7 Earthquake in Japan - no WD tripped.

4:00 Lock loss at LOCK_DRMI_1F

4:10 Trouble locking ALS-X

5:00 Kristina to the wood shop.

        The ifo still tries to lock DRMI.

5:10 Kristina back.

6:16 Still not grabing DRMI lock. Evan suggested bad BS alignment as AS air momentary locked at 10 mode. I couldn't get it to lock just by adjusting the BS so I redid the initial alignment. The earthquake earlier might have thrown the optics off.

6:50 Lock loss at REFL_TRANS.

7:19 Lock loss at REFL_TRANS AGAIN. I waited until AS90 and POP18 became stable before requesting the state.

7:31 Stopping at DRMI_LOCK. Waiting for the 6.0 earthquake in Chile to pass (it didn't affect us - no WD tripped at least).

8:06 Hugh came to find ITMY Optlev damping has been turned off. This could have been the cause of repeated lock losses through out the night. Next time I'll be sure to check SDF before even touching the interferometer....

       Evan touched PRM to bring up POP18 (and AS90) before requesting RESONANCE. Maybe helped with the lock loss at REFL_TRANS?

       The ifo is now locking. Handling the ifo off to Jeff B.

Add 150ml water to the PSL Crystal chiller. 

Operators and commissioners were reminded to check the SDF file for changes. There were problems with locking last night due in part to the ITMY OpLev. This error was apparent when looking at the SDF overview.

All were reminded to aLOG, aLOG, and aLOG. The aLOG is a valuable repository for documenting the many things that happen during a day. When in doubt aLOG it.
 
The kitchen sink has been fixed. The Facilities folks asked that food scraps not be put into the sink, as they cause plugged pipes. Food scraps should be put into the compost bins or at the least put in the garbage, but not down the sink drain.

Repairs to the RO system are complete. There is no potable water in the staging building due to a water leak. 

Beam tube cleaning continues.

No safety meeting for today.

Charge measurements will be taken today at both end stations.  

Apon arrival, I see that the IFO is back up but there are 2 red SDF channels:

One is a somewhat benign LIMIT on the BS OPLEV YAW bank.  The other is 2 new filters that are enabled on ITMY_M0_DARM_DAMP_R.  I *thought* I saw these being switched a bit via commissioners yesterday during the locking attempts that were taking a while yesterday.  There are no alogs about these being turned on.  This needs to be addressed by ops/commiss today.

OPERATORS:  The SDF and GUARDIAN overview screens are very useful at determining what is out of configuration when attempting to lock.  If the GRD overview screen is litup like a "holiday" tree on the SUS and SEI banks, things are wrong and likely inhibiting lock.  Likewise on the SDF overview screen.  If there are red indications of channel diffs, it might help locking to work through the diffs.  Yesterday we had numerous indications on both of these screens that things were not in good states to lock.

model restarts logged for Tue 09/Jun/2015
2015_06_09 10:26 h1iopsush2a
2015_06_09 10:26 h1susmc1
2015_06_09 10:26 h1susmc3
2015_06_09 10:28 h1suspr3
2015_06_09 10:28 h1susprm
2015_06_09 10:32 h1iopsush2b
2015_06_09 10:32 h1susim
2015_06_09 10:37 h1iopsush34
2015_06_09 10:37 h1susmc2
2015_06_09 10:37 h1suspr2
2015_06_09 10:37 h1sussr2
2015_06_09 10:48 h1broadcast0
2015_06_09 11:47 h1iopsusb123
2015_06_09 11:47 h1susbs
2015_06_09 11:47 h1susitmx
2015_06_09 11:47 h1susitmy
2015_06_09 17:25 h1fw1*
2015_06_09 19:16 h1fw1*
2015_06_09 21:03 h1fw1*

* = unexpected restart

Maintenance day. Restart of SUS IOP models for 18bit DAC recalibration. Reboot DMT broadcaster.

The last time it lost lock I tried my best to approach this state slowly. I have attached the screenshots of the spectrum and PRMI StripTool when it was at CARM_5PM. I waited until POP18 and AS90 became relatively stable before moved on and requested REFL_TRANS.  Didn't help...

I restarted transient injections at LHO and LLO at (06:43, 08:51) local time respectively. The injection code was not running when I logged on at each site. I've started a new log file tinj.err to try to ascertain why. Before restarting, I implemented a new check so that tinj should only attempt an injection if the detector is locked and if the intent bit is on. At LHO, I restarted tinj using the new user=hinj account.

17:13 IFO Locked, Intent bit set,

20:08 LLO has gone down, I start a DARM OLG measurement, done at 20:19

21:49 Lock loss

23:00 I start initial alignment again, start trying to lock at 23:45

We are seeing two issues with the autocal of the 18bit DAC cards (used almost exclusively by suspension models). The first is a failure of the autocal; the second is the autocal succeeding but taking longer than normal.

There are three calibration completion times: ~5.1S for unmodified DAC, ~5.3S for modified DAC, ~6.5S for modified long-running DAC

h1sus2b, h1sush34, h1susex, h1susey all have no DAC issues

h1sush2a: the third DAC is taking 6.5S to complete. This DAC is shared between PRM and PR3 models. First two channels are PRM M1 Right and Side. Last six channels are PR3 M1 T1,T2,T3,Left,Right,Side.

h1sush56: this has unmodified cards. 4th DAC is failing autocal

h1susb123: This one is strange:

On first autocal run after a computer reboot: 7th DAC failed autocal

On first restart of all models: 1st DAC failed autocal, 7th DAC succeeded

On second restart of all models: 1st DAC failed autocal, all others good (consistent restart behavior)

In all cases, 5th DAC card is running long for autocal (6.57S).

In the current situation with the 7th card now succeeding and the 1st DAC failing, the 1st DAC is driving ITMY M0 (F1,F2,F3,LF,RT,SD) and M0 (LF,RT)