Dave Barker. WP5269

I have put together a quick EPICS alarm handler for Gamma Ray Bursts and Supernovae (GRB/SN) which runs on the operator alarm machine. It is called grbsn.alhConfig (in svn under cds/h1/alarmfiles).

The GRB alert system (which runs on h1fescript0) polls the GraceDB database every 10 seconds looking for External events (Gamma Ray Bursts and Supernovae). If an event is detected, its information is written to EPICS records which are hosted on the FE model h1calcs.

To put together a quick alarm system for ER7, I am using the alarm fields of the H1:CAL-INJ_EXTTRIG_ALERT_TIME record which records the GPS time of the trigger. By setting the HIGH alarm field to the current GPS time plus one second, when the next event is recorded this PV goes into the MAJOR alarm state.

The grbsn.alhConfig file provides the operator with two buttons:

[G] button opens the guidance text. It refers to the alog entry  https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=19037

[P] button runs a script which resets the alarm level to the current event, turning off the current alarm

When a GRB/SN alarm is raised the operator should:

1. acknowledge the alarm to stop it beeping/flashing

2. press the [P] button to turn off the current alarm and prime the system for the next event

3. read the guidance to see if any further action is needed

Information on the current event can be obtained by running the script grb_latest_event_info.py. This gives you the event time as local time, so you can easily determine when the one hour stand-down time will expire.

(baffled control room crew)

After the most recent lock loss the IMC wasn't relocking and looked completely misaligned. The screenshot shows about 20 minutes of the previous lock, 30 minutes of confusion and 10 minutes on a new lock. The only dof which shows a signifcant change is the input pointing. Strange.

Observation Bit: Commissioning 

08:00 – Working on relocking
08:05 – Add 150ml water to PSL crystal chiller
08:11 – IFO Locked at LSC_FF
08:13 – Intent bit set to Undisturbed 
08:30 – Bean tube cleaning team working on X-Arm
09:11 – Set intent bit to Commissioning – Kieta running charging measurements at both end stations
09:33 – LLO ops reported a power hit this AM. Working on recovery
09:49 – Lockloss – Commissioning activities 
11:52 – Jim Blunt on site to meet with Richard
11:55 Dan – Making delivery to VPW with PU truck
11:59 – IFO Locked at LSC_FF
12:00 Keita – Running charging measurements – Intent bit Commissioning 
12:21 – Beam tube cleaning crew breaking for lunch
13:08 Daniel – Moving charge test rig from End-Y to End-X
13:45 – Beam tube cleaning crew working on X-Arm
14:52 Daniel – Back from End stations
15:00 Keita – Finished with charge measurements
15:03 - IFO locked at LSC_FF
15:05 - Set Intent bit to Undisturbed 
15:15 Bubba – Going to Mid-X to get some tubing
15:17 - Lockloss – Possible due to beam tube cleaning
15:42 - Beam tube cleaning crew finished for the day
16:29 - Turn over to Jim W. – Still working on relocking

We attempted to set up the IFO for some end station ESD measurements around 10am.  During some of the switch settings, the IFO lost lock.  Relocking is ongoing with help from commiss.  A suite of ESD/charge measurements have become increasingly more important, hence time has been allotted today.

00:00 IFO not locked. Jim said it lost lock twice at BOUNCE_VIOLIN_MODE_DAMPING. And lost lock again at ANALOG_CARM when I came.

          Knowing that it fails to grab lock repeatedly, I approach all the Guardian state with caution (wait until the spectrum becomes settle before requesting the next state)

01:13 Lock lost at REFL_TRANS. I brought everything down, readjust SR2, PR2, PR3, BS, and IM4. Ended up took me a while because I messed up the alignment.

         Couldn't get ASC-C_COMM_A_DEMOD_RFMON to reach 5 counts. I touched PR3 all I could. Any other optics I could touch to improve this?

2:18 4.7 Earthquake in Japan - no WD tripped.

4:00 Lock loss at LOCK_DRMI_1F

4:10 Trouble locking ALS-X

5:00 Kristina to the wood shop.

        The ifo still tries to lock DRMI.

5:10 Kristina back.

6:16 Still not grabing DRMI lock. Evan suggested bad BS alignment as AS air momentary locked at 10 mode. I couldn't get it to lock just by adjusting the BS so I redid the initial alignment. The earthquake earlier might have thrown the optics off.

6:50 Lock loss at REFL_TRANS.

7:19 Lock loss at REFL_TRANS AGAIN. I waited until AS90 and POP18 became stable before requesting the state.

7:31 Stopping at DRMI_LOCK. Waiting for the 6.0 earthquake in Chile to pass (it didn't affect us - no WD tripped at least).

8:06 Hugh came to find ITMY Optlev damping has been turned off. This could have been the cause of repeated lock losses through out the night. Next time I'll be sure to check SDF before even touching the interferometer....

       Evan touched PRM to bring up POP18 (and AS90) before requesting RESONANCE. Maybe helped with the lock loss at REFL_TRANS?

       The ifo is now locking. Handling the ifo off to Jeff B.

Add 150ml water to the PSL Crystal chiller. 

Operators and commissioners were reminded to check the SDF file for changes. There were problems with locking last night due in part to the ITMY OpLev. This error was apparent when looking at the SDF overview.

All were reminded to aLOG, aLOG, and aLOG. The aLOG is a valuable repository for documenting the many things that happen during a day. When in doubt aLOG it.
 
The kitchen sink has been fixed. The Facilities folks asked that food scraps not be put into the sink, as they cause plugged pipes. Food scraps should be put into the compost bins or at the least put in the garbage, but not down the sink drain.

Repairs to the RO system are complete. There is no potable water in the staging building due to a water leak. 

Beam tube cleaning continues.

No safety meeting for today.

Charge measurements will be taken today at both end stations.  

Apon arrival, I see that the IFO is back up but there are 2 red SDF channels:

One is a somewhat benign LIMIT on the BS OPLEV YAW bank.  The other is 2 new filters that are enabled on ITMY_M0_DARM_DAMP_R.  I *thought* I saw these being switched a bit via commissioners yesterday during the locking attempts that were taking a while yesterday.  There are no alogs about these being turned on.  This needs to be addressed by ops/commiss today.

OPERATORS:  The SDF and GUARDIAN overview screens are very useful at determining what is out of configuration when attempting to lock.  If the GRD overview screen is litup like a "holiday" tree on the SUS and SEI banks, things are wrong and likely inhibiting lock.  Likewise on the SDF overview screen.  If there are red indications of channel diffs, it might help locking to work through the diffs.  Yesterday we had numerous indications on both of these screens that things were not in good states to lock.

model restarts logged for Tue 09/Jun/2015
2015_06_09 10:26 h1iopsush2a
2015_06_09 10:26 h1susmc1
2015_06_09 10:26 h1susmc3
2015_06_09 10:28 h1suspr3
2015_06_09 10:28 h1susprm
2015_06_09 10:32 h1iopsush2b
2015_06_09 10:32 h1susim
2015_06_09 10:37 h1iopsush34
2015_06_09 10:37 h1susmc2
2015_06_09 10:37 h1suspr2
2015_06_09 10:37 h1sussr2
2015_06_09 10:48 h1broadcast0
2015_06_09 11:47 h1iopsusb123
2015_06_09 11:47 h1susbs
2015_06_09 11:47 h1susitmx
2015_06_09 11:47 h1susitmy
2015_06_09 17:25 h1fw1*
2015_06_09 19:16 h1fw1*
2015_06_09 21:03 h1fw1*

* = unexpected restart

Maintenance day. Restart of SUS IOP models for 18bit DAC recalibration. Reboot DMT broadcaster.

The last time it lost lock I tried my best to approach this state slowly. I have attached the screenshots of the spectrum and PRMI StripTool when it was at CARM_5PM. I waited until POP18 and AS90 became relatively stable before moved on and requested REFL_TRANS.  Didn't help...

I restarted transient injections at LHO and LLO at (06:43, 08:51) local time respectively. The injection code was not running when I logged on at each site. I've started a new log file tinj.err to try to ascertain why. Before restarting, I implemented a new check so that tinj should only attempt an injection if the detector is locked and if the intent bit is on. At LHO, I restarted tinj using the new user=hinj account.

17:13 IFO Locked, Intent bit set,

20:08 LLO has gone down, I start a DARM OLG measurement, done at 20:19

21:49 Lock loss

23:00 I start initial alignment again, start trying to lock at 23:45

Sudarshan, Kiwamu, Darkhan,

Abstract

According to the PCALY line at 540.7 Hz, the DARM cavity pole frequency dropped by roughly 7 Hz from the 17 W configuration to the 23 W (alog 18923). The frequency remained constant after the power increment to 23 W. This certainly impacts on the GDS and CAL-CS calibration by 2 % or so above 350 Hz.

Method

Today we've extracted CAL-DELTAL data from ER7 (June 3 - June 8) to track cavity pole frequency shift in this period. The portion of data that can be used are only then DARM had stable lock, so for our calculation we've used a filtered data taking only data at GPS_TIME when guardian flag was > 501.

From an FFT at a single frequency it is possible to obtain DARM gain and the cavity pole frequency from the phase of the DARM line at a particular frequency at which the drive phase is known or not changing. Since the phase of the resultant FFT does not depend on the optical gain but the cavity pole, looking at the phase essentially gives us information about the cavity pole (see for example alog 18436). However we do not know the phase offset due to time-delay and perhaps for some uncompensated filter. We've decided to focus on cavity pole frequency fluctuations (Delta f_p), rather than trying to find actual cavity pole. In our calculations we have assumed that the change in phase come entirely from cavity pole frequency fluctuations.

The phase of the DARM optical plant can be written as

phi = arctan(- f / f_p),

where          f is the Pcal line frequency;

                     f_p - the cavity pole frequency.

Since this equation does not include any dependence on optical gain, the technique we use, according to our knowledge, the measured value of phi does not get disturbed by the change of the optical gain. Introducing a first order perturbation in f_p, one can linearize the above equation to the following:

               f_p^2 + f^2
(Delta f_p) = ------------- (Delta phi)
                    f

An advantage of using this linearized form is that we don't have to do an absolute calibation of the cavity pole frequency since it focues on fluctuations rather than the absolute values.

Results

Using f_p = 355 Hz, the frequency of the cavity pole measured at the particular time (see alog 18420), and f = 540.7 Hz (Pcal EY line freq.), we can write Delta f_p as

Delta f_p = 773.78 * (Delta phi)

Delta f_p trend based on ER7 data is given in the attached plot: "Delta phi" (in degrees) in the upper subplot and "Delta f_p" (in Hz) in the lower subplot.

Judging by overall trend in Delta f_p we can say that the cavity pole frequency dropped to about 7 Hz after June 6, 3:00 UTC, this correspond to a time when PSL power was changed from 17 W to 23 W (see lho alog 18923, [WP] 5252)

Delta phi also show fast fluctuations of about +/-3 degrees, and right now we do not know the reason that causes this "fuzzyness" of the measured phase.

Filtered channel data was saved into:

aligocalibration/trunk/Runs/ER7/H1/Measurements/PCAL_TRENDS/H1-calib_1117324816-1117670416_501above.txt (@ r737)

Scripts and results were saved into:

aligocalibration/trunk/Runs/ER7/H1/Scripts/PCAL_TRENDS (@ r736)

LLO just went down and Jeff asked for a DARM OLG measurement, should be 20 minutes or so. Will revert intent bit when done.

We are seeing two issues with the autocal of the 18bit DAC cards (used almost exclusively by suspension models). The first is a failure of the autocal; the second is the autocal succeeding but taking longer than normal.

There are three calibration completion times: ~5.1S for unmodified DAC, ~5.3S for modified DAC, ~6.5S for modified long-running DAC

h1sus2b, h1sush34, h1susex, h1susey all have no DAC issues

h1sush2a: the third DAC is taking 6.5S to complete. This DAC is shared between PRM and PR3 models. First two channels are PRM M1 Right and Side. Last six channels are PR3 M1 T1,T2,T3,Left,Right,Side.

h1sush56: this has unmodified cards. 4th DAC is failing autocal

h1susb123: This one is strange:

On first autocal run after a computer reboot: 7th DAC failed autocal

On first restart of all models: 1st DAC failed autocal, 7th DAC succeeded

On second restart of all models: 1st DAC failed autocal, all others good (consistent restart behavior)

In all cases, 5th DAC card is running long for autocal (6.57S).

In the current situation with the 7th card now succeeding and the 1st DAC failing, the 1st DAC is driving ITMY M0 (F1,F2,F3,LF,RT,SD) and M0 (LF,RT)