Attached is a two hour second-trend plot of the DAC drive  for the TCS ITMX temperature setpoint and the chiller water flow rate (which shows when the chiller tripped). The data is from Wednesday night, 10pm to midnight PDT.

At 22:42 PDT the IOP stopped driving the DAC outputs (presumably due to a FIFO error) with the user supplied signals and instead outputted zero volts.

At 23:32 PDT the chlller tripped (+50mins later)

Nutsinee says that the nominal driving temperature is around 20C, the trip point is 15C, and the maximum temperature rate-of-change is 0.1C/minute. The transition from 20C to 15C would then take 50 minutes, which agrees with the timing seen on Wednesday.

The plot shows that after the chiller tripped, the control room was alerted of the problem, the front end was restarted and the chiller was reset within 10 minutes.

Clearly if the verbal alert system warned the control room at the time the DAC output went to zero, in this failure mode we will have plenty of time to restart the OAF IOP model and recover chiller operations.

P. King, J. Bartlett, J. Oberling

We swapped the PSL chillers for the backup in an effort to isolate the recent laser tripping issues.  The laser is now back up and running.

Operating hours for the chillers, as reported by the chillers themselves and the chiller MEDM screen, are:

There is a difference between the chillers and MEDM because the MEDM screen gets its info from the PSL Beckhoff PC, which uses its own internal counter to track the chiller hours instead of the hours recorded by the chillers themselves; we currently do not know a safe way to reset this Beckhoff counter.

I wrote a simple interface to download data from the Tektronix network/web-enabled oscilloscopes (TDS3034B and (presumably) the like).  You can use it to download a single data file, or download data periodically:

jameson.rollins@opsws8:~ 0$ readlink -f $(which tektronix)
/opt/rtcds/userapps/trunk/sys/common/src/python/tektronix.py
jameson.rollins@opsws8:~ 0$ tektronix -h
usage: tektronix [-h] <command> ...

Interact with Tektronix oscilloscope over the network.

positional arguments:
  <command>
    download  Download data from scope periodicially

optional arguments:
  -h, --help  show this help message and exit

Use environment variable HOST to specify network address.
(default: 10.22.10.51)
jameson.rollins@opsws8:~ 0$ tektronix download -h
usage: tektronix download [-h] [-c CHANNEL] [outdir] [count] [period]

Download data from scope periodicially

Data will be saved in a two-column (time,voltage), comma-separated CSV
file named "<time>.dat", where <time> is an ISO-formatted timestamp of
the local time of the data snapshot.

NOTE: data takes about 30 seconds to download, so period should not be
less than 30 seconds.

positional arguments:
  outdir                directory to store timestamped data ['.']
  count                 number of downloads (0=Inf) [1]
  period                time between downloads in seconds [60]

optional arguments:
  -h, --help            show this help message and exit
  -c CHANNEL, --channel CHANNEL
                        channel to download ['ch1']
jameson.rollins@opsws8:~ 0$ tektronix download
output directory: /ligo/home/jameson.rollins
fetching 2016-09-09T11:16:31.747247.dat ...done (in 29.959158s).
jameson.rollins@opsws8:~ 0$

WP6146, fix BIO bug, add PD filters.

Daniel, Keita, Dave:

h1psliss mode was restarted, followed by a DAQ restart.

model restarts logged for Thu 08/Sep/2016
2016_09_08 08:25 h1calcs
2016_09_08 08:25 h1iopoaf0
2016_09_08 08:25 h1ngn
2016_09_08 08:25 h1oaf
2016_09_08 08:25 h1odcmaster
2016_09_08 08:25 h1pemcs
2016_09_08 08:25 h1susprocpi
2016_09_08 08:25 h1tcscs

Restart of h1oaf0 models (specifically the iop) to clear a re-occurance of a dac-fifo error.

Came into the Control Room to find that the laser had tripped.  It was up and running an hour ago when I checked
it from my office.

    Same problem as before.  The error bit for the external flow sensors was not set in Beckhoff, so the problem
is not with the ADC terminals.  The flow sensors are pretty close to brand new.  Conclusion is that it's the
chiller that's glitching, for whatever reason.

Kiwamu, Stefan

We had two long-lock attempts tonight. During both we tracked the the REFL optical gain to frequency noise, as well as the ASC optical gain to an IM4 pitch dither.

The first one we started from a 20W lock that had more CO2 central heating on the X compensation plate (see Kiwamu's log).During this first lock we again saw the sign of the REFL45_I signal to frequency noise flip. This happened about 15min after reaching 50W.

The second lock started with the regular TCS settings. We reached 50W, and reduced the 45 modulation index by 2dB. Guardian now uses the now BS ASC input matrix, only using AS_A_36 with a gain of 1.5 (PIT and YAW). 
In this setup we were able to hold the lock for 1h20min.

Attached is a gain tracking plot for that 2nd lock. Note that there is a number of changes in REFL gains that will affect out ASC system:

- While the power is ramped up there seems to be a Gouy phase shift - with the signal in REFL_A_45 decreasing, and the signal in REFL_B_45 increasing.

- However, after that both 45 ASC signals show almost no change in gain.

- On the other hand, the two 9MHz signals drop their gain by about a factor of 3.

- Also note the episode around -50min - we manually tried to tweak IM4 and PR3, because we were afraid that the input matrix wasn't putting us to a good spot any more.

- Over the 80min the sideband 2-omega kept dropping slightly, but was fine otherwise. Towards the end we notced the AS_90 get rattier. We suspect that this is either due to the BS being slightly misaligned at DC, or due to some ASC loops significantly loosing gain.

Bottom line:

- It still seems possible to get the interferometyer running indefinitely at 50W, but we have to spend some more time understanding how the ASC gains change.

- But this still leaves some pretty uggly higher order modes in the interferometer - I have no idea what that will do to our noise couplings.
 

LEGEND for plot:

H1:LSC-LOCKIN_1_DEMOD_4_I_OUTPUT         LSC: laser frequency to REFL_9_I  (blue)
H1:LSC-LOCKIN_1_DEMOD_3_I_OUTPUT         LSC: laser frequency to REFL_45_I (dark red brown)
H1:ASC-LOCKIN_OSC1_DEMOD1_I_OUTPUT   ASC-REFL_A_RF45_I_PIT gain  (IM4 dither) (orange)
H1:ASC-LOCKIN_OSC1_DEMOD2_I_OUTPUT   ASC-REFL_B_RF45_I_PIT gain  (IM4 dither) (yellow)
H1:ASC-LOCKIN_OSC1_DEMOD3_I_OUTPUT   ASC-REFL_A_RF9_I_PIT gain  (IM4 dither) (light blue)
H1:ASC-LOCKIN_OSC1_DEMOD4_I_OUTPUT   ASC-REFL_B_RF9_I_PIT gain  (IM4 dither) (dark blue)

PS:

Some of the mysteriously unstable loops earlier today were due to the DOWN script now setting back all the gain changes related to the RF modulation depth reduction. This was fixed tonight.

At around 3:00 local, I increased the ring heater power on ITMY from 0.5 W (0.25 W each segment) to 1.5 W (0.75 W each segment). This has to be cancelled by applying approximately an additional 400 mW on CO2Y.

Jenne, Matt, Patrick, Sheila, Stefan, Kiwamu,

We tried two different differential CO2 settings today at 20W to see if we can improve the imbalanced RF sidebands (29535). One of the two tests was already reported by Jenne et al. on 29553.

According to the tests, the addition of a 10 uD single-pass substrate defocus to the X compensation plate seems to balance the amplitude of the upper and lower 45 MHz sidebands.

[Settings]

• Locked at 20 W
• CO2X  200 mW ->  1 W
• CO2Y  stayed at 0 W

The interferometer arrived at a 20 W lock at 22:53 local. We then added a 800 mW of laser power on to CO2X, resulting in a 1 W CO2 power at 22:55:40. As opposed to the previous test (29535), we did not change CO2Y this time and therefore it stayed at 0 W.

The measurement finished at 23:15 and we set CO2X to 700 mW. We then increased the PSL power to 50 W while maintaining the same lock.

[Results]

As expected from the previous test, the imbalance of the 45 MHz RF sidebands, as seen by the optical spectrum analyzer, was improved by this action. After roughly 20 min, we reached a point where the upper and lower sidebands almost had the same amplitude. HWSX was functioning at the beginning of the test (for 10 minutes or so), but then unfortunately started showing some meaningless signals. We have no idea why. Nevertheless, we were able to extrapolate where we would be in terms of the substrate defocus based on the initial rise. It looks like that an additional 20 uD double-pass (= 10 uD single pass) is the good defocus value for ITMX in order to get balanced upper and lower 45 MHz sidebands. A 10 uD single-pass defocus corresponds to a 400 mW CO2 laser on the X compensation plate in equilibrium [assuming a 25 uD/W actuation coefficient (single-pass), 28799].

The attached is a screenshot of OSA outputs in which scans from different time are overlaid. It is clear that the imbalance improved as time elapsed.

On the other hand, the frequency noise coupling became worse by a factor of a few comparing to the initial coupling value at 900 Hz. It did not experience a sign change. Not sure what this means.

- The ISI configuration has been changed from 'Earthquake V2' to 'Windy'.

- Each time the ISC_LOCK guardian reaches CARM_15PM it stops and reports a connection error to H1:LSC-PD_DOF_MTRX_SETTING_1_23 (see attached screenshot). Manually running ezca on the guardian machine gets the value with no error. This can be gotten around by taking the node to manual and selecting CARM_10PM. 

- Making it through the DRMI ASC engagement is hit and miss even with careful alignment of the BS and PRM mirrors.

- I was not able to make it past ENGAGE_REFL_POP_WFS until Stefan came back. Stefan and Kiwamu found the DHARD ASC loop to be unstable.

[Sheila, Kiwamu, Matt, Jenne, etc]

We changed the TCS CO2 settings after we had thermalized at 20 W for a while, and were watching several metrics.  Here I'll talk about the frequency line at 900 Hz, and Sheila will append her OSA plots as a function of time.

I injected a line at 900 Hz into H1:LSC-EXTRA_AO_2_EXC, which goes to H1:LSC-REFL_SERVO_COMEXC.  I demodulated Refl 9 and Refl 45 like Stefan did last night (alog 29509), and also added another demodulator of OMC DCPDs.  For each of these, I ran a zservo to adjust the demod phase to keep the Q signal at zero. 

In the attached plot, bright red is the PSL power, and yellow and taupe are the X and Y TCS CO2 powers.  Purple, cyan and brick red are Q-phase signals, so can be ignored.  Green is REFL 9, blue is REFL 45. Orange is OMC DCPDs, indicating the coupling of frequency noise to DARM.  Note that the range for blue, green and orange are all symmetric about zero, so while it looks like the frequency coupling is decreasing, in fact it is increasing.  Sorry for a poor choice of demod phase.

After about 40 min at 20 W, the CO2 powers were stepped differentially.  X went from 0.2 W to 0 W, while Y went from 0 W to 0.2 W.  When this happens, you can see that the frequency coupling to DARM gets worse even faster than it had been while thermalizing with the usual TCS settings at 20W.  The REFL 45 and REFL 9 traces don't change in character when we alter the TCS settings.  Clearly this is the wrong direction for freq noise coupling, but it's nice to see that we can definitely affect it.  Sheila will post her results when her plots are ready, but it looks like the sideband imbalance gets worse when we made this TCS step, so it's good that everything hangs together.

Next lock, we'll try increasing the ITMX CO2, and leave the ITMY CO2 at 0 W.  This is a combination of common and differential, but it's all we can do before stepping the ring heaters later tonight.

Jenne, Chris Whittle

When exciting H1:SUS-ITMX_L3_LOCK_L_EXC using awggui for ongoing ITMX charge investigations, we observed an output voltage with a sinusoid offset from 0 (i.e. varying between 0 and 1k counts instead of +/-1k), despite having explicitly set offset to 0. This also resulted in a pervasive comb in DARM. We saw this at excitation frequencies 16 Hz and 16.55 Hz (no others were tested).

Closing and restarting awggui resolved the issue.

16:15 Chris S. moving rigging racks from highbay back into LVEA.

15:19 Karen to Optics lab

15:32 Fil to PSL rack isntalling ISS out of loop chasis

15:50 Karen out

          Gerado to MX

16:01 Richard to LVEA

10:05 Richard out

16:48 Karen to light bulb room and H2

16:45 Fil back

17:05 Fil+Richard pulling ISS chasis back in

17:15 John to LVEA

17:33 John out

17:53 Kyle to MY

18:07 Fil out

18:24 Kyle back

18:25 Keita+KAGRA crew out of LVEA (was there for ~1hr -- tour)

20:01 Fire drill. Fire alarm in the VEAs *might* have caused a lockloss. There were high frequency noise visible in DARM as soon as the alarm started prior to the lockloss. Will look more into this.

20:26 Karen to MY

21:31 Kyle back

21:50 Kyle leaving MY

22:51 KAGRA crew drive down to EX -- sightseeing(?)

Kiwamu, Nutsinee

A code is scheduled to run at 9 pm tonight. The code will power up CO2 to 1.3W and power it back down to 0W after 6 hours. Please do not touch ITM optics.

I reset the ITMY HWS magnification to 7.5x (it's set to 17.5x by default in the code).

aidan.brooks@opsws4:~$ caput H1:TCS-ITMY_HWS_MAGNIFICATION 7.5

Old : H1:TCS-ITMY_HWS_MAGNIFICATION  17.5

New : H1:TCS-ITMY_HWS_MAGNIFICATION  7.5

Alos, I had a quick look at the change in the spherical power measured by the ITMY HWS after the IFO lost lock this morning.

1. The measured change in defocus was, very roughly, 0.8 micro-diopters. (over about 40 minutes taken from immediately after 1141749205)
2. Around 95% of the lens forms/decays in this time. 
3. The magnification was set to 17.5 instead of 7.5, so we need to multiply the spherical power by (17.5/7.5)^2 = 5.44x
4. The steady-state, double-passed, lens measured by the HWS is then 8E-7/0.95*5.44 = 4.6E-6 diopters
5. The single-pass, steady-state, lens is then 2.3E-6 diopters
6. The single-pass, steady-state, lens gain (diopters per Watt absorbed) is 4.87E-4 D/W for self absorption
7. Therefore I calculate about 4.7mW absorbed power in ITMY
8. The arm power was around 11.9kW during this time
9. Hence, the absorption is (very roughly) 4E-7

This is a very rough calculation - and assumes that the HWS-Y beam is correctly aligned (we need to combine our evidence to really confirm this).

Nevertheless, I've set the ITMY absorption to 4E-7 in the simulation.

aidan.brooks@opsws4:~$ caput H1:TCS-SIM_ITMY_SURF_ABSORPTION 4E-7

Old : H1:TCS-SIM_ITMY_SURF_ABSORPTION 4e-09

New : H1:TCS-SIM_ITMY_SURF_ABSORPTION 4e-07

For comparison, ITMX absorption estimated to be 5.7E-7.