TITLE: 06/05 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: Jim
SHIFT SUMMARY:  Commissioning all day hunting down ALS issues and inability to proceed past PREP_TR_CARM.
LOG: 

22:52 Jenne and Sheila to LVEA ISCT racks to look at electronics.

Similar to the work done on the BSC-ISI's, I've been working on tilt decoupling on the HAMs. So far I have most of the measurements needed for the the output HAMs, I just need a couple more measurements on the input HAMs to finiish. The first attached plot is the Z to X/Y coupling for HAM6. For this data, I took the Z tf at the isolation bank input with the ISI isolated with high blends (some measurements had sensor correction on, some off, it doesn't seem to matter much, off is probably best though). I then inverted the gs13 response (to get the units to nm/s), converted to nm and then divided by -g/w^2 to get the effective tilt. Doing this let me calculate the magnitude and sign decoupling element by getting the average of the Z to X/Y tf where the phase was either 0 or 180, usually over 10 to 80-100 mhz. I was also able to avoid getting the RX/RY tfs as I done for the BSCs by thinking a bit about the RX/RY to Y/X tfs. These transfer functions go as -g/w^2, so their signs are also possible to figure out, and after some thinking I realized I could model them by using (essentially) a +1 for the RX to Y tf and -1 for RY to X tf (something Jeff mentions in his eligo era log here). The second attached plot shows the improvements I was able to get for HAM6 Z to RX/RY decoupling.

It's interesting that while the Z tilt decoupling clearly needed to be done, the X/Y to RX/RY coupling does not seem so bad. I have taken several X or Y tfs, but all of the coefficients are pretty small (on order of .1%). In fact the coefficients are small enough that the X to RY or Y to RX coupling is hard to seen. The third attached plot is the HAM4 Y to RX/RY measurement. The blue trace is the Y to X tf and gives a coefficient of about .0015 for the Y to RY element. The red trace is the Y to Y tf and is dominated by the Y to Y closed loop tf. The first couple of frequency points around 5 mhz indicated a Y to RX coefficient of something like .00005. I don't fee like waiting around for a measurement long enough to resolve that any better. This seems to be typical of the "beam-line" coefficients for the HAMs.

So far the elements I've installed are:

HAM 4     Z        Y

    RX     .01    .0015

    RY    -.01

HAM 6     Z   

    RX     -.0181

    RY      .0178

I have a few more calculated coefficients, but I'm going to wait for a chance to do a before/after measurement to make sure I actually make things better.

JeffK, TravisS

We noticed a DIAG_MAIN message reporting that the EY PCal RX PD was more than 1% out of normal.  We had this warning put in place to indicate if clipping we have seen previously at EY returns.  This time, however, it turned out not to be clipping, which is a good thing, but rather that something had happened that railed the OFS loop.  Trending the TX, RX, and OFS PDs showed that they all railed at the same time (of course) on June 2.  Talking to Sheila and looking at Ed's OPS shift summary from that day, there was a lot of electronics work done at EY earlier in the day.  Not conclusive that the two things are related, but definitely possible.

Hopefully this doesn't happen very often, but if it does, the fix is very easy.  Open the PCal screen for the end station of concern (from the CAL tab on the Sitemap), click Shutter 'OFF' button circled in the attached screenshot, then click the Shutter 'ON' button.  This should fix the issue.  If it does not, please contact your friendly neighborhood PCal person.

I measured the output power as a function of laser crystal temperature for the ALS laser that was at EndX
up until recently (InnoLight Prometheus S/N 2011B).

    The laser crystal temperature was varied by ~7 degC, which corresponds to about 42 GHz of frequency
tuning.  At the time of the measurements, the laser settings were:
diode 1 temperature             21.00 degC
diode 2 temperature             20.30 degC
diode current                   1.837 A
doubling crystal temperature    32.9 degC

    The laser was set to the following in order to put it into the middle of the mode hop free region
laser crystal temperature       26.11 degC
doubling crystal temperature    33.09 degC

    The output power of the laser was 1.600 +/- 0.002 W at 1064 nm and 48.1 mW at 532 nm with these
settings.

CP4 Fill completed in 244 seconds. Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 244 seconds. TC B did not register fill. LLCV set back to 41.0% open.

My bad!  I had neglected to remove this ladder from having used it to access IP3 and IP4 following the recovery of the ITMx incursion a few weeks ago.  It was also in contact with the "Thermos" cooler (laser enclosure?) located at the base of the "giraffe" pier on the South side of the OMC tube near HAM4.

J. Kissel
II 4705
WP 7010

After recent review of open FRS/Integration Issue tickets, I've found and remembered that we haven't yet connected the OMC SUS's USER Watchdog to the ISI HAM6 (and hasn't been since before the OMC SUS was initially installed before O1). 

A while back, when we re-remembered this we'd argued that, if/because the OMC SUS OSEMs are covered in the IOP / Independent Software Watchdog (SWWD), that we need not both connecting the USER watchdog. However, I think now (a) since LLO has their's connected, and (b) if we're going to rely soley on the SWWD, this should be done for all chambers, not just HAM6, and it should be raised to a more general review of watchdog philosophy / ECR / integration issue, that is more work than we want to do today.

So, in prep for making the change, I've connected the USER watchdog at the top-level model of 
   /opt/rtcds/userapps/release/sus/h1/models/h1susomc.mdl. 

We'll restart the model if/when convenient today or if not tomorrow during maintenance. This will not require a DAQ restart.

See attached before vs. after comparison of the simple change in the lower right-hand corner.

Laser Status:
SysStat is good
Front End Power is 34.06W (should be around 30 W)
HPO Output Power is 158.5W
Front End Watch is GREEN
HPO Watch is GREEN

PMC:
It has been locked 3 days, 19 hr 9 minutes (should be days/weeks)
Reflected power = 16.11Watts
Transmitted power = 59.46Watts
PowerSum = 75.57Watts.

FSS:
It has been locked for 0 days 0 hr and 45 min (should be days/weeks)
TPD[V] = 3.415V (min 0.9V)

ISS:
The diffracted power is around 3.2% (should be 3-5%)
Last saturation event was 3 days 19 hours and 9 minutes ago (should be days/weeks)

Possible Issues:
Head 1-4 flow error, see SYSSTAT.adl

Averaging Mass Centering channels for 10 [sec] ...
2017-06-05 10:43:01.360996

There are 5 T240 proof masses out of range ( > 0.3 [V] )!
ETMX T240 2 DOF X/U = -1.017 [V]
ETMX T240 2 DOF Y/V = -0.987 [V]
ETMX T240 2 DOF Z/W = -0.743 [V]
ITMX T240 1 DOF X/U = -0.664 [V]
ITMX T240 3 DOF X/U = -0.685 [V]

All other proof masses are within range ( < 0.3 [V] ):
ETMX T240 1 DOF X/U = -0.064 [V]
ETMX T240 1 DOF Y/V = -0.03 [V]
ETMX T240 1 DOF Z/W = -0.008 [V]
ETMX T240 3 DOF X/U = -0.05 [V]
ETMX T240 3 DOF Y/V = -0.161 [V]
ETMX T240 3 DOF Z/W = -0.052 [V]
ETMY T240 1 DOF X/U = -0.008 [V]
ETMY T240 1 DOF Y/V = 0.18 [V]
ETMY T240 1 DOF Z/W = -0.224 [V]
ETMY T240 2 DOF X/U = 0.147 [V]
ETMY T240 2 DOF Y/V = -0.231 [V]
ETMY T240 2 DOF Z/W = 0.032 [V]
ETMY T240 3 DOF X/U = -0.244 [V]
ETMY T240 3 DOF Y/V = -0.019 [V]
ETMY T240 3 DOF Z/W = 0.271 [V]
ITMX T240 1 DOF Y/V = -0.213 [V]
ITMX T240 1 DOF Z/W = -0.15 [V]
ITMX T240 2 DOF X/U = -0.162 [V]
ITMX T240 2 DOF Y/V = -0.135 [V]
ITMX T240 2 DOF Z/W = -0.182 [V]
ITMX T240 3 DOF Y/V = -0.131 [V]
ITMX T240 3 DOF Z/W = -0.075 [V]
ITMY T240 1 DOF X/U = -0.024 [V]
ITMY T240 1 DOF Y/V = -0.046 [V]
ITMY T240 1 DOF Z/W = -0.058 [V]
ITMY T240 2 DOF X/U = 0.125 [V]
ITMY T240 2 DOF Y/V = 0.138 [V]
ITMY T240 2 DOF Z/W = 0.008 [V]
ITMY T240 3 DOF X/U = -0.248 [V]
ITMY T240 3 DOF Y/V = 0.053 [V]
ITMY T240 3 DOF Z/W = -0.263 [V]
BS T240 1 DOF X/U = -0.166 [V]
BS T240 1 DOF Y/V = -0.03 [V]
BS T240 1 DOF Z/W = 0.219 [V]
BS T240 2 DOF X/U = 0.067 [V]
BS T240 2 DOF Y/V = 0.195 [V]
BS T240 2 DOF Z/W = 0.007 [V]
BS T240 3 DOF X/U = 0.028 [V]
BS T240 3 DOF Y/V = -0.068 [V]
BS T240 3 DOF Z/W = -0.118 [V]

Assessment complete.

2017-06-05 10:45:39.355333
All STSs prrof masses that within healthy range (< 2.0 [V]). Great!

Here's a list of how they're doing just in case you care:
STS A DOF X/U = -0.609 [V]
STS A DOF Y/V = 0.203 [V]
STS A DOF Z/W = -0.448 [V]
STS B DOF X/U = 0.492 [V]
STS B DOF Y/V = 0.314 [V]
STS B DOF Z/W = -0.307 [V]
STS C DOF X/U = -0.37 [V]
STS C DOF Y/V = -0.661 [V]
STS C DOF Z/W = 0.497 [V]
STS EX DOF X/U = 0.018 [V]
STS EX DOF Y/V = 0.624 [V]
STS EX DOF Z/W = -0.01 [V]
STS EY DOF X/U = 0.002 [V]
STS EY DOF Y/V = 0.213 [V]
STS EY DOF Z/W = 0.443 [V]

Assessment complete.

Ed, Jason

Everything looks normal. The ever degrading HPO diode box 1 is causing a strange anomalous increase in the DC output power due to the changing thermal lens in head 1 resulting in an operating point which is, ironically,  actually better for the laser.

TITLE: 06/05 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
    Wind: 2mph Gusts, 1mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.11 μm/s
QUICK SUMMARY:  Arrived to find the arms locked in green (they have been locked in green for ~12 hours according to trends).  Will consult with commissioners as to how to proceed today.

   Perform monthly PSL Chiller Filter Checks. No debris or air found in either filter. There is some yellowing of the filter pleats but this is normal. All is normal with both the Crystal or Diode chiller filters. 

   Closing FAMIS task #8297.

Summary: 

After swapping a few electronics that we broke while investigating the ALS glitches at EY, the glitches are still there, but not nearly as bad as they were.  We have had these glitches go away on their own before when nothing was done, so that might be what is happening.  Now they are at End X, and about as bad as they were at End Y for most of the week.  ALS still will not lock for longer than 10 minutes.  It might be that this problem will go away on its own, but will probably come back again.  

• After seeing that the behavior of the VCO at end Y tune mon changed around 6:30 UTC June 2nd, and based on Keita's alog last night, I swapped it for a spare I found in the EE shop this morning.  The unit I removed is S1200566, the unit I replaced it with is S12000562.  The VCO is no longer railing and the tune mon is near zero when the tune input is terminated.  The tune monitor BNC however has about 900 mV output even with the tune input terminated.  After this the arm would lock and I was not getting any VCO errors
• I found that the Seismic configuration was left in Large earthquake mode since Friday, and changed it to windy.  I found this because the arm transmitted powers were fluctuating more than normal for both arms, and also the oplevs showed extra noise since Friday evening.  Now the arms are both locked and quiet.
• At 1:08-1:23 local time, I started 15 minutes of locking the Y arm in green without automation (the PLL automation is on, not the PDH automation).  At 1:32-1:47 I did another 15 minute test with the only difference being that the PDH CM board was controlled by the tester, not the automation. At 1:50 I locked the arm using the automation, in the normal way. During all of these tests, there are a few glitches, but not as many and not as large as we were seeing mid week and last weekend. 
• I attempted to lock ALS, which didn't hold for more than a few minutes.  I then noticed that we are having the type of glitches we had been struggling with in the Y arm in the X arm now. (1st attached screenshot shows the Y arm and X arm both locked in the normal configuration with automation).
• I went to end X, and put the cable for the PDH signal into IN1 of the PLL board.  The X arm locked for green with in 1 polarity negative, and -4 dB of gain.  (For Y arm, we used positive polarity and 6 dB of gain).  No glitches in this configuration. 
• I also swapped the cables to substitute the spare FET demodulator for the PFD.  I added a 20dB attentuator to the RF cable into the demod, with this in I got a ugf of 27 kHz for an in1 gain of 13 dB. The autolocker works fine to lock this, you just have to force it to ignore the error message from the PFD by hitting the "Force" button. The arm was locked this way for 15 minutes starting at 0:03:30 UTC June 5th for about 10 minutes, the glitches were just as bad as the normal configuration. 
• I used Keita's set up from End Y to use the CM board tester at End X.  The cavity was locked this way from 0:47 UTC to 0:54 UTC June 5th, glitches are just as bad as the normal configuration.  

Things that are not the cause of glitches at EX:

From PLL bypass test:

• PLL servo board
• PLL servo board controls
• Laser
• PDH sensing, including diode, demod, and LO
• PLL automation

Other things that are not the problem:

• PFD (glitches still seen when it was swapped for a mixer)
• PDH servo board control cables swapped (so PDH control cables and automation are not the problem)
• loop instabilities (X end PLL UGF is 24 KHz with 47 degrees of phase margin, PDH Ugf is 5.6kHz with 56 degrees of phase margin).
• The beatnote is a clean sine wave, there are 8MHz sidebands on it but they are 40dB smaller than the carrier. 

Things that we haven't checked:

• PDH servo board, but this is not likely to be the problem since it wasn't at End Y and swapping it has some risks.
• VCO, we could try swapping this for an IFR but I wasn't able to find one in the shop today. 
• BBPD
• A problem on the light coming out of the fiber. it still seems like glitches on the PSL are unlikely because the glitches are not at the same time in the two end stations. 

I recommend that the first thing in the morning operators take the ISC lock guardian to locking arms green, wait there for 10-20 minutes, and look at second trends of the arm transmissions.  You can compare them to the screenshot I've attached here, if the glitches are gone or much smaller it would be worth doing an initial alignment and trying to lock to DC readout, if the glitches are still there it is not worth trying.