Lockloss due to PSL tripping off.  Currently on the phone with Jason working to remotely restart if possible.  Will update as I know more.

We have been locked in Observing for almost 2 hours.  No issues to report.

I used the lockloss2 script that automatically checks for sus saturations and plots them using the lockloss tool, and saw that one of the three locklosses (2016-12-05 16:02:42 UTC) in the last day or so was probably caused by a glitch on SR3.  The attached screenshot shows the timeline, there is clearly a glitch on the top mass of SR3 about 0.2 seconds before the lockloss.  

The dither outputs (which we use for the cage servo) don't show anything unual until after the lockloss, which means that this is not a cage servo problem.  Looking at top mass OSEMINF LF and RT are the two that seem to have a glitch first, at about the same time. 

I've added a lockloss template /ligo/home/ops/Templates/Locklosses/channels_to_look_at_SR3.txt  for any operators who have an unexplained lockloss and want to check if it is simlar to this one. 

Back to Observing at 2:27 UTC.  No issues with relocking.  A slight tweak of PRMI was all that was needed.

Lockloss at 1:58 UTC.  No obvious cause.  Was coincident with me rebooting the OPS workstation.  OPS workstation 2nd monitor shut off, first monitor was showing symptom of needing to be rebooted (slow, jumpy mouse movements, etc.),  verified that the cable was still plugged in, and hardbooted.  Seems to have come back fine with both monitors, but seemed I have never seen this issue before so might need some further investigation.

We are now back in Observing mode.  No issues were encountered during the lock sequence, went all the way to NLN the first try after Sheila finished here camera work.  I ran Keita's a2l check script which showed that running a2l would be prudent.  I then ran a2l.  Set to Observing at 1:50 UTC.

I repeated the measurement from alog 31399.

The attached plot looks very similar for all traces but PSL-ISS_PDB_REL—indicating that the changes described in alog 32206 significantly reduced the extra noise in the first loop ISS path.

We also see that the new REFL_A_RIN channel (see alog 32191) is working fine.

Between 10 and 30 Hz there is coherence between REFL_A_RIN and the DCPDs. There is a clear shelf in both spectra pointing towards a parasitic interferometer path.

PT170 and PT180 were installed in March 2016 when the X/Y beam manifold volumes were vented. PT140 was installed in Sept. 2016. Attached is trend from time of installation. Looks like PT170 & PT180 have been drifting up since after closing and re-opening isolation GVs 5/7 for HAM6 vent. We burped in accumulated gas from gate annuli but I'm not sure what the second hump is from April 9th. No log entry found.

6373 increase rate of h1ascimc model

Daniel, Jim, Dave:

The h1ascimc model processing rate was increased from 2048 to 16384.

6374 add 16kHz LSC channel to DAQ

Daniel:

6378 Add channels to DAQ broadcaster

TJ(Detchar), Dave:

Slow channels were added to the DAQ DMT broadcaster

6371 Relocate digital camera17

Richard, Jim:

Digital camera #17 was relocated to PRM

DAQ restart

Dave:

DAQ was restarted once at 11:49PST to support model work.

I accepted SDF values for MC1 yaw and MC3 yaw today that appeared to be swaped, so have looked into alignment of MC1 and MC3 in yaw for 120 days.

It does appear the alignment slider values were swapped, however looking at the IMC WFS and when they were relieved, it also appears that the WFS relief moved the MC1 yaw alignment slider toward the old MC3 value and the MC3 yaw alignment slider toward the old MC1 value (WFS offsets go to zero).

Today IMC WFS have offsets on MC1 and MC3 yaw of around 27.  It might be interesting to run a WFS relief and see where the sliders end up.

Attached are 120 days: MC1 yaw OSEMs, MC1 IMC WFS output, MC1 yaw slider, IMC WFS A yaw, IMC WFS B yaw, MC3 yaw OSEMs, MC3 IMC WFS output, MC3 yaw slider

WP# 6379

Peter, Kiwamu,

As a part of the PSL ISS/mystery noise study, we swapped a mirror to reduce the light power going to the ISS box during the maintenance period today. This is a temporary configuration and we will decided on the final configuration based on the results we get out of this configuration.

Since the ISS box is not used for stabilizing the intensity, we don't expect any impacts on H1 noise or stability performance.

[M32 swapped, HWP rotated]

We swapped mirror M32 with an window, W1-PW-1025-1064-UV-45P without pulling out the entire mount. This decreased the light power coming into the ISS to 4 mW (measured by a Vega) as expected from presumably 400 mW. We rotated the half wave plate in the box to maximize the transmission and therefore we think PDB is now sensing much higher P-polarizing light than S-pol. Checking the reflection of the polarizing BS, we didn't visually see any light on a laser card which is good. After the swap, the power on PDB was measured to be 2 mW as expected due to a 50/50 BS in the box. After the fine alignment, PDB had a DC output of about 5 V. We also confirmed that the ADC was not saturating. We placed a Kenteck beam dump behind M32 to capture the large transmission.

We had a difficulty aligning the beam back to the center of PDB due to the narrow space and view. We spent an hour trying to get back to good alignment. During this process, we touched M32 and the steering mirror right in front of PDB. Once fine alignment was established, we re-centered the light on the QPD in the box by touching the steering mirror in front of it. Also, we removed an optical filter labeled "NG5 2mm". This had been attached to the entrance hole of the ISS box in the inside.

The attached are pictures showing the optical layout after today's activity. You can see M32 and the newly installed Kenteck beam dump.

TITLE: 12/06 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
    Wind: 8mph Gusts, 6mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.48 μm/s
QUICK SUMMARY:  Patrick was halfway done with IA when I arrived.  I finished IA and have locked DRMI a couple of times now.  I stopped at DRMI to let Sheila go to the LVEA to adjust the focus on the newly installed SRM digital camera.  Carrying on with locking now.

 

Leaving "temporary" rotating shaft pumps run at BSC8 overnight as LLO is still offline.  The expectation is to shut these down at the end of maintenance tomorrow or when LLO is back up (which ever occurs last, i.e. maximize pumping time)

Output from /opt/rtcds/userapps/release/sys/h1/scripts/VerbalAlarms/Lock_Summary.py. The lock numbers are not correct?

Lock# 2
Times are in: GPS (UTC)
Start time: 1164811816.0  (Dec 3 14:49:59 UTC)
  Itbit Engage: [0, 1164811954.0, 1164853675.0]
  Itbit Disengage: [0, 1164853582.0, 1164931220.0, 1164954645.0]
End time: 1164954645.0  (Dec 5 06:30:23 UTC)
Total length: 142829.0 , 39hr 40min 29sec
Total Science: 119173.0 , 33hr 6min 13sec


Lock# 3
Times are in: GPS (UTC)
Start time: 1164960365.0  (Dec 5 08:05:48 UTC)
  Itbit Engage: [0, 1164961121.0, 1164962290.0]
  Itbit Disengage: [0, 1164961954.0, 1164988988.0]
End time: 1164988988.0  (Dec 5 16:02:46 UTC)
Total length: 28623.0 , 7hr 57min 3sec
Total Science: 27531.0 , 7hr 38min 51sec

TITLE: 12/06 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Locking
INCOMING OPERATOR: Travis
SHIFT SUMMARY: Maintenance all day. Handed off initial alignment to Travis. LVEA and VEAs have been swept by Corey and then Kiwamu and Peter when they finished working in the PSL enclosure. Kyle has a pump running at PT180. HAM2 and HAM3 ISI and MC1, MC2 and MC3 SUS tripped during model restarts. Cheryl brought back the alignment of the input optics. The camera for SRM was swapped for a GigE one. It still needs to be focused. I set the ISI state to SC_OFF_NOBRSXY during maintenance. It is now back to WINDY. The ESD HV power supply was swapped at end Y.
LOG:
16:01 UTC Joe to LVEA
16:03 UTC Lock loss
16:45 UTC Filiberto pulling network cable for GigE camera, climbing on HAM4
16:48 UTC Put ISC lock guardian to down
16:48 UTC Bubba bringing boom lift from end Y back to corner
16:50 UTC Set SEI config to SC_OFF_NOBRSXY
16:53 UTC Gerardo to LVEA to swap SRM cameras, VP9 on output beam tube, north side, near HAM4
16:54 UTC Peter to PSL enclosure to start reference cavity alignment
16:55 UTC Richard to LVEA to help Gerardo
16:59 UTC Jason to PSL enclosure to help Peter
17:10 UTC Kyle staging for PT180 work, climbing around BSC8
17:13 UTC Jeff to LVEA to start work on cleanroom curtains
17:13 UTC Joe out of LVEA
17:18 UTC Kiwamu to optics lab and possibly LVEA to look for optics
17:25 UTC Filiberto and Mark to end Y to swap ESD HV power supplies
17:35 UTC Filiberto turning off ESD HV at end Y, Betsy set bias to 0
17:58 UTC Set IMC_LOCK guardian to auto and then offline
18:17 UTC Jason and Peter done
18:18 UTC Richard to LVEA to unplug illuminator for HAM5
18:18 UTC Gerardo to LVEA to look for other cameras on spool piece
18:19 UTC Chris to LVEA to clean
18:32 UTC Gerardo back
18:33 UTC Filiberto and Mark done at end Y, Betsy starting charge measurements
18:34 UTC Peter toggling noise eater
18:41 UTC Peter back
18:44 UTC Gerardo to end Y to check chilled booster water pump glycol leak in mechanical room
18:58 UTC Aaron restarting calibration pipeline
19:07 UTC Aaron done
19:11 UTC Dave restarting models, MC1, MC3, HAM2 ISI, MC2, HAM3 ISI tripped
19:18 UTC Corey taking PSL enclosure to science mode
19:21 UTC Joe back to LVEA
19:23 UTC Gerardo back from end Y, did not go into VEA or use card reader
19:27 UTC Gerardo to LVEA to look for parts in west bay

Cheryl looking at IM alignment

19:42 UTC Gerardo back
19:48 UTC DAQ restart
19:56 UTC Jeff B., Joe, Chris done in LVEA
20:00 UTC Mark to end Y to check setting on new ESD power supply
20:15 UTC Starting initial alignment
20:17 UTC Set ISI config back to Windy
20:18 UTC Mark back
Corey done LVEA sweep, Kyle still has pump running on PT180.
20:56 UTC Betsy to LVEA to look for part
20:59 UTC Kiwamu starting WP 6379, stopped initial alignment between PRM align and PRM align offloaded.
21:19 UTC Betsy done
22:04 UTC Jeff B. getting garb from cleaning area
22:12 UTC Jeff B. done
22:19 UTC Alfredo and Elisabeth to mid Y to inventory cables
23:20 UTC Kiwamu and Peter done
23:31 UTC Alfredo and Elisabeth done
23:36 UTC Starting initial alignment

Changed smoothing factor on CP4 cyropump level from 0.999 to 0 to see if it helps with sporadic noise causing alarms at operator console.

3:08 pm local

Took 39 min. to fil CP3 at 50% open on LLCV. Raised nominal level from 17% to 19%.

Summary:

OPLEV EX laser glithes were reported to be causing DARM glitches by detchar (e.g. alog 31810) even though EY is no worse or maybe somewhat glitchier (1st attachment, this is a week's worth of trend before Jason made an adjustment). It seems like this is due to a hidden design feature that there's a fixed 0.1uf cap across transimpedance resistor, which gives different pole for different resistor.

As a quick fix I'll insert digital zp(160,16) for all EX segments, as it seems to me that we cannot change the transimpedance without accessing the receiver module on oplev pyron.

Details:

T1600085 tells us that the effective transimpedance and the whitening gain of ETMX and ETMY are [2*10k,18dB] and [2*100k,0dB] (a factor of 2 comes from the differential drive, physical transimpedance is either 10k or 100k). They both use two stages of whitening filters too.

D1100290 shows that there's a 0.1u cap across the transimpedance resistor. The pole formed by 0.1uF cap and [10k, 100k] resistor is [160Hz, 16Hz].

Analog signal of each segment including whitening gain is

ETMX ~ 8*2*10k*zp([],160Hz)*photocurrent (a factor of 8 due to 18db whitening gain)

ETMY ~ 2*100k*zp([],16Hz)*photocurrent.

DC level is about the same but ETMX has an effective zp(16,160) whitening relative to Y. Look at RIN (second attachment) and PIT and YAW signals (third).

Residual power glitch that isn't canceled out by power scaling goes into the oplev PIT, and the effect is  worse for ETMX due to this difference because X and Y has the same damping filter with similar gain. For f>300Hz the PIT signal is limited by the electronics noise, so the power glitch at this frequency directly goes to oplev damping without any cancellation.

Inserting zp(160,16) for all EX segments will, for the moment, make EY and EX about the same.