Values in the system manager are flat lined. There are multiple lines of errors in the EPICS IOC of the following (screen shot attached):

Error code 4: ~~~MYSTERY ERROR!!!~~~ Go Google "ADS return codes"!

and

Error code 1861: ~~~MYSTERY ERROR!!!~~~ Go Google "ADS return codes"


Googling "ADS return codes" gives:

4
Insert mailbox error
No ADS mailbox was available to process this message.
Reduce the number of ADS calls (e.g ADS-Sum commands or Max Delay Parameter)

1861
timeout elapsed
Check ADS routes of sender and receiver and your firewall setting

Recovered BSC1, BSC3, BSC9 and BSC10 after ITMX, ETMX, ITMY, and ETMY SUS models restart. 

We measured the WFS sensing matrix yet again. 

The green ones are consitent acros the measureemtns we've made to within a factor of 2.  ETMX PIT is the most consistent WFS B pit signal, the sign has stayed the same but the gain has changed by a factor of 3 .

We have some new settings for the Comm Handoff, so I repeated the usual measurements:

COMM PLL Servo Board

• Input gain: 27dB
• Common Fitler 1: ON
• Common Filter 2: ON
• VCO Comp: ON
• Low Pass: ON

CM Servo Board

• Input 1 gain: 11dB
• Input 1 pol: POS
• Input 2 gain: 6dB
• Input 2 pol: NEG
• Common Comp: ON
• Fast Gain: 6dB
• Fast pol: POS

LSC-CARM Filter

• Gain: 80
• FM1, FM4, FM5, FM8

LSC-REFL-SERVO-SLOW

• Gain:1
• FM1

MC2_M3_LOCK_L

• Gain: -300
• FM9

MC2_M2_LOCK_L

• Gain: 0.10
• FM3, FM4, FM10

MC2_M1_LOCK_L

• Gain: -1
• FM1, FM2

MC2_M3_ISCINF

• Gain: 1
• Filters OFF

1. Common path OLTF: Handoff_COMM_OLTF_mag/phase.txt
2. Inject into fast path, cf/1-cs OLTF: Handoff_cfast_OLTF_map/phase.txt
3. Inject into slow path, cs/1-cf OLTF: Handoff_cslow_OLTF_mag/phase.txt
4. Crossover using LSC_CARM inputs: /ligo/home/sheila.dwyer/ALS/HIFOX/COMM/Crossover_April1.xml

The UGF of the comm handoff is now 3kHz with 80 deg phase margin. The crossover between the fast and slow path is at 13 Hz with 19 deg phase margin.

The gds tools (awggui, diaggui, diag, etc) have been updated to 2.16.12 to provide access to the dmt servers using dmtviewer.

The dmtviewer program connects to dmt servers specified by the DMTWEBSERVER environment variable.  Since the dmt servers are not part of the cds system, a kerberos ticket is required to access the servers.  The command "kinit username" where username is your LIGO.ORG username will give you a ticket.  Don't forget to use the command "kdestroy" when you're finished to remove the kerberos ticket.

Chris, Sheila

Our measurements of the COMM noise with the WFS locked have an rms of 20Hz, measured down to 0.1 Hz.  To make sure this was reasonable, we turned off the refl bias path so that the laser was locked only to the green beat note. The transmitted power fluctuations are noticably less than the HWHM of 40Hz, so this seems believable. (StripTool is the second screenshot attached). 

We wanted to try locking the arm half way down the fringe so we could make a measurement of the noise using the transmitted power.  Since the IR trans PD is normalized to 1 when the cavity transmission is maximized, the calibration of this signal is just a high frequency gain of 84 (FWHM of the arm cavity), and to correct for the refl bias loop gain we have a pole at zero and a zero at 3Hz.  The correct calibration has a more complicated frequency dependence, so we would have to do the real calibration to make a comparison above about 42 Hz.  Neither of these traces have the cavity pole at 42 Hz corrected for.  You can see that the side of fringe measurement agrees with the normalized refl PD signal well up to 100 Hz . (1st screen shot attached)

We repeated the measurement in the normal configuration, where we lock on resonance using the refl bias path, and the noise is the same (30Hz rms down to 0.02Hz). The RMS is dominated by the pitch mode, so we will work on increasing the WFS bandwidth tomorrow. Tonight we tried OpLev damping (this added noise around 1 Hz) and turning up the pitch OSEM damping gains (inconclusive so far). 

We redid the measurement with the WFS outputs held, and saw that the noise was high again (100Hz rms down to 0.1Hz), with the WFS on again we were back to the low noise state. So we can safely say the WFS are helping.

We also checked that the noise is the same from 3-100Hz with and without the refl bias path engaged. 

Arnaud, Apollo, Thomas

We made a lot of progress on the ETMY OL, the receiver is 90% finished with only a cable left to run from the whitening chassis to the QPD but we've confirmed all the mechanical components fit well together.  The transmitter pylon ended up being too close to the viewport to fit all the parts (nozzles, adapters and bellows) so we need to move it back a few inches to work but that requires moving a pipe bridge, Apollo is going to start on that tomorrow morning. We are still on track for a full working system by tomorrow night, which includes calibration and whitening.

Alexa, Sheila, Keita, Fred, Chris

This morning after we moved the WFS, Alexa remeasured the sensing matrix and got (in counts/urad):

We also checked the phasing by injecting a signal to the laser frequency (through the PDH servo board) and looking at I and Q from each segment.  Then we struggled to lock the loops. 

Later on we remeasured the sensing matrix again, several times. We saw that the measurement isn't particularly sensitive to the centering on the WFS.  We tried using DTT to do the exicitation, and tried using AWGGUI and letting the excitation run for a few minutes before we took the measurement to avoid any trasients, but that also did not seem to have a big impact. 

Some elements were consistent (within 20-30%) each time we measured and consistent between Alexa's morning measurement and the after noon measurements.  These were EMX Yaw to WFS A (-900counts/urad  average) and ITMX yaw to WFS B (562 counts/urad average) ETMX yaw to WFS B consistently was small and didn't have much coherence, so we have set it to zero.  Each time we measured ITMX YAW we got a magnitude of 700-750, but the phase varied, (136, -134, -38, -21, 19).  We decided to also set this to zero, and now have a yaw matrix that is just: -1.1 WFS A to DOF 1 (ETMX Yaw) and 1.897 WFS B to DOF2 (ITMX Yaw).  This seems to be working OK. 

For pitch we got values for WFS A that are consistent with Alexa's measurement from this morning, but not for WFS B.  For WFS B PIT we got -630 to -853 coutns/urad ETMX pit and 566 counts/urad ITMX pit, a sign flip compared to the earlier measurement. 

Right now we have all four DOFs locked, using the new YAW matrix and Alexa's measurement from this morning for PIT.  This seems to be stable, we pushed the gain up a little, so the current gains are -0.002 DOF 1+2 Y , -0.002 DOF 1 PIT, and 0.005 DOF 2 PIT.  These have been locked with good build up for about 20 minutes. 

I have left 2 excitations running, on ITMX Yaw we are injecting 1 count at 3 Hz into the L2 Lock Yaw filter, and for ETMX 0.3 counts at 1.5 Hz.  Hopefully this will stay locked a good part of the night so that we can look at how stable the sensing matrix was overnight. 

Aidan. Thomas. Greg. Eric G. Dave H.

Today's work

Flow meter issue

Wired up flow meter. The wiring was 

• Terminal Block on Flow Meter "+" connected to +18V (pin 3) on J10-FEMALE on CO2 controller chassis, D1101835
• Terminal Block on Flow Meter "-" connected to ANALOG IN (pin4) on J10-FEMALE on CO2 controller chassis, D1101835

CO2 Controller Chassis

• Bypassed remote On/Off switch from Beckhoff as this wasn't working. Bypass was done by applying a jumper to Header 2 in D1101835
• Plugged in cable 40X from chiller (shuts down laser if chiller reports a fault)
• Bypassed IR sensor input by applying a jumper
• Plugged in flow meter (shuts down laser if flow drops below 2.5GPM - we checked this. Nominal flow is around 3.5GPM)
• Plugged in laser temperature sensor (shuts down laser if temperature of laser exceeds around 30C). There was a suspicious instantaneous jump in the voltage corresponding to the temperature at the time the laser was turned on (and I mean instanteous, not quick). There was a corresponding drop in the voltage of that channel when the laser current was turned off.

Leaky laser was enabled

Laser (SN 20510.20816D) was run. Maximum output power was 49.4W

Beckhoff issues

CO2 binary outputs not working - traced problem to Beckhoff System Manager not mapping CO2 output hardware to PLC variables.

Plumbing

• Further leak hunting on the table. Moderate leak from main feeds onto the table. About 2 litres of water leaked out onto the corner of the table and the floor. A couple of minor leaks on the manifolds were fixed.
• Installed Laser #3 on the table. This one showed no evidence of leaking from the housing when the chiller was running.

Laser Enable Procedure

It actually takes a large number of ducks to be in a row to turn these CO2 lasers on. So here's how to do it ...

1. Ensure that the laser power cables are connected from the Instek Power Supply (Mech Room) to the CO2X table feedthroughs and, inside the table, connected to the RF driver

2. Mech Room: Turn on the chiller. Check flow rate is approximately 3.5GPM and pressure is abotu 60psi on the chiller. Check that H1:TCS-C_CO2_X_FLOWRATE is about 3.5GPM
3. Mech Room: Turn on Instek power supply. Set output voltage to 29.5V. Set the maximum current to ~28A. Press the enable output button. Check that the current drawn is around 350mA.
4. Mech Room: Turn on the oscillator to 40.68MHz, 11dBm. Ensure the output is connected all the way to the laser table, through the feedthrough and into the Sine-To-TTL box
5. Laser Controller: Ensure all inputs are plugged in.
6. Beckhoff: Set H1:TCS-C_CO2_X_LASERPOWERSWITCH to Valid
7. Laser Table: Ensure the ISS Chassis is turned on
8. Laser Table: Ensure the BO chassis is turned on
9. Laser Controller: Select pulse width modulation (PWM) or continuous wave (CW)
10. If PWM: connect the external controller
11. Laser Controller: (Turn Key off). Turn Key on.
12. Laser Controller: Press the Red Gate ON/OFF button
13. If PWM: turn the dial on the external controller to the desired PWM %.
14. If PWM: press the enable button on the external controller.

The success of this can be gauged by checking:

1. The Instek controller current (H1:TCS-ITMX_CO2_PWR_SUPPLY_I_OUTPUT): should be about 20A at full CW. Or > 1A when laser is running at any power level.  
2. The pick-off on the laser head (H1:TCS-ITMX_CO2_LSRPWR_HD_PD_OUTPUT) should rise to about 45-50W (if calibrated correctly).
3. The temperature of the laser (H1:TCS-C_CO2_X_LASERTEMPERATURE) should increase to about 23-24C

Jim Warner, Greg Grabeel

Hugh noted there were some issues with a limited range of motion on HAM 4. Jim and I checked for rubbing but were not able to find anything readily apparent. We replaced the parker valve on the NW corner horizontal actuator. At ~2:45pm we started running the actuator in bleed mode. At ~4:30pm we changed the valve positions to run mode. There were no leaks on the parker valve. I did an air purge on the accumulators shortly after switching over to the run state. Jim will be running linearity and transfer function tests to see if this fixes the issues. 

ETMY ISI tf running from opsws0, should run til mid-morning. HAM5 ISI tf running from opsws1, should run in slightly less time. HAM4 HEPI local basis test running from opsws2, just a couple of hours. Anyone care to join a pool on which ones crash, with extra payout for guessing the reason?

- 9:30 am Travis, Gary, Giles and Margot to the LVEA, West bay monolithic work.
- 10:05 am Dale +1, to the LVEA, tour the LVEA.
- 12:33 pm Aaron to Y-End, check on OpLev cable.
- 12:50 pm Thomas and Arnaud to End stations, X-End to pick up tools, then to Y-End optical lever work.
- 12:55 pm Karen to Y-End.
- 1:14 pm Travis and Margot to Y-End, tool retrieval.
- 1:45 pm Travis, Gary, Giles and Jason to LVEA, west bay area for monolithic work.
- 1:55 pm John to LVEA, west bay area, monolitich work inspection.
- 2:37 pm Andres to LVEA, 3rd IFO parts search and retrive by HAM3.
- 3:22 pm Dale and Margarita to LVEA, LVEA tour.
- 3:22 pm Cyrus to X-End, to retrieve dead imac.
 

Andres R. & Jeff B.

   We made several centering adjustments to the BOSEMs and reset all to 50% light. The latest set of transfer functions show the undamped coupling issues in the 03/25/14 data have been reduced or eliminated. The plots for the last set of TFs are attached below. The plots for the power spectra taken on 03/25/14 are also included.   

I upgraded and rebooted the tape backup machine this afternoon to clear up a problem which turned out to be just the tape head needed cleaning. The strange thing is that I cleaned the head just three days ago, I'm not sure why that didn't appear to take.

I've installled OS updates on opsws0,7 and operator1, and moved opsws0 and operator1 to the workstations subnet.  These were the machines missed in the last round of updates.  All of the control room workstations have now been moved to the new subnet.

With the upcoming TCS CO2 work, we needed to work out any bugs in software land before trying to turn on the laser. A few additions (rotation stage) and deletions caused the old code to not work very well and so we re-mapped the configuration and re-linked the software to hardware.  This solved the problem of getting the hardware channels all the way through to EPICS.

A few more details:
- In order to not trip all the watchdogs in HAM2/3, Sheila turned off the connection between ISC to SUS temporarily as well as putting the MC Guardian in a down->paused state.
- We created a new boot project for PLC3.
- Committed the new system manager to SVN.

A. Staley, S. Dwyer, J. Kissel 

After speaking with Sheila and Alexa, trying to understand the spectrum in LHO aLOG 11026, I realized that -- because of so much unexpected excess noise -- the current ALS setup has diverged significantly from the baseline plan. As such, we've worked together to create a diagram that is to-the-best-of-(my)-knowledge accurate, and makes sure to show how every control signal gets around to its actuators, including passes through digital land. The only things I did not include were the demodulator boards between the REFL diodes (ARM REFL, MC REFL, and the two green XARM and YARM REFLs) and lame things like whitening, AA, and AI chassis that facilitate getting in-and-out of digital land.

In particular, I draw your attention to the unorthodox and new features that were not part of the original design plan:
(1) The PRM is misaligned. This way we can use IFO REFL red diode signal (after normalizing / linearizing it with the red arm cavity trans) to measure the arm and offload at low frequency to the CARM board in the second input. This is unorthodox because normally we would feed REFL straight to the CARM board in all analog and retain the large bandwidth, but because the noise is so large, it must be linearized in the front-end (the only place the Arm TRANS signal exists), and fed out one of the corner station LSC DACs. Note that this configuration is just a special technique for these particular series of measurements and is not designed to become a normal part of the lock acquisition scheme.

(2) The Voltage Controlled Oscillator (VCO) actuated, Phase-Locking Loop (PLL) that's wrapped around the corner-station COMM and DIFF Phase-Frequency Discriminators (PFD) are used to reduce the noise enough to keep the PFD in the linear, phase detecting mode (check out pgs 6-8 of the AD9901 datasheet -- it's a really nice explanation of how it works).

(3) Before the hand-off, there's another temporary step which is to use the Beckhoff system ("Slow ADC" in the diagram) to feed directly to the PSL VCO (again at low frequency) to reduce the noise enough for the COMM hand-off to work. 

This is most certainly an impressively complicated system!

Note that this diagram has been committed into the noisebudget repo here:
${NBSVN}/trunk/HIFO/Common/
and I imagine will become an "as built" DCC document in the glorious future when everything is working beautifully.

This trip happened around the time of a beckhoff restart.  A beckhoff restart causes a crash of the IMC guardian, which causes it to stop.  Its not clear to me why this should cause MC2 to get a large signal, but it seems to.  This causes a cascade of trips.  Even though I don't know why this happends, creating a safe state in the IMC guardian so that it handels missing channels better could help with the problem. 

Another solution is to have the SUS WDs not trip HEPI.  Can we get that fix soon?

HAM2 also tripped at the same time, Jeff and Hugh brought that back. 

Margot and I performed all of BSC10 closeout tasks listed in my previous alog regarding this closeout.  We pulled the FC from the ETMy optic a little after 11am, and the door was going on at ~1:30.  It took "so long" (ha) because we ended up having to spend some extra time calling in the troops about one of the ISC viewports having what appeared to be a scratch on it's inside surface.  More details to come as needed.