After a long morning, I was able to check out HAM6's sensors, re-gap a couple CPS's, float the ISI, adjust lockers and plug everything in. Surprisingly, no locker shim adjustments were needed (every aLIGO ISI has needed small adjustments) and all other adjustments went smoothly. No major issues, but the corner 2 and 3 GS-13's are swapped. I didn't check to see if the in-air cabling at the chamber is swapped, yet, but I'm pretty sure of the in-vac cables. I've attached power spectra for CPS's and GS-13's, as well as an MEDM snapshot.

After many iterations of testing to narrow down the problem.  It would appear the with the SAT AMP attached to the Coil driver was when the oscillation appeared in the OSEM readbacks.  Putting the Test box in place of the SAT AMP did not show the problem.  Odds are it was an oscillation on the power that caused a problem on the PD circuit.  After shutting off the Coil driver and re-installing all of the cables there is no oscillation.

Here is the current status of the aLIGO SEI work regarding the HAM chambers. Everything in red is a change from last week's status. Everything in green is available.

Thursday, 25 July 2013 - The Manifold Cryopump Baffle was installed in H1 End-X!!! (pic 7)

The baffle was lifted and positioned between the Spool and the Gate Valve. It was a tight fit; there were inches of clearance (pic 3). We were stopped once to move the cable trays on the top of the Spool for the Assembly Fixture to insert the Baffle (pic 1 & 2). The Baffle was inserted into the Spool with less than 1 inch clearance around the Baffle (pic 4) and it went in SMOOTH! There were 2 men in chamber. After a few trial attempts to attach the Flexure Rods, the L-Brackets were removed to facilitate the install. Once the Flexure Rods were attached, the Assembly Fixture was removed. One blade released as expected and the z- was set with the Spacing Bar. There was some twist which was corrected at the Flexures. Final measurements to the end of the Spool have the baffle position ~3.25" on the East side and ~3.5" on the West side. Baffle rest 0.5" spacing to the Compression Rings all around. The Copper Plates with o-rings were attached and spacing to magnets set. The cut-out around the Ion Pump (pic 5) looks really good.

The problem of attaching the Flexure Rods will be discussed to see if we can reduce the amount of effort needed for future installs (pic 6).

All photos are available in Resource Space - https://ligoimages.mit.edu/?c=1346&k=434c8ef92c
(Resource Space has pictures of the life of the baffle starting at oxidation, to assembly, ending with installation, but not in any order.)

Many thanks to those who participated in this effort -
Apollo's Bubba, Mark, Randy, Scott & Tyler, Doug Cook, Joe DeRenzis, Jodi Fauver, Gerardo Moreno, Mitchell Robinson, Scott Shankle, Mike Smith, Chris Soike, Calum Torrie, Mike Vargas, Thomas Vo, Nichole Washington, John Worden and the entire Hanford Family!

Looked at electronics for HAM6 Cap Position Sensors. Hugo and Jim reported signals were not stable. 
When only corner 1 and 2 were powered up, signals behaved as expected. When corner 3 was powered up, all signals became unstable, some jumping about -2000 counts every few seconds. Looked at electronics in CER but could not find any issues. Disconnected CPS field cables in CER and placed a dc voltage on all inputs. Looked at outputs and could not find any offsets or oscillations. Went out to the floor and disconnected the following:
All grounding cables
CPS sensors going to the satellite racks next to the chamber
timing sync cable
Communication cables going back to CER
power cables

Reconnected all cabling, and looked at looked at signals with Hugo. Everything was within range, and seems to be responding. We are suspecting we might have had a grounding issue. 7-25-2013

Curious as to why the BSC-ISIs had tripped last night, I used the awesome "plot last WD trip" feature on the ISI watchdog screens. They had tripped at the following times:
ETMY: Jul 26 2013 06:02:08 PDT (Jul 26 2013 13:02:08 UTC, 1058878944)
ITMY: Jul 26 2013 06:02:10 PDT (Jul 26 2013 13:02:10 UTC, 1058878946)
BS:   Jul 25 2013 18:52:17 PDT (Jul 26 2013 01:52:17 UTC, 1058838753)

The HAM-ISIs and all HEPIs survived the night.

J. Kissel, H. Paris, A. Pele, B. Shapiro

After spending all afternoon getting all the chambers up into their best (local) performance, I've used Stefan's instructions to lock the Y ARM on Red. So easy -- even I could do it! Thanks to all of those who've written scripts to automate the ALS and IMC :-). Interestingly, and I forgot to ask about it, but there's no need to run any down script when the lock is lost. One just runs the two scripts again (assuming your Beam Splitter alignment remains good).

OK, I wrote the aLOG too soon. There have been several lock stretches, and the ISI-BS has tripped. I detailed time line is written below, for as long as I stayed here.

Here's the configuration of the SEI/SUS during these stretched:

Cavity Lock: 
Chamber    HEPI                                   ISI                               SUS
HAM1       Locked                                 n/a                               n/a
HAM2       Floating, Alignment Offsets Only       Level 2 Isolated                  MC1, MC3, PRM damped Level 1.5; PR3 damped Level 1.0
HAM3       Floating, Alignment Offsets Only       Level 2 Isolated (eLIGO Blends)   MC2, PR2 damped Level 1.5 
BS         Level 2 Isolated (Position Locked)     Level 3 Isolated*                 BS damped Level 2.0
ITMY       Locked                                 Level 3 Isolated                  ITMY damped Level 2.1
ETMY       Level 2 Isolated (Position Locked)     Level 3 Isolated                  ETMY damped Level 2.1, TMSY damped Level 2.0

* I noticed ISI-BS had tripped around 2:53 UTC, but it may have been down for some time. I didn't bother bringing it back up, 'cause I didn't want to blow the cavity lock. Or another three hours.

All Optical Levers are well centered.

All BSC-ISIs have been brought up to the configuration outlined in LHO aLOG 7226, but just for posterity, this means:
Level 3 Isolation Filters
Blends at ST1 "T250mHz" and ST2 "250mHz"
GND to ST1 STS2 Sensor Correction is ON (The corner station ISIs are both using the beer-garden STS2)
ST1 to ST2 T240 Sensor Correction is ON
ST0 to ST1 HEPI L4C Feed-Forward is ON (with "FF01_2" filters)
The Input Gains for the L4Cs and GS13s are OFF (I *think* this means they're in low -gain mode). They're ON for the T240s.

----------
Time Line (all times UTC, Jul 25 2013)
2:27 Locked on Red
2:46 Lost Red Lock
2:47 Regained Red Lock
    2:50 Glitch in CARM_IN1!
    2:53 Noticed ISI-BS had tripped. All other ISIs are still fully operational
    2:57 Glitch in CARM_IN1!
3:00 Lost Red Lock
3:06 Regained Red Lock
    3:11:23 Glitch in CARM_IN1! 
    3:24:47 Glitch in CARM_IN1!
3:27 We begin to ignore the IFO and go home, be cause there's enough data in the can to get a 0.01 [Hz] measurement in the past (assuming those glitches don't spoil the spectra)... but may the lock last through the night!

[Richard Dave Jeff Brett Arnaud]

Following aLog 7055

Today and yesterday we tried to understand why the beamsplitter suspension looks to be moving so much. In order to see higher frequency contents, we took a spectra with this time, test points channels (sampled at 16kHz). A large peak appears at 1900Hz for M1 top mass osem (2nd plot of the attached pdf) and M2 middle mass osem (3rd plot), as we saw few weeks ago on the TMSY after the power outage (see aLog 6338). The first and last plots are showing respectively the BS ISI motion (X and Y direction projected at the suspension point calibrated in um) and etmy suspension osem top mass signals, and none of them are showing such a feature.

To solve this, we tried fruitlessly several things

- power recycled IO chassis and front end h1susb123

- unplugged/replugged cables between satellite box/coil driver/AA chassis

to be continued [...]

While trying to get he ISI-BS up and running (to either Level 2 or Level 3) we saw instabillity whenever turning the isolation loops on, and has been so for most of the week.

Judging by the high DC content of the supersensor signals coming out of the Blend filters, we deduced that there was some offset that was (not?) accidentally applied.

The CPS typically have some level of bias voltage, we compensate for this with a digital offset, so the error point of isolation loops -- which are DC coupled -- is small. Those offsets used to be set in the local basis. We are preparing (see E1300548) to roll off a new update to set those offsets in the cartesian basis. The latter way allows to keep track of the platform's alignement over time, in the basis of interest.

Looking more into the ISI-BS screens, we found that offsets were set in the "current" block of the blend filters. Those offsets are the equivalent of the upcoming update's "cartesian biases", and were probably set there to get the benefits of the update earlier. However, some BURT restore had left the local basis offsets on as well.

Having both offsets on at the same time, created a huge DC offset at the input of the isolation loops, which the actuators had a very difficult time pushing to zero, and eventually causing the (or many) watchdog(s) to trip, usually very early in the ST1 turn on process.

Local CPS biases were turned off. It was then possible to turn on Level 3 isolation loops on ISI-BS. 

One can still determine the alignment of the ISI by looking at the input to the CPS input to the blend filters.

I thought this was fixed by removing one of the dust monitors in the H1 PSL enclosure (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=7170), but the errors have returned:

...
Error: ../commands.c: 49: Sent command � not echoed
Received �
...

J. Kissel, H. Paris, A. Pele

0) Identify the source of the trip. Take a look at the SUS WD screen, and see what path tripped the watchdog. Take a look at the ISI performance matrix, be sure you don't see tons of red. Take a look at the OSEM speedometers, make sure they're not swinging wildly.

Reset and restore SUS
1) Ramp down offsets, by hitting the "Pitch and "Yaw" button after the purple "ALIGNMENT OFFSETS" box.
2) Untrip the inner most watchdog (for the QUAD, you'll need to reset the reaction chain as well!)
3) Untrip the USER DACKILL Watchdog
4) Untrip the IOP Watchdog
5) Damping loops should automatically turn on (they're never turned off, really), but turn on the alignment offsets again.

Reset ISI
1) Ramp down FF01 filters, by setting the gain to 0 (only FM2's "FF01_2" in X,Y, and Z are on during Level 2)
2) Though the green arrows, lines, and blocks may be deceiving, make sure that the FF12C and FF12 banks are also not outputting anything either.
3) Turn off GND to ST1 and ST1 to ST2 sensor correction. Open each "SENSCOR" Bank and ramp the MATCH gains to zero, and turn off the outputs.
    (It's a 60 second ramp, so be patient.)
    (These will only be on if the previous user wanted them on, don't be alarmed if they're already off).
    (I say turn off the outputs because that's what makes the lights on the overview screen go red. Obviously, you don't *need* to do both.)
4) Run ctrlDown ${OPTIC/CHAMBER} from commands window
5) Untrip "Rogue Excitations" WD under ERRMON screen linked in bottom right corner
6) Untrip ST1, ST2, and USERDACKILL Watchdogs but hitting "reset all" (if that doesn't untrip the DACKILL, open up the screen and manually hit reset")
7) Damping loops should automatically turn on (they're never turned off really).

Reset and restore HEPI
1) Ramp down MASTER Gain (or just set to 0.0, since the WDs are tripped and nothing's going out anyway)
2) Run ctrlDown from commands window
3) Untrip inner most watchdog
4) Untrip USER DACKILL Watchdog
5) Slowly (!!) ramp up offsets. I use the follow ezcastep command, in absence of a script:
ezcastep 'H1:HPI-BS_MASTER_GAIN' '+0.01,100' -s '0.5'
ezcastep 'H1:HPI-BS_MASTER_GAIN' '+0.01,100' -s 0.5

From here on, you can follow Vincent's instructions starting at step 3, which I'll repeat here and add a little myself:


3) Turn on HEPI
     ITMY: locked. Don't worry about it. 
     BS: can only be Level 2 isolated, the IPS blends are not installed, so the command script for Level 3 is unstable. Select Level 2  for BS from the command window.
     ETMY: We have only tried Level 2. It's unclear whether Level 3 works. Select Level 2 for BS from the command window. 
4) Isolate ISI
  - Ramp up ST0 to ST1 feed-forward in X,Y and Z on BSC-ISI (there's no RX, RY, or RZ FF. This is the "FF01" bank at the top).
     - Start with Gain = 0, inputs ON, outputs ON
     - It's unclear whether FM2 "FF01_2" is associated with Level 2 isolation and FM3 "FF01_3" is associated with Level 3 isolation, and we've found that FM2 "FF0_2" works for both, so for now, we'll stick to FM2 for either level of isolation. Turn it ON
     - Making sure there's a >5 sec ramp, ramp gains to 1.0 (order and speed doesn't matter).

   Turning on the isolation loops for Level 2 and 3 are a little bit rough for the highly-sensitive inertial sensors (though the motion is not that). One must increase their watchdog thresholds during the turn on process in order for the watchdog not to trip during the turn on process. Good thresholds are: 
     - T240s: 200000
     - L4Cs: 30000
     - GS13s: 40000
Such that the system is still protected, tighten the thresholds on the CPS to 15000.
   - Make sure you have blend filters set to "750 mHz" for ST1 and "250mHz" for Stage 2, using the "SWITCH ALL" button on the top middle of the blend filter screens. We were only able get ISI-ITMY to Level 3 with ST1 set to "250 mHz."
   - Hit Level 2 or Level 3. You must be very patient, and be sure you've follow all of the above instructions to a T, or one, a few, several, or all watchdogs will trip on you and it's back to step 1. Budget 20 minutes.
   - IF THE WATCHDOG TRIPS DURING THE ISOLATE PROCESS, GO BACK to "Reset ISI" Step 4).
   - After waiting some time (a few minutes), the T240s will settle down to within +/- 2000 [cts] on the input to INPUT FILTERs. 
   - Switch the ST1 blends to "T240mHz" in all DOFs.
   - Note that if you're browsing through filter banks (specifically the ISO bank, but others as well), there will be several filters that are empty that the script will turn on, raising your suspicion when things go awry. This is such that the controller turn on process can be generic, don't worry about it. 

Stop Here if you don't want/need sensor correction or green cavity offloading

6) HEPI Green Arm Locking Offload -- on H1:HPI-ETMY only
This is only necessary if you want to keep the arm locked with the green ALS on the time scale of hours. If you don't need the green for long, or if you've switched to red control of the cavity, you don't need this. However, if you do, everything in HEPI is on as it should be. The way to turn it on in the LSC screen: SITEMAP > LSC > Overview > look for "YARM" in the bottom middle of the screen. Open the bank and turn ON the output.

7) Sensor correction
It's unclear (at least) if this is commissioned well, the low frequency performance of the cavities have not really been characterized with it on (i.e. they saw it cause excess low-frequency once, via the camera, and said "don't bother" and it stuck). However, to bring it up, 
    GND to ST1 (from GND STS2s) sensor correction:
    - In the "GROUND" bank of the ST1 SENSCOR filters,
    - Make sure the "MATCH" filter is on (it's just a ~few percent gain correction filter), ever other 
    - Turn on the output of the "MATCH" bank 
    - Make sure there's a very large tramp in the MATCH bank
    - Ramp the gain to +1.0
    ST1 to ST2 (from T240s) sensor correction:
    - In the ST2 SENSCOR filters,
    - (No MATCH filter)
    - Turn on the output of the "MATCH" bank 
    - Make sure there's a very large tramp in the MATCH bank
    - Ramp the gain to +1.0

I moved the list of current dust monitor locations from the whiteboard in the control room to the wiki here: https://lhocds.ligo-wa.caltech.edu/wiki/DustMonitorLocations

Please update it if you know that a dust monitor has moved.

(Jeff, Kiwamu, Stefan)

With the H1:LSC-REFLAIR_A_RF9_I calibrated in Hz, and the open loop transfer function measured, here is the noise it sees: Input Mode Cleaner transmitted frequency noise.
Also plotted is dark noise (shutter closed).

We do not know yet what the ugly noise ~1/f^3 noise is.