Today, we looked for ground loops on ISI-BSC8. We found 3 ones (H2:SEI_BSC8_GS13-2, H2:SEI_BSC8_GS13-3, H2:SEI_BSC8_CPS-2 cables).

The connector of H2:SEI_BSC8_CPS-2 at the interface chassis was opened to fix the grounded back shell.

The ground loops on BSC-ISI GS13 cables are probably created by the relief screws that attach the connectors of the in-vacuum cable to the feedthroughs. The problem seems very similar to the one observed on BSC6 last Friday (but this time, the screws are not tight at the feedthroughs but there are at the pods). Tomorrow morning, the screws will be tightened down. Hopefully, it will solve the problem.

Benno W, Rick S, Michael R

We swapped out the ISS PDs with our spare set to correct for noise we were seeing in the out of loop PD. This swap helped, but we were still seeing some noise, so we tried disconnecting cables, tightening the screws holding the ISS box to the table, which all helped. We were able to lower our out of loop PD noise to below what it was after installation. In this configuration, the QPD and DC output cables are disconnected from the rack outside the PSL and from the ISS box. Currently we have the QPD cables disconnected from the rack, and PDB DC is hooked into the outer loop PD input. Additionally, we were able to get rid of a peak at ~78 kHz by adjusting the gain from -1.7 to -2.1. UGF is 46 kHz with phase margin of 32 degrees.

We adjusted the aligment of the DBB to lower the higher order modes. After optimizing the aligment, we realized the power on the RPD was too low, and compensated by adjusting DBB_M3 to get more light.

Plots (in order they appear):

DBB mode scan on 3/7 (pre-alignment)

DBB mode scan on 3/9 (post alignment)

ISS rpn on 3/7 - old PDs, QPD cables connected to ISS box and rack, DC output cables connected to ISS box only

ISS rpn on 3/9 - new PDs, QPD cables connected to ISS box and rack, DC output cables connected to ISS box, PDB DC connected to outer loop input, cover on, all screws tightened

ISS rpn on 3/9 - new PDs, QPD and DC cables disconnected from rack and ISS box, all screws tightened

In the previous week's boot fests, it unclear when FMY was taken down, but it had been left in the "empty" state (no matrix elements, no gains, no offsets, etc.). I have BURT restored it to a nominal working state, using
/opt/rtcds/userapps/release/sus/burtfiles/fmy/h2susfmy_safe.snap

Note, after "turn on" (which doesn't really turn anything on), I did my usual sanity checks:
- Is the master switch off and the watchdog tripped? Yes, PASS
- Checked the input and output matrices; do the number's make sense? Yes, PASS
- Checked the OSEMINF banks; are there reasonable offsets and gains? Yes, PASS
- Checked the alignment screens; are speedometers for centered and not going crazy? Yes, PASS
- Checked the BIO screen; are we in State 1, with the monitors indicating our request? NO FAIL

Tried flipping the state around (on both stages, entering in State requests for 0, 1, 2, etc). According to the readbacks (or "monitors") the TOP/M1 stage's low-pass filter is switching appropriately, but the TEST/COIL enable switch does not toggle. The MID/M2 stage shows nothing flipping, which is usually an indication that the coil driver boxes themselves are power down.

Given the stop-work order, this does not surprise me. If we intend to move on and resume work, we should look at these analog drivers to be sure they're ON.

This morning, we finished mounting the cable lacing brackets on the PenRe and the UIM in order to facilitate the non/grounding issues (disassembled brackets last Fri).  After we were done, we observed some pitch in R0 so we had to play more with the brackets and cable routing.  We've left the suspension fully hanging (free).  TMS is up next to remove EQ hardware, followed by SEi for unlocking, then on to TFs.

Today, Dan Hoak and I finished checking all the cabling associated with H2 ETMY.  The summary is:

1.  All cables (SUS, ISC, SEI) going to the vacuum system, and inside the vacuum system, are now properly functioning with respect to grounding and shielding.  This statement refers to in-vacuum and in-air cables with respect to shields only (not internal pin functions which have been largely exonerated by subsystem checkout).

2.  There are two L4C cables (L4C-2 and L4C-3) that show an in-vacuum short from shield to ground.  We believe this is due to the strain relief screw not being tightened down yet.  This is not a cable flaw by itself as the screw is supposed to be tightened, and will be tightened at final install time (per Jim), it's just something to put on the final checklist.

3.  Every shield associated with the ring heater drive cables is grounded at end closest to the ring heater.  This is due to the way the metal collet that terminates the shield is attached to the structure.  This does not present a problem to any other neighboring subsystem in terms of noise, but may have implications on the 60 Hz content of the ring heater drive spectrum.  A design change is needed to fix this problem if it is deemed worth fixing (I don't have a sense as to whether it is possible to pickup enough energy into a heater connection to actually move an optic or conversely to radiate enough energy from the ring heater drive to bother another subsystem.  My instinct is it's not a huge deal).

4.  I noticed that the RTD used to sense local ring heater temperature is not attached to anything.  It just floats near the ring heater on the end of a length of wire.  It would seem better if that sensor were attached somewhere to make the readings more meaningful and repeatable.

Here are some details of measurements taken today on ISC, SUS, and SEI cables:

1.  The SUS cables going to the chamber from the satellite boxes were disconnected from each satellite box and a resistance check was performed from the backshell to ground.  All connections are free from ground contact.

2.  The same SUS cables were then checked to see if pin 13 of each connector was connected to the cable shield as it is supposed to be.  Two cables were found (SUS-1 and SUS-19) without a connection.  We later found out that this was due to poor workmanship on the cable construction as can be seen from the attached photos.  The SUS cable shield connection is made by pinching a short piece of wire precariously between the two halves of the backshell.  This is not a reliable connection.  The ISC cable has a soldered connection from the backshell to the cable shield rendering it far more reliable over time.  We fixed the SUS cable problems by better positioning the shield wire connection between the two halves of the backshell, but it calls into question the reliability of the shield connection for this type of cable.

3. Verified that all cables incident upon the air side vacuum feedthroughs have no connection between the backshell and shield.  A connection here would ground the cable shields once they plug into the vacuum feedthroughs, it would not be apparent until the mock feedthroughs are removed because the mock feedthrough uses a short length of unshielded ribbon cable to achieve the needed mirroring of the pinout.  

4.  For all SUS and ISC cables, we verified that once the airside cable is disconnected, the inside vacuum shields are isolated from ground.  The SEI cables were similarly verified yesterday to be satisfactory with the exception of L4C-2 and L4C-3 as previously noted. 

Attached are plots of dust counts > .5 microns.

Today Betsy fixed the ground points in the suspension wiring where the metal backshells of the connectors serving the UIM and PUM were grounded by the saddle clamp.  I will check to see the status of the SUS shield grounds tomorrow.

Vincent and I worked through the ISI cable shields for the L4Cs, GS13s, Coil Drives, and Trilliums.  We found two anomalous grounds, one on L4C-2 and one on L4C-3.  Both of these grounding points were traced to the connectors at each L4C can the vacuum system.  If the connectors are unplugged from the L4C's, then the grounding issue goes away.

I speculate that the ground is caused by the fact that the strain relief screws that attach the backshells of the connectors to the L4C can are bridging the gap between the can and the metal backshell of the connector.  If these screws were screwed in (as they should be prior to inserting the cartridge), the problem may go away.  It's near impossible to get up high enough to test this theory at present, so the final verification will have to be done during the installation of the cartridge when the connectors are more accessible.

The good news is that so far, I have not seen any instances of the cable shields casually contacting the ISI structure as they route all over the place.  There are numerous contact points, but the outer black PEEK weave seems to reliably prevent the shields from grounding.  I was more than a little worried that the larger sample size of cables and contact points would reveal a weakness in my small statistics of reliability.

Following advice from Fabrice and Callum, I and Dan rotated them 90 degrees and put them in a (hopefully) correct direction.

We didn't use thick washers between the absorber flame and the base plate, partly because it would have been a pain to do so, partly because the screws available were somewhat short (it didn't go all the way through the base plate) and using washers would have reduced the number of screw threads that were engaged.

The ISI, HPI and TMS models were fixed and now all the H2 frontends with the exception of the PSL are running RCG2.4.2.

We found a DAC/ADC reconfiguration problem on h2seib8, which Rolf is investigating. I had some timing issues at EY when restarting the SUS frontends, which I cannot reproduce.

SUS FMX will not compile, I've taken it out of the DAQ for now.

X1 HAM-ISI Simulink model does not have more than 3 ADCs. Earlier today, CDS team confirmed that it was safe to run unattended excitations on HAM-ISIs, in the staging building.

A quick Transfer Function measurement (2h) was performed on HAM-ISI Unit t#2 today. DAC output channels were monitored. No abnormal behavior was observed.

Transfer function measurements, and their excitations, are running on HAM-ISI Unit #2 over the weekend.

Today, Travis, Jason and I finished extracting the ITMy lower QUAD structure.  There was a small issue with the elevator controller unit in that it failed to lower all the way.  We addressed this issue by swapping it with the controller from our second elevator setup (having backups is a VERY good thing).  We also encountered some interference between the optic stop "fingers" and the arm, resulting in removing the fingers altogether and reseating the elevator on the QUAD.

The QUAD was removed from the BSC repair arm and inserted into a lower structure storage container for it's flight back to the fiber welding cleanroom in the LVEA.  Currently, it resides in the storage container in an anterior cleanroom awaiting disassembly and a decision on the path forward.  Dale Ingram documented most of the extraction process by camera and his pictures/videos should somewhere sometime.  

We continued our investigation by disconnecting everything under the ISI table from outside.

As it turned out, all TMS and ISC cables are fine, but using DVM with alcohol-wiped tips we identified problems in SUS BOSEMs and TCS ring heater:

1. L2 (penultimate mass) BOSEM cable shield,
2. L1 (second mass from the top) BOSEM cable shield,
3. Ring heater cable sheild

are all locally (i.e. in-vac) connected to the metal structures, and if you look closely it's really obvious (Rich said "How can this NOT be grounded?!").

ETM/ERM:

The first picture shows the cable connector clamp on the penultimate mass. Can you see that the clamp doesn't have any insulation? It's just two metal plates sandowitching the metal connector shell. Of course the penultimate mass is electrically connected to ISI via suspension wire.

Also, a pinching clamp for the cable (only the bottom half of which is shown at the top of the photo) doesn't have insulation, though the cable itself has an insulation mesh.

I couldn't take a good picture of the L1 connector clamp, but it's the same design.

This is a serious ground loop right there for SUS (but not for ISC and TMS) and we should discuss any possible fix.

TCS:

The second picture shows the elaborately designed cable bracket thingie that directly connects the ring heater cable shell to the mirror cage.

ISI:

We haven't disconnected ISI cables simply because there are many,  and we know that the ISI cables are going to be revisited after the cartridge insertion. But somebody should do that to see if there's some funny thing is going on like SUS and TCS (and ISC/TMS before we fixed one clamp problem).

We just looked around and saw two cable shield rings resting on metal bolt of ISI (third and forth picture).

Attached are plots of dust counts > .5 microns.

Rolf and Dave

Report on ISI driving of ITMY March 1st 2012

The problem is introduced by the addition of a fourth ADC into the hepi/isi models. There is a hard limit set in the code for the decimation filtering history arrays to only support up to 3 ADC modules. The addition of a fourth caused the ADC decimation filtering history to write into the space used for DAC upsampling filer history data, thereby causing some form of this ADC input to be output to the first DAC module. I believe this has been in the code since the early blue box days. While other arrays used by ADC/DAC data have been updated to use MAX_ADC_MODULES and MAX_DAC_MODULES some time ago to set array sizes, as defined in cdsHardware.h (presently 12 ea), these history arrays apparently got overlooked. This hard limit would not effect IOP tasks, or any models running at 64K, as they do not perform up/down sampling.

a new rcg tag 2.4.2 was created as a copy of 2.4.1 with the MAX ADC/DAC substitution installed. All LHO models were recompiled and restarted against 2.4.2. with the exception of:

h2susb6: h2sustmsy model would not compile, missing links. Running 2.4.1 on this front end for now

h2seib6: would not reboot, needs further investigation

h2psll0: hold off until friday, running 2.4.1 on this front end for now

[Continuing from Corey]
- Retractable arm inserted on BSC8, H2ITMY Quad removed.
- Many high (10k-30k) dust count alarms in the Biergarten, but the offending dust monitors could not be located. The only dusty activity during much of this time was vacuum cleaning in the area and retractable arm installation by the door of BSC8
- CDS worked on cornering the H2ITMY fiber-breaking issue. Many reboots of h2seib6, h2seib8, and the entire DAQ as code and models were updated.
- H2 PSL and CDS are working into the evening.
- OSB doors locked at 1630.

The SEI Engineers at MIT were a bit concerned about letting the horizontal actuators take the full shear of the vertical motion when the Quad payload is removed.  So, I disconnected them.  Almost completely that is...  The SE unit has a bolt that I fear is galling.  Everything else that can has been loosened and there are slots so we feel this will be OK.

Pods #94 and #71 were experiencing the problems laid out in alogs 2308 and 2321. So today I de-podded both of them. During the de-podding I noticed no issues with the feet or any other kinematic devices.
I did find a broken flexure on #71, however (pictures attached). Which is odd since the GS-13 tested fine on arrival. Somewhere in between its arrival and testing it completely failed. It's possible it was severally fatigued during shipping and only recently failed.
Pod #94 has had its brains swapped with another failed GS-13 (mechanical issues) and will hopefully be tested sometime tomorrow to see if that solves the gain issues. If not that should point us toward some mechanical problem that I haven't seen before.
Both of these pods will become test pods (if their issues can be fixed) until they can be sent to LLO for retesting.