model restarts logged for Mon 16/Jun/2014
2014_06_16 14:47 h1nds1
2014_06_16 14:57 h1nds1
2014_06_16 16:25 h1fw1
All restarts were unexpected. These systems relate to the default NDS, which may not be a coincidence.
Summary:
OM2 was rebalanced, but one of the BOSEMs is bad (one of the PD pin is grounded to the tip tilt body). We will replace it with a spare tomorrow.
We found that the shielding of QPD sled picomotor cablesand WFSA DC cable were grounded to the ISI. The former was fixed by loosening the DB25 connector mating screws and pulling the DB25 back a bit. The latter was fixed by loosening an ultra-tight cable clamp at the stage zero of the ISI.
Please do never tighten the cable clamps so hard that the cable cannot move. You should be able to move the cable through the clamp by pulling gently, though the cables should stay in place by friction when left alone.
Details:
I pulled the OM2, rebalanced, installed again, made sure that BOSEM flags don't interfere, set the BOSEM depth, and THEN found that some of the ground loops were back, and one of them was of course OM2 itself (not the cable).
Pulled OM2 again, and it turns out that the coil with a tiny gap that was OK before was touching, causing the grounding issue. By pushing or pulling the connector using my finger, I was able to make or break the connection, and when I remove my pressure it didn't go back to the original state.
I applied aluminum foil as a shim between the coil bobbin and the connector to make the bottom gap larger, and cocked the connector more to make the side gap larger (I needed to make one of the mating screws somewhat looser than the other), so that there is a clear visible gap all around the metal shell of the connector. It was tested good on the work table, so we installed it back to the table.
When we started setting the depth of the BOSEMs, UL coil was stuck at -8000 or so counts (it was not in the morning, but Dan recalls that that coil was stuck at that value on Friday).
I pulled the coil out while keeping the electric connection intact, and as soon as the coil came out suddenly the open value was -32k counts. As soon as the coil touches the ISI surface, it went back to -8000 counts.
It turns out that one of the pins for the PD for the BOSEM is short circuited to the coil bobbin, and of course the coil bobbin is grounded to the tip-tilt frame. In the morning, for some magical reason, it should have been that the PD pin was not short circuited, or the coil is not grounded to the tip-tilt frame.
Anyway, this is clearly a bad part, and we need to replace it with a spare.
Also UR coil open voltage is somewhat small (-20000 cts), so we'll see if replacing this BOSEM changes it.
Installed SUS Watchdog Chassis D1300642 in rack SUS-C6-U26. Unit is powered up, and the only connections made to the unit are the binary inputs/outputs. Filiberto Clara
Started with the 750mHz. It would trip the Actuator with the script so went to hand method. Going dof at a time, was able to get the RZ RX & RY dofs on with boosts and they were pretty quiet. Could get Z on too but it would go unstable quickly when either the X & Y dof was included. Checked the local to cartesian matrices and they are correct. Looked at the parameter file in case the loops were created with incorrect matrices...but I could not find that info in the parameter file so not sure if you can determine that. I did then spot the Blend filter and it said 01_28. So, I changed the blend to 01_28 and tried the script again but it would still trip out quickly. Same story on the 250mHz. However, it is stable with the 100mHz blend...weird.
Attached is a isolated/damped only comparison. The reference traces are with the lvl1 isolation and 100mHz blends. The current rraces only have damping engaged. This isolation does a fair amount of good in various places especially below a few htz but it appears to have resonances and other ill effects at various places above several htz....
- Hugh out to HAM6
- Betsy to Beer garden then to the west bay
- Jeff out to beer garden with film crew
- Jason to ITMX alignment
- Fred and film crew in the LVEA
- Sheila and Greg into the PSL for pictures
While doing the AA/AI rework last week, installed grounding plugs (per L1200193) to all SUS coil drivers at both EY and EX. The plug is attached to the DB9 connector labeled Test Input. This completes all installed electronics in CER, EX, and EY. Filiberto Clara
5 viewports were installed on MCB 2 today. Ref. W P 4673.
Per bugzilla integration issue 463, looked at AA/AI chassis to verify units have the proper insulating film used to isolate the -15V regulators to the chassis metal wall. All AA/AI chassis at EX have been modified or verified to have the appropriate insulating film. SUS S1100321 S1105306 S1105295 S1103819 S1202273 S1202611 S1202819 S1202612 S1202820 SEI S1300132 S1300135 S1300136 S1300140 S1300144 S1300611 ISC (including Whitening chassis) S1102763 S1102781 S1102777 S1101606 S1101587 S1101631 S1101642 All electronics have been reconnected and powered up. All units were placed back in appropriate rack and slot location. Filiberto Clara
Per bugzilla integration issue 463, looked at AA/AI chassis to verify units have the proper insulating film used to isolate the -15V regulators to the chassis metal wall. All AA/AI chassis at EY have been modified or verified to have the appropriate insulating film. SUS S1107601 S1105299 S1105300 S1103818 S1202280 S1103775 S1103772 S1103774 S1103773 SEI S1103403 S1103401 S1103406 S1103409 S1103510 S1103501 ISC (including Whitening chassis) S1102755 S1102764 S1102765 S1101600 S1101638 S1101595 S1101579 All electronics have been reconnected and powered up. All units were placed back in appropriate rack and slot location. Filiberto Clara
Dave reported this morning that TMSx osems readback were moving too much. Looking at a spectra from one of those osems, a 1777Hz line appears to be the main source of oscillation. This looks similar to a problem we had few months ago with one of the sat boxes of the BS (cf alog 7237). We will reboot the sat box and see if that changes it.
The weather station at Mid X has been restarted after replacing an isolator between the weather station and the comtrol serial interface box. Wind alarms have been set to 30 and 40 MPH. End X is online, but the anemometer is not working at this time.
Keita is removing a tiptilt so I locked up the ISI. It was already out of balance and probably should have remained locked anyway. The cables that were moved from stage1 was enough to unbalance the table.
Once Keita returns the TipTilt, SEI will need to rebalance the table.
No restarts reported.
Today the PSL is down for maintainence so that means
Frontend is down
PMC is unlocked
FSS is disabled and ref cav unlocked
ISS at 8% diffracted power with 0 volts on PDA and PDB
- Keita working in HAM6
- Fil and Richard working on AA and AI chassis
- Peter, Patrick, and Olli working on PSL maintainence
- Jodi working on 3rd IFO in LVEA
- Robert going out to the vault
- Gerardo working on CPB
I have created new MEDM screens for the new software watchdog (SWWD) system. These are macro substituted screens, launched from H1CDS_SUS_SWWD_LAUNCHER.adl which is accessed from the SITEMAP via the SWWD option under the WD pulldown. Currently there are two new MEDM screens for EY and HAM6.
model restarts logged for Fri 13/Jun/2014
2014_06_13 11:08 h1broadcast0
2014_06_13 11:08 h1dc0
2014_06_13 11:08 h1fw0
2014_06_13 11:08 h1fw1
2014_06_13 11:08 h1nds0
2014_06_13 11:08 h1nds1
2014_06_13 11:11 h1iopiscey
2014_06_13 11:13 h1iscey
2014_06_13 11:13 h1odcy
2014_06_13 11:13 h1pemey
no unexpected restarts. DAQ restart to support SWWD work, ISCEY to clear DAC error.
Thanks to the efforts of SUS I was able to get in and look at the ring heaters of ITMy today. There are no apparent problems with breakage of either the glass formers or the macor retaining pieces. However, the center macor piece (D1002545) looks to have wandered off center quite significantly. I didn't have any pictures from that angle while the quad structure was still in chamber. So, unfortunately, I am unable to determine when it wandered off, but it definitely warrants looking at ITMx closely before it leaves BSC 3 if possible. It should be noted that this is the older style of ring heater, but the macor element standoff is the same throughout all but the most recent version.
As requested a picture with a few labels, showing how the macor has wandered off the center point where it should be.
The old ring heaters on ITMy have been removed from the old quad structure. While I was packaging the upper and lower ring heater I noticed quite a bit of particulate coming out. Looks like glass, certainly warrants a closer inspection.
Attached is a picture incdicating which connection on the BOSEM is shorted to the body. According to T060233, it's one of the pins on the photodiode (which has the round hole; the IRLED has the slit, inside the BOSEM body). With the BOSEM removed from the TT assembly we were able to use an allen key as a flag to block the shadow sensor and see some variation in the signal. With the BOSEM touching a conductive surface the signal is stuck at about -6800 counts. (Thanks to Keita's careful shimming, the cable connector shield is not electrically connected to the body of the BOSEM.)