[Jenne and Betsy in PSL enclosure; Jason logged in remotely; Marc, Fil, Rick, Fernando as EE support; consults with Daniel]
At today's commissioning meeting, we decided that one candidate thing to try to help mitigate the lockloss-causing glitches would be to swap in one of the spare TTFSS boxes. We also noted that if it was going to happen, it had to happen today, or wait until Mon/Tues next week. So, we tried swapping in a new FSS box. We were unable to lock the RefCav with either spare without significant oscillations, so ended up putting back in the original box, and that's how we'll leave things at least for the weekend.
Each time we changed out a box, we followed the steps 1-5 of our work permit 12182 (listed also below). In particular, before unplugging the 37 pin connector on the back, we had one of the support folks turn off the high voltage in the mezzanine of the CER (I think it is a pair of Kepcos, one +150V, one -150V). Then, once we had the 37 pin connector secured in the new box, one of the support folks went back to the mezzanine and turned back on the high voltage.
For both spares, the autolocker was able to find a ref cav resonance and hold it there. However, even with both the FAST and COMMON gain sliders set to the minimum values of -10dB we saw significant oscillation of the loop. An idea I had later (have not yet checked with anyone else to see if it's crazy or not) is to try one of the spare boxes, but with an attenuator on one of the cables, to see if it really is just a loop gain thing, or if there is something else going on. This could potentially be tried next week.
We first tried the 'blue box' spare, since it is supposed to be the same (including form factor) as the in-service unit. We noted that this box has had trouble in the past, but Marc said he'd taken a look and made some fixes, so it should work. Since that one caused oscillations and had previously been known to be challenging to work with, we tried also the 'silver box' spare, which I think has also been referred to as 'Rev-C'.
The silver box spare is a different form factor and different front/back panel than the blue box style, but it's supposed to be a drop-in replacement. There is no space to mount the water-cooled cold block, so if we had left the silver box TTFSS in place, we were just going to leave the cold block hanging around on top of the box, but not mounted to anything. Also, on the blue boxes, there is a SLOW output on the back panel that goes to a RAMP IN on the front panel. However, on the silver box that SLOW output is on the front panel, and it needs to be connected to the FAST OUT2 also on the front panel. Finally, the spigot labeled "PC" on the blue boxes is labeled "EOM" on the silver box. To install the silver box, you'll also need 2 adapters to go from TNC (on the silver box) to BNC (what the cables are, and what the blue box is), so bring those into the PSL enclosure with you.
Steps from the work permit submitted by Jason:
1)Unlock FSS RefCav.
2)Turn off High Voltage power supply in CER mezzanine.
3)Enter PSL enclosure and disconnect in-service TTFSS box.
4)Install spare TTFSS, hooking the same cables to the same ports on the TTFSS box; turn HV supply back ON.
5)Try to lock RefCav.
If it locks:
6)Measure TF to set common gain for ~500kHz UGF.
7)Measure crossover (use SR785, take a spectrum of IN1 port, crossover is at ~20kHz), set Fast gain to minimize hump in spectrum.
If it will not lock, then follow the same steps above to re-install the in-service TTFSS box.
Scans from the FSS after reducing gain.
In Marc's files above the A one should be magnitude for 100kHz to a few MHz frequency span, B should be phase for that measurement. C should be magntiude zoomed in on the 1.7MHz region, D should be phase for that measurement.