R. Short, J. Oberling
Today we were originally planning to take a look at the TTFSS, but noticed something before going into the enclosure that we think is the cause of some of the PSL instability since the completion of the NPRO swap. We were looking at some trends from yesterday's ISS OFF test, recreating the plots Ryan C. had made, when we noticed the ISS get angry and enter into an unlock/relock cycle that it couldn't get out of. While it was doing this we saw that the PMC Refl power was up around 40W 30W and "breathing" with PMC Trans; as PMC Refl would increase, PMC Trans would decrease, but the sum of the 2 was unchanged. We then unlocked the ISS. As we sat and watching things for a bit after the ISS unlock we saw that PMC Refl would change by many Watts as it was breathing. We tried unlocking the IMC, the behavior was unchanged; we then tried unlocking the FSS RefCav, and the behavior was still unchanged. It was at this point we noticed that the moves in PMC Refl were matched by very small moves (1-2 mW) in the NPRO output power; as the NPRO output power went up, PMC Refl went down, and vice versa. This looked a whole lot like the NPRO was mode hopping, and 2 or more modes were competing against each other. This would explain the instability we've seen recently. The ISS PDs are located after the PMC, so any change in PMC output (like from competing modes) would get a response from the ISS. So if PMC Refl would go up, as we had been seeing, and PMC Trans drops in response, also as we had been seeing, the ISS would interpret this as a drop in power and reduce the ISS AOM diffraction in response. When the ISS ran out of range on the AOM diffraction it would then become unstable and unlock; it would then try to lock again, see it "needed" to provide more power, run out of range on the AOM, and unlock. While this is happening the FSS RefCav TPD would get very noisy and drop, as power out of the PMC was dropping. You can see it doing this in the final 3 plots from the ISS OFF test. To give the ISS more range we increased the power bank for the ISS by moving the ISS Offset slider from 3.3 to 4.1; this moved the default diffraction % from ~3% to ~4%. We also increased the ISS RefSignal so when locked the ISS is diffracting ~6% instead of ~2.5%.
To try to fix this we unlocked the PMC and moved the NPRO crystal temperature, via the FSS MEDM screen, to a different RefCav resonance to see if the mode hopping behavior improved. We went up 2 RefCav resonance and things looked OK, so we locked the FSS here and then the ISS. After a couple minutes PMC Refl began running away higher and the ISS moved the AOM lower in response until it ran out of range and unlocked. With the increased range on the diffracted power it survived a couple of these excursions, but then PMC Refl increased to the point where the ISS emptied its power bank again and unlocked. So we tried moving down by a RefCav resonance (so 1 resonance away from our starting NPRO crystal temperature instead of 2) and it was the same: things looked good, locked the stabilization systems, and after a couple minutes PMC Refl would run away again (again the ISS survived a couple of smaller excursions but then a large one would empty the bank and unlock it again). So we decided to try going up a 3rd resonance on the RefCav. Again, things looked stable so we locked the FSS and ISS here and let it sit for a while. After almost an hour we saw no runaway on PMC Refl, so we're going to try operating at this higher NPRO crystal temperature for a bit and see how things go (we went 2 hours with the ISS OFF yesterday and saw nothing, so this comes and goes at potentially longer time scales). The NPRO crystal temperature is now 25.2 °C (it started at 24.6 °C), and the temperature control reading on the FSS MEDM screen is now around 0.5 (it was at -0.17 when we started). We have left the ISS Offset at 4.1, and have moved the RefSignal so it is diffracting ~4%; this to give the ISS a little more range since we've been seeing it move a little more with this new NPRO (should the crystal temperature change solve the PMC Refl runaway issue we could probably revert this change, but I want to keep it a little higher for now). Ryan S. will edit DIAG_MAIN to change the upper limit threshold on the diffracted power % to clear the "Diffracted power is too high" alert, and Daniel, Vicky, and Camilla have been re-tuning the SQZ and ALS lasers to match the new PSL NPRO cyrstal temperature. Also, the ISS 2nd loop will have to be adjusted to run with this higher diffracted power %.
Edit: Corrected the initial high PMC Refl from 40W to 30W. We saw excursions up towards 50W later on, as can be seen in the plots Ryan posted below.
Sadly, after about an hour and a half of leaving the PSL in this configuration while initial alignment was running for the IFO, we saw similar behavior as what was causing the ISS oscillations as seen last night (see attachment). There's a power jump in the NPRO output power (while the FSS was glitching, but likely unrelated), followed by a dip in NPRO power that lines up with a ~2W increase in PMC reflected power, likely indicating a mode hop. This power excursion was too much of a change out of the PMC for the ISS, so the ISS lost lock. Since then, there have been small power excursions (up to around 18-19W), but nothing large enough to cause the ISS to give out.
As Jason mentioned, I've edited the range of ISS diffracted power checked by DIAG_MAIN to be between 2.5% and 5.5% as we'll be running with the ISS at around 4% at least for now. I also checked with Daniel on how to adjust the secondloop tuning to account for this change, and he advised that the ISS_SECONDLOOP_REFERENCE_DFR_CAL_OFFSET should be set to be the negative value of the diffracted power percentage, so I changed it to be -4.2 and accepted it in SDF.
Last attachments are of the situation as we first saw it this morning before making adjustments, and a zoomed-out trend over our whole work this morning.