H1 called for assistance as the SQZer lost lock and has been struggling to relock, its been stuck waiting for the TTFSS to lock, it says "Check crystal frequency, can try 45MHz" which is a notification I haven't seen before and I can't find any reference in the WIKI. I'm calling out for SQZ help
I got ahold of Jenne, who suggested following alog 70050 which I did not notice that DIAG_MAIN was also complaining about. Doing this which is reseting the ISS did not help this TTFSS issue. Also at Jennes suggestion I'm going to try and get us into observing without SQZing
Back into observing with NO_SQUEEZING at 11:09 UTC, I followed the WIKI. I forgot that I also had to edit the IFO node to add to the exclude list to get us back into observing ( we should add this to said WIKI page). I reverted the attached SDF diffs
The usual trick of manually moving the crystal frequency to find the beatnote (suggested by the error message Ryan reports above) doesn't work in this case. The attached screenshot shows that at about 3 am the power on the FSS PD nearly doubled, but the fiber power, polarization, and power out of the squeezer laser didn't change. We lost the beatnote at that time (BEATnote strength drops to -40dBm. Moving the laser frequency around doesn't change this.
Note that I thought I could move the laser frequency while we were in observing without squeezing, because the crystal frequency is not monitored in SDF. However changing this changes other things which are monitored, and did temporarily knock us out observing. Some automation brought us back to observing quickly enought that I didn't notice the first time and so did this twice.
Vicky, Sheila, Camilla. Went SQZ_MANAGER to DOWN which cleared any wrong sdfs that had been reverted. The H1:SQZ-FIBR_SERVO_DEMOD_RFMON should not be this low (-40dB), even in old TTFSS issues (75337). We tried touching everything on the TFFSS screen to no avail. We tried changing polarization into "to beatnote" fiber. Turning down this power dropped the H1:SQZ-FIBR_PD_DC_POWER form 0.4 to 0.3. Rather than halving it. Going out to check powers on SQZT0.
On SQZT0 Sheila and I found that even with the PSL fiber unplugged and SQZ beam blocked, H1:SQZ-FIBR_PD_DC_POWER reported power.
Fil replaced the D2300329 Fiber Beat Note Box with our spare. This solved our TTFSS issues. It appears something in the power supply for the box had failed. Details in 77418.