I haven't found any particular pattern yet of why this is happening, but sometimes we make it through ENGAGE_SOFT_LOOPS the first time around, and sometimes we lose lock there, and then make it up to NomLowNoise the next time. I noticed this because we get to ~300 kPc and then lose lock, which shows up as a small blip up in the range plot during acquisition.
If an on-shift operator has some time, it might be interesting to try to understand why this is happening, so that we can fix it. On several of the times that we make it through on the first time, we are back locked extremely quickly!
The attached screenshot is 3 days of range and guardian state, just showing that sometimes we lose lock there, and sometimes we make it through.
I'm not sure that this is it, but I wonder if the IFO isn't happy when the A2L gains aren't at their 'centered' values (see my side note in alog 54949). Recall that initial alignment is set to the centered values of the optics.
The DOWN state, and through the CARM offset reduction sequence, there are no changes made to the A2L gains for the test masses; they are left at whatever value they had at lockloss. DHARD is turned on using whatever A2L values were there. In PREP_ASC_FOR_FULL_IFO we set the A2L gains to the values that will center the beam on the optic, then we turn on the ADS lines. In Engage_ASC_FOR_FULL_IFO we start servoing the PRM to make the spot centered on ITMY, and then in ENGAGE_SOFT_LOOPS we start moving the arms to center the spots on the ETMs.
Since the A2L gains are set to their centered values before we start engaging the full IFO ASC, I'm having trouble coming up with a reason that this should be causing egregious problems. However, it feels awfully suspicious that we seem to be making it to NomLowNoise the *second* attempt pretty regularly, where the first attempt will always be with the high power A2L values (if we lost lock from NomLowNoise, or anywhere after MOVE_SPOTS), and the second attempt (if we lost lock after PREP_ASC_FOR_FULL_IFO) will be with the centered values.
I have put a few lines of code into prep for locking to set the A2L gains to their centered values so that we always start with the same situation every lock. If this is bad, operators can comment out lines 427-430, near the center of PREP_FOR_LOCKING in ISC_LOCK.