Reports until 21:18, Sunday 21 July 2019
H1 General
jeffrey.bartlett@LIGO.ORG - posted 21:18, Sunday 21 July 2019 - last comment - 15:01, Monday 22 July 2019(50690)
Ops Evening Mid-Shift Summary
    Lost lock at  00:12 (17:12). 

While relocking: 

   (1). ALS-Y would not build up or lock. Tweaked ETM-Y in pitch and yaw to get it to lock. Then dressed up TMS-Y in pitch to get build up above 1.0. It took two attempts to get past DRMI_1F.   Lost this lock attempt at PREP_DC_READOUT_TRANSITION. OMC had Failed to find Carrier message. 

   (2). Green locked OK. IR Diff was not found, so I tweaked it by hand. Locking stuck again at DRMI_1F. After 12 minutes, and because the AS_Air spot looked so bad in the first lock attempt, I tried to engage CHECK_MITCH_FRINGES, but it never reached it. Lost lock around DRMI_1F.

   (3). Next couple of lock attempts made it past DRMI_1F OK. Then started getting OMC Did Not Find Carrier or Wrong Mode? messages. After losing lock while trying to sort out OMC issue, stopped at ENGAGE_SOFT_LOOPS to work with the OMC to get it READY_FOR_HANDOFF. Again lost lock.

   (4). Called Keita. Stopped at PREP_ASC_FOR_FULL_IFO to work on getting OMC to lock. Was able to get OMC to READY_FOR_HANDOFF. But would continue to lose lock at ENGAGE_ASC_FOR_FULL_IFO. 

   (5). Running an Initial Alignment now and will try to relock when complete.   
Comments related to this report
jenne.driggers@LIGO.ORG - 15:01, Monday 22 July 2019 (50717)ISC

This was a concerning reacquisition since it seems like it was such a struggle, so I had a look at each of the locklosses during the acqusition attempts.  The summary is that it looks like this is a situation where the alignment was just too far off after the 35+ hour lock, and that all of the intermediate locklosses are due to ASC problems.  There were 8 locklosses, an initial alignment, one more lockloss, and then the IFO made it to NomLowNoise on the 2nd attempt after initial alignment.  In hindsight (which always has better vision than when you're in-the-moment), we should likely have taken the IFO to initial alignment earlier in this sequence. 

Mike made the point earlier today that probably we should set a threshold, and if the previous lock has been longer than XX hours, just start by doing an alignment because it's always going to need it.  It'll take some alog mining to see precisely what that threshold should be, but (based on memory and feelings), I think that 20 hours is a fine place to set the line, until we've had time to dig through past alogs.  So.  If we have a lockloss, and the previous lock was longer than 20 hours, please run an initial alignment before trying to acquire lock.  And, make note of this in the lockloss spreadsheet.  Thanks!

What follows is just a list of the locklosses, and what I see about them (and that most of them are due to the alignment being too far off, or some other ASC problem).

  • 1247792959: IFO at PREP_DC_READOUT_TRANSITION, OMC wasn't locked yet. 
    • OMC guardian reattempted to lock itself, but hadn't finished, so ISC_LOCK never tried to go to DC readout (which is good).
    • ADS PIT4 was not converged yet, indicating that the Xarm needed to keep moving in pitch.
    • There was a pretty significant ASC pitch ring up at 1.9 Hz, seen in many different signals, including DC1, DC2, MICH, DHARD, SRC1, SRC2, INP1, and PRC2. 
      • This was an ASC lockloss, perhaps becuase the Xarm SOFT was still so far out.
  • 1247794401: IFO had just started DRMI ASC.
    • Nothing really obvious to me on the ASC signals that would cause a lockloss, but PRC2 yaw was still very far out, and MICH yaw was being pulled away from zero (so perhaps it wasn't able to follow along with the big PRC moves).
      • This was an ASC lockloss.
  • 1247796076: IFO was at CARM_150PM
    • Not sure what this one was, doesn't look so obviously ASC related, although these are difficult states to get through if the alignment of the arms isn't good. 
  • 1247798175: IFO was in PREP_DC_READOUT_TRANSITION, OMC wasn't locked/
    • OMC guardian went to OMC_LOCKED, even though the DCPD sum was only 0.01 mA, so I'm not sure how it got through OMC_LSC_ON.  But, the OMC guardian caught that it wasn't locked within ~1 sec, requested DOWN, and started acquiring the OMC again. 
    • Lockloss while OMC was still looking for the carrier.
    • Another ASC ring up at 1.9 Hz, and PIT4 wasn't converged yet
      • ASC lockloss, of the same type as the first attempt in this list.
  • 1247799443: IFO was in ENGAGE_SOFT_LOOPS.
    • No 1.9 Hz ring up.
    • ADS output signals for PIT3 went above 2000 counts (more than twice what I'll usually let it go when I'm moving spots via changing the A2L gains), and YAW3 went to 1400 counts (which is still high) when the arm ADS loops came on.
      • Probably an ASC lockloss, just because other loops couldn't keep up with that big move.
  • 1247801221: IFO was in ENGAGE_ASC_FOR_FULL_IFO.
    • ASC lockloss.
  • 1247802547: IFO was in ENGAGE_ASC_FOR_FULL_IFO.
    • ASC lockloss.
  • 1247803660: IFO was in ENGAGE_ASC_FOR_FULL_IFO.
    • ASC lockloss.
  • Break for initial alignment
  • 1247806916: IFO in carm to TR
    • Not sure what this one was.  Doesn't look ASC, no glitch in green arms.