Now that we're back to observing with regularity after a week full of intense troubleshooting, I decided it would be nice to have a summary of the events that contributed to IFO locking issues in the past week and what was done along the way to fix them all in one place, so that's what I'll attempt do in this alog. Anyone should be encouraged to add comments with things I missed or additional commentary. Things I believe to be the most significant changes either contributing to or fixing IFO locking issues I've bolded.
- Monday Oct. 20
- H1 is generally locking reliably despite high microseism
- CSOFT_P and INP1_P gains updated - alog87582
- Power scaling reference channel used in LASER_PWR Guardian is changed to IM4 TRANS - alog87581
- H1 has a lockloss and things look strange immediately after - alog87595
- Issues discovered to be a result of the power scaling change from earlier in the day, which is reverted - alog87598
- While troubleshooting, the dark offset script is run and many SDF diffs are accepted, including filter changes on the LSC POP A RF45 diode - see first screenshot in alog87598
- This changes whitening settings on this diode which are now incorrectly set when going through SDF_REVERT
- Dark offsets are doubled as a result of different settings which are also accepted in SDF
- H1 is unable to relock with high microseism as the reason given
- Tuesday Oct. 21 (Maintenance Day)
- Power scaling reference channel function implemented in LASER_PWR Guardian to avoid previous issue - alog87603
- Power supply for EX HEPI pump controller and Beckhoff vacuum controls replaced - alog87606
- HAM7 CPS switched to local timing scheme - alog87608
- Sat amps for lower stages of MC1, MC2, MC3, PR3, and SR3 updated - alog87620
- P/SR3 estimators installed and engaged - alog87616
- This began causing oscillations at 0.6 Hz while locking
- Power scaling channel update reverted due to LSC instability while relocking - alog87624
- BRSY moved out-of-loop - alog87634
- H1 is unable to relock overnight with unstable LSC as the reason given - alog87640 and others
- Wednesday Oct. 22
- ASC gain changes reverted, 2x MICH ASC gain added, A2L gains in MOVE_SPOTS changed - alog87645 and comments
- P/SR3 estimators discovered to be the culprit of 0.6 Hz instability, HBW ASC discovered to fix 1 HZ ASC ringup when it appears - alog86758
- P/SR3 estimators disengaged - alog86759
- H1 struggles to relock overnight with ETMY roll mode and 1 Hz ASC ringups as the reasons given - alog87660 and alog87672
- Thursday Oct. 23
- Problematic roll mode is found to actually be on ETMX, not ETMY - alog87698
- Most of the day is spent finding good damping settings for the ETMX roll mode; H1 does no observing and no locking at all overnight - alog87708
- Friday Oct. 24
- DHARD_P steps commented out of LOWNOISE_ASC - alog87712
- ETMY sat amp discovered to be causing saturations on reaction chain - alog87713
- Sat amp swapped, fixing saturations - alog87722
- Incorrect whitening settings and drastically different dark offsets on LSC POP A RF45 are finally discovered and fixed - alog87728
- DAMP_BOUNCE_ROLL state is moved from right before powerup to after LOWNOISE_LENGTH_CONTROL - not logged, but evidenced in ISC_LOCK code history
- Since we've changed the roll mode damping to ETMX, we don't want it on during ESD transitions, hence this change in state order
- Unbeknownst to those involved at the time, this state has been turning on DARM actuation to ETMY which would cause its bounce mode to ring up, but later states had been fixing this before this change
- H1 is unable to relock from ETM bounce modes now being rung up and high winds; no observing or locking overnight - alog87734
- Saturday Oct. 25
- Accidental ETMY actuation from DAMP_BOUNCE_ROLL discovered; state removed from Guardian path - alog87738
- CSOFT_P gain updated and gain change removed from THERMALIZATION node
- This means no damping is currently being done on any bounce or roll modes, but these have not been seen to be a problem since
- H1 returns to observing for an appreciable length of time and locking resumes its automatic nature through the weekend