Danny, Craig, Dan Brown
Here's a chronological list of observations and things we did today after the Dolphin crash to try and get the interferometer back up and locking.
- We reset all the watchdogs and set all the alignment sliders to before the crash, most needed adjusting. After putting everything back the alignment still wasn't great.
- We did initial alignment again but Y wouldn't level off. It hovered around 1 for awhile when setting the arm to INITIAL_ALIGNMENT, but it would keep periodically dipping to 0.8 then back up to 1 every 4-5 seconds.
- It turns out the FIR filters H1:ISI-ITMY_ST1_SENSCOR_GND_STS_X_FIR_GAIN, H1:ISI-ITMY_ST1_SENSCOR_GND_STS_Y_FIR_GAIN, H1:ISI-ITMY_ST1_SENSCOR_GND_STS_Z_FIR_GAIN were all 0. Looking at ITMX they were all set to 1 so we changed them manually. Seemed to fix the ALS Y alignment dipping problems we had.
- FIND_IR: TRY_NORM was registering high even with no light, so we ran userapps/isc/h1/scripts/dark_offsets/dark_offsets.py. However, the script set it to a negative offset so we did it by hand.
- We were still struggling to find IR. We adjusted comm offset to find it ourselves. Although this was an issue with the first few locks it seemed to be fine later in the day.
- We were losing lock at OFFLOAD_DRMI_ASC, so we increased the ramp from 2s to 20s to not kick it so hard. Didn't have any issues with this state for the rest of the day.
- During ENGAGE_REFL_POP_WFS, PRC1 and INP1 pitch ASC were railing causing us to lose lock. We switched on INP1 manually and stepped PRM pitch until the PRC1 error signal was closer to zero, letting PRC2 and INP1 follow. During this time PRG drop from 47 to 42, engaging CHARD dropped us to 40 and got very noisy so we may be clipping now. This all turned out badly in the end. When CSOFT_P gain was increased it became unstable and we lost lock. We think this means our input alignment isn't great right now. At this stage we also lost FSS lock?
- We changed these offsets to stop PRC1 wandering off and keep us at PRG~47:
- H1:ASC-POP_A_PIT_OFFSET from -0.08 to -0.713
- H1:ASC-POP_A_YAW_OFFSET from -0.24 to -0.08
- We lost lock at increase power. FSS also unlocked again.
- Stayed at decrease 45 modulation for awhile and we lost lock again, FSS also railed again. A 21Hz instability was ringing up in OMC DCPD output. It looks like it was coming from DHARD pitch from the lockloss tool.
- H1:PSL-POWER_SCALE_OFFSET for some reason got set to 1.3 instead of 1.8 on lockloss. This led us to believe our RF18 and RF90 were significantly higher.
- Eventually we reached nominal low noise at 20W.
As Craig mentioned Dan adjusted ASC-POP_A_QPD pitch and yaw offsets at nominal low noise and saw improvement in the recycling gain and DARM (mostly at 10-20 Hz).
It seems PRMI needs touching up after a lock loss each time at the moment, BS pitch in particular
Referencing the errors with the SEI_CONF sensor correction errors, the best way I've found to fix the seismic configuration after front end restarts, dolphin crashes or whatever, is to select something like the SC_OFF or LARGE_EQ_NOBRSXY state, letting SEI_CONF arrive, then going back to the state you actually want. There were other configuration errors on ITMY (specifically the stage 1-2 sensor correction was off) that did not get fixed by turning on stuff by hand. TJ says that re-requesting a state will cause the guardians to re-run the state, but I haven't seen that yet and TJ wasn't sure if that code is running on all the SC nodes yet.