camilla.compton@LIGO.ORG - posted 02:46, Saturday 21 October 2023 - last comment - 10:35, Monday 23 October 2023(73628)
CDS dolphin crash, causing lockloss and HAM1 watched tripp
8:05UTC We had a 8:05UTC lockloss where HAM1 HEPI WD tripped (ATC type trip)
8:18UTC I saw no obvious issue with ground motion or HAM1 so Untripped HAM1
Then noticed that CDS screen was red with timing card, adc, dac, dk errors on machines sus123, sush2a, sus34, sus56, iopoaf, and ioplsc0.
08:24UTC Left voice message with Dave Barker. 08:32UTC Voicemail left with Erik.
08:35UTC Dave Barker online and looking at front end crashes. alog 73629
08:45UTC Dave restarting computers as all DAC cards look okay. I requested ISC_LOCK to IDLE and all corner station (not HAM7) SUS to SAFE (Safe State for IFO wiki) and Dave bypassed SEI.
09:10UTC CDS Okay
SUS WD untripped and back to nominal configuration.
SEI ISI WD tripped (payload type) when Dave bypassed them reset those WDs too.
09:35UTC All SUS and SEI nominal. Resumed locking with ISC_LOCK via DOWN, INIT, NLN.
09:45UTC No light seen on green arms so plan to run baffle align scripts (62024).
Comments related to this report
camilla.compton@LIGO.ORG - 04:47, Saturday 21 October 2023 (73634)
Put ISC_LOCK into IDLE to run the baffle align scripts so it wasn't taken to INITAL ALIGNMENT by H1_MANAGER. TMS ones sucessful but the ITM baffle align screens failed and I checked the ITM oplevs moved sliders to get oplevs in same potison as in NLN. This moved ITMX 70urad and ITMX 45urad in pitch. A little light now on green cameras/ AS AIR. Reran ITM baffle align scipts.
10:20UTC began Inital Alginment. ALSX PLL Unlocked.
10:50UTC Not much light on X-arm green. Moved BS back to BS oplev postion, 10urad PIT.
11:20UTC Tried moving PR following sliders, greens arms PDs or cameras not better, reverted.
camilla.compton@LIGO.ORG - 06:51, Saturday 21 October 2023 (73635)
Summary: To lock green arms I put ITMs, BS, PR2 and PR3 back to Oplev positions before trip/last time we were aligning. Also touched ETMs and TMSs as usual in green arms aligning. During intal alignment I touched H1:LSC-XARM_GAIN (and reverted), PR2, PRM, SRM.
Trended the IM's LKIN signals, all seem similar to before trip, IM4 changes when locking usually.
Moved PR2 to oplev values (again). Starting (P 1566.8, Y 3227.6) Ending (1589.8, 3237.6).
Moved PR3 to oplev values. Starting (P-120.2, Y149.2). Ending (-117.2, 156.2). This made the green camera images and ITMY flashes MUCH better. After ITM, ETM, TMS alignment tweeks, have both green arms locking.
12:34UTC Started Initial Alignment. A few placed need to restore optics to their oplevs from last initial alignment position.
12:59 UTC stuck at XARM_IR_LOCKED
Again adjusted PR2. Starting (1566.8, 3237.6), ending (1581.8, 3237.6), doubled H1:LSC-XARM_GAIN gain, touch PR2 while watching TRX flashes (and then restored gain) as Corey describes in 69917 and wiki.
Couldn't Acquire PRX. Sliders for Oplevs for PRM restored. Starting (-1377.9, 220.1) Ending: (-573.9, 335.1).
SRC_ALIGNMENT offloaded a wring alignment and SRM was saturating with L filters having really large numbers. Took ALIGN_IFO to DOWN and this cleared. Moved SRM to oplev value. Starting (2231.2, -3158.3) after (2247, -3199) before requesting SRC_ALIGN_OFFLOADED.
13:48 UTC INITAL ALIGNMENT Complete.
camilla.compton@LIGO.ORG - 10:35, Monday 23 October 2023 (73665)OpsInfo
Documented in FRS 29441. Ryan C got us back to NLN at 20:16UTC by restoring all the sliders to when we were locking before the previous lock and then running another initial alignment alog 73637.
Restoring sliders with (sitemap > ASC > Init Align Compat > ! Restore) should have been the first thing to do as during the computer restart the sliders are reset to their safe.snap files, which are not (currently) recent values. Tagging OpsInfo and updated Recovery from Dolphin/Computer Crash wiki.
Summary: To lock green arms I put ITMs, BS, PR2 and PR3 back to Oplev positions before trip/last time we were aligning. Also touched ETMs and TMSs as usual in green arms aligning. During intal alignment I touched H1:LSC-XARM_GAIN (and reverted), PR2, PRM, SRM.
Documented in FRS 29441. Ryan C got us back to NLN at 20:16UTC by restoring all the sliders to when we were locking before the previous lock and then running another initial alignment alog 73637.
Restoring sliders with (sitemap > ASC > Init Align Compat > ! Restore) should have been the first thing to do as during the computer restart the sliders are reset to their safe.snap files, which are not (currently) recent values. Tagging OpsInfo and updated Recovery from Dolphin/Computer Crash wiki.