After discussions with control room team: Jason, Ryan S, Sheila, Tony, Vicky, Elenna, Camilla
- Prior to September 13th:
- No IMC locklosses 80951
- NPRO/FSS glitches present but not bothering us (June 80520, last year 67718).
- September 12th: IMC gain redistribution 79998
- September 13th: First IMC lockloss
- September to October:
- We start noticing FSS glitches and do of tests to understand these.
- They were present in NPRO only with everything else unlocked (part of reason for swapping NPRO)
- Found that these glitches are not new, present in June 80520 and even last year 67718. Maybe they got worse, or maybe we just stopped surviving them as well.
- IMC has periods of being unstable (have to wait in DOWN)
- October 9th: IMC gain redistribution reverted 80566
- We have 6 days with only one IMC lockloss
- October 15th: NPRO current increased 80687
- IMC lockloses started again
- Ocotber 18th: NPRO current reverted 80746
- October 23rd to 29th: NPRO swapped 80929
- Swapped to new faraday and EOM, both found to be dodgy
- Ocotber 30th: Back to NLN
- Had issues with NPRO 81073: found to be NPRO mode hopping
- IMC locklosses continued
- October 30th to November 1st: Tests.
- CARM OLG 80956
- IMC OLG 80979
- FSS OLG 80993
- If the PSL ISS is unlocked the IMC can stay locked: 81056
- Attempt to swap TTFSS (reverted) 80983
- Hardboot of PSL chassis h1psl0: 81048
- November 6th: PSL mode hopping was fixed by changing NPRO temperature, 81107.
- PSL now stable
- IMC has periods of being unstable
- IMC locklosses continue
Conclusions: The NPRO glitches aren't new. Something changed to make us not be able to survive them as well in lock. The NRPO was swapped so isn't the issue. Looking the timing of these "IMC" locklosses, they are caused by something in the IMC or upstream 81155.
Tagging OpsInfo: Premade templates for looking at locklosses are in /opt/rtcds/userapps/release/sys/h1/templates/locklossplots/PSL_lockloss_search_fast_channels.yaml and will come up with command 'lockloss select' or 'lockloss show 1415370858'.
Updated list of things that have been checked above and attache a plot where I've split the IMC only tagged locklosses (orange) from those tagged IMC and FSS_OSCIALTION (yellow). The non IMC ones (blue) are the normal lock losses and (mostly) only once we saw before September.