jason.oberling@LIGO.ORG - posted 09:49, Thursday 02 March 2023 - last comment - 10:46, Friday 03 March 2023(67721)
PSL Down, NPRO Failure
At roughly 4am the PSL went down due to a shutoff of the NPRO. The NPRO shut off due to an over-temperature error for its internal pump laser diode B. We have confirmed that this is a real signal coming from the NPRO and is not a failure in the Beckhoff signal chain, so it looks like we need to replace the NPRO. This will at minimum necessitate a realignment and a re-do of the mode matching between the NPRO and the 1st amplifier. We're spooling up the spare NPRO now and will proceed with the replacement as soon as it's ready. WP incoming.
WP 11022
Matt Heintze had a good question of whether there are any obvious precursors to yesterday's failure that we can look for, to perhaps preemptively swap things in the future. However, I don't see any obvious patterns.
The first two attachments have all of our PSL NPRO-related channels, one zoomed in on the few minutes before the "LD2" (laser pump diode #2) temperature controller failed, and the other zoomed out for several weeks. The LD2TEMP channel started to go up, several of the output powers went wobbly, and the temperature guard shut things down quite quickly (which is what it should do, so it worked well).
The next attachment (3rd) is zoomed in to the time when the first temperature guard tripped. It's perhaps difficult to say since this is all 16 Hz data, but when I zoom in further it looks like the tempguard trips after 1/16th of a second of elevated LD2 temp (so, it certainly could have been faster, we just don't have data that's faster). It looks like the power output of the LD2 and the NPRO overall were starting to become wobbly at least 2/16ths of a second before the tempguard trip.
The 4th attachment is zoomed out a bit to show that about 30 seconds after the tempguard trip (which is cursor T1), the NPRO power has sagged enough that the NPRO power watchdog trips (cursor T2), and so the whole LASER_ENABLED goes down. About 3 seconds later the amplifier outputs have sagged enough (presumably because they've been shut down) that their power watchdogs notice.