stefan.ballmer@LIGO.ORG - posted 21:58, Friday 30 September 2016 - last comment - 02:21, Saturday 01 October 2016(30131)
Various issues
We repeatedly had CHARD run away when switching to LOWNOISE_ASC.
The investigation was not helped lockloss tool, which started crashing suddenly. Guaridan also started having issues (connection errors, white epics channels.)
Comments related to this report
jameson.rollins@LIGO.ORG - 02:21, Saturday 01 October 2016 (30134)
For partial explanation of the guardian issue see comment to next log.
I'm guessing that the issue with the lockloss tool might have been an overlong delay finding the latest lockloss times due to excessively verbose logging of the ISC_LOCK node when it's in connection error. This mostly exposes the weakness of the lockloss tool relying on parsing the ISC_LOCK node logs for determining lockloss times, but secondarily points to the logs being maybe unnecessarily verbose under these particular connection error conditions. If the lockloss tool problem was *not* due to a long wait time for returning the list of lockloss times, please let me know what the error was so that I can investigate.
I have an improved version of the lockloss tool that finds locklosses much faster via NDS. I'll push it out after I push a minor guardian update on Tuesday. It should make the lockloss tool much faster and more robust.
I also realize there's an issue with the log display part of the lockloss tool. This is completely orthoganal issue to the plotting, and will also be fixed with the next guardian minor release.
For partial explanation of the guardian issue see comment to next log.
I'm guessing that the issue with the lockloss tool might have been an overlong delay finding the latest lockloss times due to excessively verbose logging of the ISC_LOCK node when it's in connection error. This mostly exposes the weakness of the lockloss tool relying on parsing the ISC_LOCK node logs for determining lockloss times, but secondarily points to the logs being maybe unnecessarily verbose under these particular connection error conditions. If the lockloss tool problem was *not* due to a long wait time for returning the list of lockloss times, please let me know what the error was so that I can investigate.
I have an improved version of the lockloss tool that finds locklosses much faster via NDS. I'll push it out after I push a minor guardian update on Tuesday. It should make the lockloss tool much faster and more robust.
I also realize there's an issue with the log display part of the lockloss tool. This is completely orthoganal issue to the plotting, and will also be fixed with the next guardian minor release.
Very sorry about the trouble.