Reports until 14:01, Wednesday 06 November 2024
H1 TCS
oli.patane@LIGO.ORG - posted 14:01, Wednesday 06 November 2024 - last comment - 08:38, Tuesday 12 November 2024(81106)
TCS Monthly Trends FAMIS

Closes FAMIS#28454, last checked 80607

CO2 trends looking good (ndscope1)

HWS trends looking good (ndscope2)

You can see in the trends when the ITMY laser was swapped about 15-16 days ago.

Images attached to this report
Comments related to this report
camilla.compton@LIGO.ORG - 08:39, Thursday 07 November 2024 (81122)

Trend shows that the ITMY HWS code stopped running. I restarted it.

camilla.compton@LIGO.ORG - 08:38, Tuesday 12 November 2024 (81220)CDS

Erik, Camilla. We've been seeing that the code running on h1hwsmsr1 (ITMY) kept stopping after ~1hours with a "Fatal IO error 25" (Erik said related to a display) attached.

We checked that memory is fine of h1hwsmsr1. Erik troubleshooted this back to matplotlib trying to make a plot and failing as there was no display to make the plot on. State3.py calls get_wf_new_center() from hws_gradtools.py which calls get_extrema_from_gradients() which makes a contour plot, it's trying to make this plot and thinks there's a display but then can't plot it.  This error isn't happening on h1hwsmsr (ITMX).   I had ssh'ed into h1hwsmsr1 using -Y -C options (allowing the stream image to show), but Erik found this was making the session think there was a display when there wasn't.

Fix: quitting tmux session, logging in without options (ssh controls@h1hwsmsr1),  and starting code again. The code has now been running fine for the last 18 hours.

Images attached to this comment