Just realized that DIAG_MAIN has been flashing messages that the HWS code has stopped. They were so brief I had to look in the log: 2019-04-07_01:23:59.022623Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ETMY code stopped 2019-04-07_01:55:22.025647Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ETMX code stopped 2019-04-07_01:59:59.028707Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ITMY code stopped 2019-04-07_05:07:38.016456Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ETMY code stopped 2019-04-07_05:13:36.027085Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ETMX code stopped 2019-04-07_05:43:37.025772Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ITMY code stopped 2019-04-07_07:13:21.025773Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ITMX code stopped 2019-04-07_07:18:26.014392Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ITMX code stopped 2019-04-07_08:31:50.012318Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ETMX code stopped 2019-04-07_08:51:17.019917Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ETMY code stopped 2019-04-07_09:27:15.028043Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ITMY code stopped 2019-04-07_11:12:11.026664Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ITMX code stopped 2019-04-07_11:50:04.023291Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ETMX code stopped 2019-04-07_12:34:56.023720Z DIAG_MAIN [RUN_TESTS.run] USERMSG 0: HWS: HWS ETMY code stopped
Luckily, the HWS code is okay. The test in DIAG_MAIN was looking to see if H1:TCS-{}_HWS_LIVE_ACQUISITION_GPSTIME was static for longer than 5 seconds. That seems to be much too short, so I bumped it up to 60 seconds.
Good catch Patrick.