Jamie, Sheila, everyone,
Over the past several days, TJ's verbal alarams have been warning us about ETMX software watchdog trips which aren't really happeneing. This is interseting though, since we've noticed that sometimes this seems coincident with a huge glitch in DARM that can be seen in the spectrum. The verbal alarm script is checking the channel H1:IOP-SEI_ETMX_DACKILL_STATE. It sometimes jumps to a value of 3 for about a second and comes back to 0.
Three incidents from Friday night happened in the 10 to 20 seconds proceeding these times (UTC):
8/1/2015 7:37:40, 5:32:10, 4:00:00
One of these incidents a huge glitch is visible in the DARM time series before the DACKILL state changed.
Two questions probably need further investigation, is DACKILL behaving the way we want it to, and are the glitches in DARM cauing the DACKILL state to change or is something else causing both DARM glitches and the change in DACKILL state?
Dave and Jim suggested 2 more channels to look at for this time when there was an obvious glitch in DARM, and the SEI_ETMX_DACKILL state changed a second or so later. Indeed, the sus IOP state word changed at the same time as the DACKILL changed, although there is no timing error. Like the DACKILL state, this seems to happen after the glitch.
