Greg, Dave:

While investigating IOP-SUS ADC+Timing errors, I found an event at 03:07:01 Sunday 4/16 PDT in which all of the SUS-EY front end models reported a STATE_WORD of 16 (DAQ error). At the same time, the DAQ Data Concentrator reported a sequence of 23 data blocks with invalid CRC check sums (except for h1susetmypi which reported 24). Greg confirms that the GDS/DMT calibration code did see an invalid data flag at that time.

The CDS overview for the end station sus systems is attached. The EX CRCs incremented at 04:39 4/14 UTC, the EX CRCs incremented at 10:07 4/16 UTC (21:39 Thursday and 03:07 Sunday PDT respectively). Greg says that the EX event on Thursday did not cause a calibration issues as only EY data is being used.

Trending over the past 150 days (to start of O2) shows CRC error burst have occurred 6 times at EX and 8 times at EY.

Detailed time plot of Sunday morning's 03:07 EY event suggests that the data error is contiguous for 23 blocks of data (1/16th second blocks), meaning 1.44 seconds of bad data.

Investigation is continuing.