We have had another DAC issue on h1seih23/45 which will require a restart of all the models on these front end computers.
Would the first person into the control room today please call me on my cell phone and we can get these systems going again.
I have performed all the diagnostics data gathering; as before the first DACs on both systems are reporting empty FIFOs.
On Tuesday I can install new IOP software on these systems to run through empty fifo errors.
Opened FRS11748 for this issue.
Fault was cleared by restarting all models on h1seih23 and h1seih45. Sheila monitored the restarts from the control room. FRS was subsequently closed. I've marked zero hours lost because issue happened when there were no commissioning activities. If this is incorrect, please edit the FRS accordingly.
In this instance the time sequence was:
1225270147
Nov 03 2018 01:48:49 PDT
1225270150
Nov 03 2018 01:48:50 PDT
Note that the delta time between IOP DAC errors is 3 seconds, for the previous 3 events the deltas were 25, 15, 5 ,so the 'multiples of 5 seconds' pattern has been broken and was presumably coincidence.