Two unrelated problems with the DAQ ongoing:
1. disk9 of the DCS raid system for h1fw0 failed at 10pm Friday night. h1fw0 lost its file system at 1:50am this morning and has not been running since then.
2. h1susim front end has failed and has caused a DAQ error for the following frontends: h1susb123, h1sush2a, h1sush34, h1sush56, h1susauxh2, h1susauxh34, h1susauxh56, h1sieb1, h1sieb2, h1seib3, h1seih16, h1seih23, h1seih45, h1psl.
both end stations have good DAQ data.
I've emailed LDAS regarding the disk failure in the LDAS server room.
For problem 2, I'm going to try restarting the DAQ data stream from those frontends.
tried restarting MX streams on h1susb123, but no dice. I suspect the fact that h1sush2b has completely failed is actively causing the problem.
Looking at the hourly burts, h1susim front end failed between 11pm and midnight Friday night. The related DAQ errors happened at the same time.
here is the cds overview during this DAQ problem