david.barker@LIGO.ORG - posted 08:22, Monday 08 June 2015 - last comment - 11:19, Monday 08 June 2015(18976)
details of DAQ broadcaster fault from Friday 5th June
Attached is the output from dmesg on h1broadcast0. It is reporting an divide error on the daqd process and suggesting a reboot. We should consider scheduling this for tomorrow's maintenance.
Non-image files attached to this report
Comments related to this report
The failure mode of the GDS broadcaster (no updates to GPS time, etc.) are consistent with a failure of the 'myri10ge' network driver. That driver controls the 10G adapter taking in data (at 16Hz) from the data concentrator. If it gets no data, it does not update GPS time, etc. that are in the data packet. As for solutions, perhaps restarting machines more often that once every 9 months will help. We are also look at updating the OS on the DAQ computers, which would include newer drivers - they are built into the kernel in Linux 3.2.