cyrus.reed@LIGO.ORG - posted 09:06, Tuesday 22 April 2014 - last comment - 11:00, Wednesday 23 April 2014(11496)
DAQ Driver Updates
I am about to start upgrading drivers on the DAQ as specified in WP4583. There will be NO DATA recorded during the period when the data concentrator is rebooted - as it is due for a full FSCK at boot, this will be a period of 10-15 minutes. For the remaining system upgrades, there will be intermittent access to the recorded data as those systems (h1nds1, h1fw1, h1broadcast0) are rebooted. Changes to h1nds0 and h1fw0 will happen at a later time. Further updates will be posted to this entry with specific downtime.
DAQ Downtime Report
h1dc0: 16:07:40 - 16:18:00 UTC There is NO data recorded by the DAQ during this period.
h1nds1: 16:26:00 - 16:28:20 UTC
h1fw1: 16:33:55 - 16:55:50 UTC There is NO data available via h1nds1/h1fw1 for this period. Use h1nds0/h1fw0 for frames ending/starting in this timeframe.
h1broadcast0: 17:06:55 - 17:16:00 UTC
Most installs were uneventful. However, on h1fw1, the MTU was not set to 9000 in /etc/conf.d/net as it is on h1fw0, which prevented daqd from running after restart. I changed /etc/conf.d/net to match and rebooted to fix; I have no idea how it ever worked before. On h1broadcast0, I disabled the items in local.start that are only useful for a data concentrator; h1broadcast0 being a clone of a data concentrator had these unnecessary additions.
Technical Details
(l inadvertently left these out of the original entry)
The change is to upgrade the Myricom ethernet adapter drivers for the DAQ broadcast network to version 1.5.3.p3, compiling them with the MYRI10GE_ALLOC_ORDER=2 option and using the big_rxring firmware at driver load. This is to attempt to reduce the number of dropped frames that are seen occasionally, most often on the framewriters, that also trigger 'retransmission request' errors in the daqd log. And additionally, on the data concentrator to make use of the MYRI10GE_THROTTLE option to see if tuning the packet emission rate has any effect for the receiving systems. The primary method of measuring any change is to use the SNMP monitoring of the DAQ broadcast switch to monitor the dropped/paused frames per host port. The same changes on the test stand indicate some improvement.