Dan, Greg, Jim, Dave:
Dan found that one of the LDAS Q-Logic fiber channel switches is reporting receive errors. These errors correlate to the SATABOY RAID controller reported errors, and in turn to the fw0 restarts.
We first changed which the long-haul single mode fiber optics pair which is used between buildings. Instead of using the first pair of the A-Block, we moved to the first pair of the B-Block. The change was done at 15:00 PDT this afternoon. This did not fix the problem, h1fw0 restarted 50 minutes later and Dan saw errors on the Q-Logic switch.
Following the philosophy of "one change at a time", we are swapping out the single mode fiber optics cable in the LDAS server room which connects to the Q-Logic switch to the LDAS patch panel. It that does not improve the situation we will do the same in the MSR.