The iligo cds main NFS server hanford2 suffered a disk failure at 00:05 this morning. The disk is in the external 16 disk-raid scsi unit. The spare disk was then taken by the system and is in the process of being regenerated. It is currently 53% complete 17.5 hours later, so should take another 17 hours (10am tomorrow). One concerning point is that two of the drives are not being accessed, the failed disk (sdk) but also a member of the raid (sdg) which should be used to the same degree as all the others. I'll let the regen complete before investigating whats going on with sdg (syslog does not report two disk failures).