Reports until 16:30, Wednesday 18 September 2013
H1 CDS
cyrus.reed@LIGO.ORG - posted 16:30, Wednesday 18 September 2013 - last comment - 16:39, Wednesday 18 September 2013(7797)
NTP Server now stratum 2 (antenna/cable failure)
It appears that either the antenna or cable has failed for the control room NTP server, as indicated by the failure LED on the amplifier interface (could also be the interface...).  This caused the NTP server to drop to stratum 16, meaning clients would no longer sync to it (particularly after rebooting it in the course of troubleshooting).  I have restored the default configuration to the NTP server, in which it will also reference three public NTP servers operated by Symmetricom.  This means the NTP server is now a stratum 2 server, rather than stratum 1 as it would be with the GPS reference operating, but it will at least provide time again.  In the process I also swapped the lan 1 and 2 connections so that the admin interface is accessible where it should be and allow the external NTP servers to be used.  Lan2 (now) provides a legacy NTP IP for old CDS equipment which will eventually go away.  I will need to revisit the NTP configuration again when the antenna is operational to see if it is feasible to leave the other NTP references in place as a backup, or if it is possible for them to negatively influence the GPS derived clock in which case they will need to be removed.
Comments related to this report
james.batch@LIGO.ORG - 16:39, Wednesday 18 September 2013 (7798)
As a result of the NTP server antenna failure, the time of day on opsws4 and opsws5 computers needed to be reset, as they were off by several seconds which was enough to stop diag from operating.  Also reset the time of h1iscey and h1iscex which were rebooted during the time when no suitable ntp server could be found, as a result their time of day clocks were off by several hours.