Reports until 12:49, Monday 14 October 2019
H1 CDS
david.barker@LIGO.ORG - posted 12:49, Monday 14 October 2019 - last comment - 17:56, Monday 14 October 2019(52458)
EY Dolphin switch replacement

Jonathan, Erik, Dave:

WP8354

This morning we replaced the EY Dolphin IXS600 switch (with bad ports) with a new unit. We then saw h1cdsrfm problems, it could not activate its EY IXH611 card. After various reboots/restarts (which I'll detail later) we put the original switch back in.

At this time all the hardware is identical to early this morning, but h1cdsrfm still has issues with its EY IXH611 card.

Comments related to this report
jonathan.hanks@LIGO.ORG - 12:59, Monday 14 October 2019 (52459)
Link 

We ran dis_diag on h1cdsrfm to see if there was anything useful.  One note, the temperature on the 2nd adapter (EY) is high.

For a comparison we also ran dis_diag on h1iscey which is on the same rack to give a comparison.
Non-image files attached to this comment
jonathan.hanks@LIGO.ORG - 17:08, Monday 14 October 2019 (52464)
Link 

I've attached some diagnostic information from h1cdsrfm.

Output from dis_admin_commandline

MACHINE NAME              : h1cdsrfm
SERIAL NUMBER             : IXH-002348
ADAPTER TYPE              : IXH611
CONNECTED                 : Yes
ONLY ETHERNET OUT         : No
DISHOSTS NODEID           : 104
ACTUAL NODEID             : 104
ACTUAL ADAPTER STATE      : Disabled
NUMBER OF PORTS           : 1
PREFETCH MEM SIZE         : 256
CURRENT CONFIGURATION     : Ok
PARITY ERRORS             : 0
SYSTEM ERRORS             : 0
PORT link width           : x8
PORT active               : No
PORT protected            : Yes
PORT operational          : No
PORT state                : Disabled
Partner information is    : invalid
Partner adapter number    : 0
Partner port number       : 0
Partner board type        : Not determined
Partner number of ports   : 0


Output from the IX Switch's info command

LINK1_STATUS         : G1 x8 (1/0)
LINK2_STATUS         : G1 x8 (1/0)
LINK3_STATUS         : G1 x8 (1/0)
LINK4_STATUS         : NC
LINK5_STATUS         : G2 x8 (0/0)      ---- This is cdsrfm
LINK6_STATUS         : NC
LINK7_STATUS         : NC
LINK8_STATUS         : NC
Non-image files attached to this comment
jonathan.hanks@LIGO.ORG - 17:17, Monday 14 October 2019 (52465)
Link 

The temperature has stayed high for the 2nd adapter.  Another interesting point is the state of the topology settings.  Adapter 2 is switch topology with topology auto detect disabled.  Where the other adapters where topology switch with topology auto detect off.
jonathan.hanks@LIGO.ORG - 17:56, Monday 14 October 2019 (52467)
Link 

From the dis_nodemgr.log file.  Note that there are 2 calls to dis_config for adapter 2, the last one sets the topology to 4 (not 1 which would be switch).

Oct 14 2019 15:43:40 Unconfigured adapter with serialno 0x93d  (is valid) found (adapterno 100)
Oct 14 2019 15:43:40 Unconfigured adapter with serialno 0x928  (is valid) found (adapterno 101)
Oct 14 2019 15:43:40 Unconfigured adapter with serialno 0x92c  (is valid) found (adapterno 102)
Oct 14 2019 15:43:40 Adapter number changed: Adapter[0] adap_conf.adapter_no 0 adap_stat.adapter_no 100
Oct 14 2019 15:43:40 Setting NEW link width 8 OLD was -1 (shadow=118)
Oct 14 2019 15:43:40 Adapter 0 AdapterNo 0 Setting NEW nodeid 96 OLD was -1
Oct 14 2019 15:43:40 set dis_config: call: /opt/DIS_5_7_0/sbin/../sbin/dis_config -s 2365 -a 0 -slw 8 -n 96 -slw 8
Oct 14 2019 15:43:40 Adapter number changed: Adapter[1] adap_conf.adapter_no 1 adap_stat.adapter_no 101
Oct 14 2019 15:43:40 Setting NEW link width 8 OLD was -1 (shadow=8)
Oct 14 2019 15:43:40 Adapter 1 AdapterNo 1 Setting NEW nodeid 100 OLD was -1
Oct 14 2019 15:43:40 set dis_config: call: /opt/DIS_5_7_0/sbin/../sbin/dis_config -s 2344 -a 1 -slw 8 -n 100 -slw 8
Oct 14 2019 15:43:40 Adapter number changed: Adapter[2] adap_conf.adapter_no 2 adap_stat.adapter_no 102
Oct 14 2019 15:43:40 Setting NEW link width 8 OLD was -1 (shadow=8)
Oct 14 2019 15:43:40 Adapter 2 AdapterNo 2 Setting NEW nodeid 104 OLD was -1
Oct 14 2019 15:43:40 set dis_config: call: /opt/DIS_5_7_0/sbin/../sbin/dis_config -s 2348 -a 2 -slw 8 -n 104 -slw 8
Oct 14 2019 15:43:40 Adapter 0 configured with 24 reachables
Oct 14 2019 15:43:40 Adapter 1 configured with 24 reachables
Oct 14 2019 15:43:40 Adapter 2 configured with 24 reachables
Oct 14 2019 15:44:01 Setting NEW topology 4 OLD was 0
Oct 14 2019 15:44:01 Adapter 2 AdapterNo 2 Setting NEW nodeid 104 OLD was 104
Oct 14 2019 15:44:01 set dis_config: call: /opt/DIS_5_7_0/sbin/../sbin/dis_config -s 2348 -a 2 -stt 4 -n 104 -slw 8