Sunfire x4600 M2 NIC replacement

Rockyc3400 · July 8, 2014, 8:57am

Hi All,

I am having an Sun fire X4600 M2 server where one of the interfaces is down.
I had requested for the replacement but had a bit problem in identifying the port.
As per the server service module, there are 4 NIC ports and one port is not flashing (e1000g0) but as per server, the port e1000g0 is fine but e1000g2 is down.

Also as per service manual, there are only for ports for this server but in the server output, i have 6 interfaces.

e1000g0         link: up        speed: 1000  Mbps       duplex: full
e1000g1         link: up        speed: 1000  Mbps       duplex: full
e1000g2         link: down      speed: 0     Mbps       duplex: half
e1000g3         link: up        speed: 1000  Mbps       duplex: full
e1000g4         link: up        speed: 1000  Mbps       duplex: full
e1000g5         link: up        speed: 1000  Mbps       duplex: full

e1000g0: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
e1000g0:1: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2
e1000g1: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu 1500 index 7
e1000g2: flags=xxxxxxx<UP,BROADCAST,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,FAILED> mtu 1500 index 3
e1000g3: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu 1500 index 6
e1000g4: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
e1000g4:1: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 4
e1000g4:2: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 4
e1000g5: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 5
e1000g5:1: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 5
e1000g5:2: flags=xxxxxxx<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 5.

Could you please help me in understanding this and how to proceed further with this to replace the NIC cable.

Regards,
Rocky

DukeNuke2 · July 8, 2014, 9:17am

Please post the output of prtdiag -v and dladm show-dev .
The dladm command can differ, depending on the Solaris version you are using.

Rockyc3400 · July 9, 2014, 2:15am

Hi Duke,

Below are the outputs of dladm show-dev and prtdiag -v

e1000g0         link: up        speed: 1000  Mbps       duplex: full
e1000g1         link: up        speed: 1000  Mbps       duplex: full
e1000g2         link: down      speed: 0     Mbps       duplex: half
e1000g3         link: up        speed: 1000  Mbps       duplex: full
e1000g4         link: up        speed: 1000  Mbps       duplex: full
e1000g5         link: up        speed: 1000  Mbps       duplex: full

BMC Configuration: IPMI 1.5 (KCS: Keyboard Controller Style)
==== Processor Sockets ====================================
Version                          Location Tag
-------------------------------- --------------------------
Dual-Core AMD Opteron(tm) Processor 8220 SE CPU 1
Dual-Core AMD Opteron(tm) Processor 8220 SE CPU 2
Dual-Core AMD Opteron(tm) Processor 8220 SE CPU 3
Dual-Core AMD Opteron(tm) Processor 8220 SE CPU 4
==== Memory Device Sockets ================================
Type    Status Set Device Locator      Bank Locator
------- ------ --- ------------------- --------------------
DDR2    in use 0   DIMM0               BANK0
DDR2    in use 0   DIMM1               BANK1
DDR2    in use 0   DIMM2               BANK2
DDR2    in use 0   DIMM3               BANK3
unknown empty  0   DIMM4               BANK4
unknown empty  0   DIMM5               BANK5
unknown empty  0   DIMM6               BANK6
unknown empty  0   DIMM7               BANK7
DDR2    in use 0   DIMM8               BANK8
DDR2    in use 0   DIMM9               BANK9
DDR2    in use 0   DIMM10              BANK10
DDR2    in use 0   DIMM11              BANK11
unknown empty  0   DIMM12              BANK12
unknown empty  0   DIMM13              BANK13
unknown empty  0   DIMM14              BANK14
unknown empty  0   DIMM15              BANK15
DDR2    in use 0   DIMM16              BANK16
DDR2    in use 0   DIMM17              BANK17
DDR2    in use 0   DIMM18              BANK18
DDR2    in use 0   DIMM19              BANK19
unknown empty  0   DIMM20              BANK20
unknown empty  0   DIMM21              BANK21
unknown empty  0   DIMM22              BANK22
unknown empty  0   DIMM23              BANK23
DDR2    in use 0   DIMM24              BANK24
DDR2    in use 0   DIMM25              BANK25
DDR2    in use 0   DIMM26              BANK26
DDR2    in use 0   DIMM27              BANK27
unknown empty  0   DIMM28              BANK28
unknown empty  0   DIMM29              BANK29
unknown empty  0   DIMM30              BANK30
unknown empty  0   DIMM31              BANK31
unknown empty  0   DIMM32              BANK32
unknown empty  0   DIMM33              BANK33
unknown empty  0   DIMM34              BANK34
unknown empty  0   DIMM35              BANK35
unknown empty  0   DIMM36              BANK36
unknown empty  0   DIMM37              BANK37
unknown empty  0   DIMM38              BANK38
unknown empty  0   DIMM39              BANK39
unknown empty  0   DIMM40              BANK40
unknown empty  0   DIMM41              BANK41
unknown empty  0   DIMM42              BANK42
unknown empty  0   DIMM43              BANK43
unknown empty  0   DIMM44              BANK44
unknown empty  0   DIMM45              BANK45
unknown empty  0   DIMM46              BANK46
unknown empty  0   DIMM47              BANK47
unknown empty  0   DIMM48              BANK48
unknown empty  0   DIMM49              BANK49
unknown empty  0   DIMM50              BANK50
unknown empty  0   DIMM51              BANK51
unknown empty  0   DIMM52              BANK52
unknown empty  0   DIMM53              BANK53
unknown empty  0   DIMM54              BANK54
unknown empty  0   DIMM55              BANK55
unknown empty  0   DIMM56              BANK56
unknown empty  0   DIMM57              BANK57
unknown empty  0   DIMM58              BANK58
unknown empty  0   DIMM59              BANK59
unknown empty  0   DIMM60              BANK60
unknown empty  0   DIMM61              BANK61
unknown empty  0   DIMM62              BANK62
unknown empty  0   DIMM63              BANK63
==== On-Board Devices =====================================
 LSI serial-ATA #1
 Gigabit Ethernet #1
 Gigabit Ethernet #2
 ATI Rage XL VGA
==== Upgradeable Slots ====================================
ID  Status    Type             Description
--- --------- ---------------- ----------------------------
1   available PCI-X            PCIX SLOT1
2   in use    PCI-X            PCIX SLOT2
3   available PCI Express      PCIExp SLOT3
4   in use    PCI Express      PCIExp SLOT4
5   in use    PCI Express      PCIExp SLOT5
6   available PCI Express      PCIExp SLOT6
7   available PCI Express      PCIExp SLOT7
8   available PCI Express      PCIExp SLOT8

DukeNuke2 · July 9, 2014, 2:46am

I would say you have an extra PCI card with two network interfaces in your system.

Check the device pathes to your NICs in /etc/path_to_inst and see if one of the onboard devices is e1000g2. Here is a list of the onboard NIC device path:

Network 0 (e1000g0)   /pci@0,0/pci1022,7458@10/pci8086,1011@1
Network 1 (e1000g1)   /pci@0,0/pci1022,7458@10/pci8086,1011@1,1
Network 2 (e1000g2)   /pci@0,0/pci1022,7458@10/pci8086,1011@2
Network 3 (e1000g3)   /pci@0,0/pci1022,7458@10/pci8086,1011@2,1

Rockyc3400 · July 9, 2014, 3:22am

Duke,

I was able to get the data from path_to_inst but could not understand this.

"/pci@0,0" 0 "npe"
"/pci@0,0/pci10de,5d@e" 3 "pcie_pci"
"/pci@0,0/pci10de,5d@e" 1 "pcieb"
"/pci@0,0/pci10de,5d@e/pci108e,125e@0" 0 "e1000g"
"/pci@0,0/pci10de,5d@e/pci108e,125e@0,1" 1 "e1000g"
"/pci@0,0/pci1022,7458@10" 1 "pci_pci"
"/pci@0,0/pci1022,7458@10/pci8086,1011@1" 2 "e1000g"
"/pci@0,0/pci1022,7458@10/pci8086,1011@1,1" 3 "e1000g"
"/pci@0,0/pci1022,7458@10/pci8086,1011@2" 4 "e1000g"
"/pci@0,0/pci1022,7458@10/pci8086,1011@2,1" 5 "e1000g"
"/pci@0,0/pci10de,5c@9" 0 "pci_pci"

Rocky

DukeNuke2 · July 9, 2014, 6:32am

Instance 2 of e1000g is the one that is not working. This is the first network port on the motherboard assembly of your server... So, you need to check the cable and the switch port which is running to the net0 port on the server (not PCI slot), should be the bottom left! If everything is ok and the port is still not working, the port might be dead and you need to replace the servers mainboard.

Rockyc3400 · July 9, 2014, 8:25am

Duke,

I had requested a check and media team person also says the same(bottom left) port is not blinking and that is the one which needs to be replaced.

But my doubt is that when i went through service manual of this server(could not paste the pictures), it says bottom left is e1000g0 and upper left is e1000g2. So as per your analysis and service manual, e1000g0 should be replaced but the server says e1000g2 is down.
I am totally confused at this stage as to how to proceed also when i hit dladm, it shows 6 devices now from where did the other two come into picture when the server has only four ports.

I am very much afraid and much taking so much clarification because this is a prod and there is no link aggregation.

Rocky

DukeNuke2 · July 9, 2014, 8:28am

I think I explained everything in my older posts... Please read everything carefully and come back if you still don't understand what is happening!

Rockyc3400 · July 24, 2014, 3:55am

Hi Duke,

Thanks for your support regarding this. I had checked with media team and still there has been a confusion. Hence we have asked hardware vendor engineer's help to investigate and suggest for replacement (if possible with an site visit).

Regards,
Rocky