Hi
OS centos 7
Server Dell R740XD
Broadcom Adv. Dual 10GBASE-T ethernet
intagreted NIC 1 Port 2: Broadcom Gigabit Ethernet
I am having the following problem on my server with the features I mentioned above.
When I want to give ip to the port on the 10G network card, it does not receive it.
But when I give ip to the integrated nw card, I can also give ip to the 10 Gb network card.
When I drop the ip on the 1G network card, it drops on the 10G2 card
Where could the problem be?
You are probably not the only one. This hardware can be very fussy about add-in cards.
Read this:
If I had this problem I'd probably be inclined to attempt to switch off the integrated NIC completely via the BIOS and then see whether I could control the 10GB NIC.
Does the 10GB interface work if the integrated NIC is disabled?
Maybe your problem is that CentOS uses network manager.
Then you must define a connection with nmcli or nmtui command, and you should not directly assign an IP with the ip command.
Hello,
If the network cards are not managed by NetworkManager, and are still using the old RHEL network interface configuration system, then it may be worth looking at the per-interface configuration files in /etc/sysconfig/network-scripts/ to see if each ifcfg-<interface name> file has the correct DEVICE and HWADDR entries set, just in case the MAC address or device name has gotten mixed up or duplicated in the interface configuration files.
Yes, new problem.
The server was being used with a 1G network card. When I looked at the server, I saw that there was a separate 10g card installed and I wanted to use it.
But when I gave the ip, it received the ip, but when I disable the network interface of the 1G card, the ip I gave to the network interface of the 10G card also goes.
I have to up the internet interface of the 1g card via idrac
I don't use NetworkManager, I do everything through network interface files.
cat ifcfg-em1
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
NM_CONTROLLED=no
IPV4_FAILURE_FATAL=no
##IPV6INIT=yes
##IPV6_AUTOCONF=yes
##IPV6_DEFROUTE=yes
##IPV6_FAILURE_FATAL=no
##IPV6_ADDR_GEN_MODE=stable-privacy
NAME=em1
UUID=***********
DEVICE=em1
ONBOOT=yes
IPADDR=x.x.x.x
NETMASK=x.x.x.x
GATEWAY=x.x.x.x
DNS1=x.x.x.x
DNS2=x.x.x.x
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
NM_CONTROLLED=no
IPV4_FAILURE_FATAL=no
##IPV6INIT=yes
##IPV6_AUTOCONF=yes
##IPV6_DEFROUTE=yes
##IPV6_FAILURE_FATAL=no
##IPV6_ADDR_GEN_MODE=stable-privacy
NAME=p2p1
UUID=
DEVICE=p2p1
ONBOOT=no
IPADDR=*****
NETMASK=****
GATEWAY=****
DNS1=****
DNS2=****
ı dont use NetworkManager,
I'm using using the old RHEL network interface configuration system.
everything looks fine in these files.
This is how I always set the settings on all my servers, strangely I only had this problem on this server.
Hello,
Thanks for sharing the interface configuration files. The only two things I'd note are that p2p1 is currently set to not come up on boot (ONBOOT is set to "no" for this interface, but is set to "yes" for the on-board em1), and that there does not appear to be any UUID specified in the config for p2p1. Generally I always ensure that every interface I'm using has either a valid HWADDR or UUID line on a RHEL system, to be absolutely sure that the system can correctly tell the NICs apart.
Is there anything interesting in the output of either ip addr show dev <interface name> and ethtool <interface name> ? When you bring the interfaces up and down, do you see anything in the output of dmesg that might explain this strange behaviour ? And lastly, if you do a grep <interface name> /var/log/dmesg, were there any unusual messages at boot time for either or both interfaces ?
grep em1 /var/log/dmesg
[ 3.919860] tg3 0000:18:00.0 em1: renamed from eth0
grep p2p1 /var/log/dmesg
[ 3.430266] bnxt_en 0000:5e:00.0 p2p1: renamed from eth2
ip addr show dev em1
2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether ******** brd ff:ff:ff:ff:ff:ff
inet *******/16 brd ******** scope global em1
valid_lft forever preferred_lft forever
inet6 **********/64 scope link
valid_lft forever preferred_lft forever
Hello,
Thanks - do you have the output of the ethtool checks, as welll as the ip command for p2p1 ? Also, whilst I appreciate you don't want to post potentially identifying information here, can you confirm that the MAC addresses displayed for each NIC are in fact different, and are (as far as you know) correct ?
ethtool em1
Settings for em1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 100baseT/Full
1000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: off
Supports Wake-on: g
Wake-on: d
Current message level: 0x000000ff (255)
drv probe link timer ifdown ifup rx_err tx_err
Link detected: yes
ethtool p2p1
Settings for p2p1:
Supported ports: [ TP ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 12
Transceiver: internal
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000000 (0)
Link detected: no
I have disabled the p2p1 internet interface.
yes the mac addresses are different.
I also added the mac addresses of em1 and p2p1 internet interfaces to the config files but still no luck.
Strangely, only the mac address of em1 appears on the switch.
Check if port(s) on switch are in LACP (or similar grouping) configuration
If so, break such configuration on switch side, and try again.
Are you using bridges on that host ?
Regards
Peasant.