IPMP configuration and detach problem

gxmsgx · August 11, 2009, 8:31am

[LEFT]Hi,
I've a problem when try to detach nic e1000g1.

IPMP configuration:

# ifconfig -a
...cut...
e1000g1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 7
        inet 0.0.0.0 netmask 0
        ether 0:c:29:67:16:ef
e1000g2: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 6
        inet 0.0.0.0 netmask 0
        ether 0:c:29:67:16:f9 

# cat /etc/hosts
...cut...
192.168.72.30   clab-cl
192.168.72.33   clab-e1000g1 
192.168.72.34   clab-e1000g2 

# eeprom local-mac-address?
local-mac-address?=true

# cat /etc/hostname.e1000g1
clab-e1000g1 netmask + broadcast + \
group ipmp0 deprecated -failover up \
addif clab-cl netmask + broadcast + failover up

# cat /etc/hostname.e1000g2
ether 0:c:29:67:16:ef
clab-e1000g2 netmask + broadcast + \
group ipmp0 deprecated -failover up \
addif clab-cl netmask + broadcast + failover up

# svcadm restart physical:default
/var/adm/messages
clab2 Cluster.PNM: [ID 185191 daemon.error] MAC addresses are not unique per subnet.

# ifconfig -a
...cut...
e1000g1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 11
        inet 192.168.72.33 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:ef 
e1000g1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 11
        inet 192.168.72.30 netmask ffffff00 broadcast 192.168.72.255
e1000g2: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 12
        inet 192.168.72.34 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:ef 
e1000g2:1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 12
        inet 192.168.72.30 netmask ffffff00 broadcast 192.168.72.255

Detach test (e1000g2,e1000g1)

# if_mpadm -d e1000g2
/var/adm/messages
clab2 in.mpathd[475]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g2 to NIC e1000g1

# ifconfig -a
...cut...
e1000g1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 15
        inet 192.168.72.33 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:ef 
e1000g1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 15
        inet 192.168.72.30 netmask ffffff00 broadcast 192.168.72.255
e1000g1:2: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 15
        inet 192.168.72.30 netmask ffffff00 broadcast 192.168.72.255
e1000g2: flags=89040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,OFFLINE> mtu 1500 index 16
        inet 192.168.72.34 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:ef 

# if_mpadm -r e1000g2
/var/adm/messages
clab2 in.mpathd[475]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g2

# if_mpadm -d e1000g1

# ifconfig -a
...cut...
e1000g1: flags=89040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,OFFLINE> mtu 1500 index 15
        inet 192.168.72.33 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:ef 
e1000g2: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 16
        inet 192.168.72.34 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:ef 
e1000g2:1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 16
        inet 192.168.72.30 netmask ffffff00 broadcast 192.168.72.255
e1000g2:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 16
        inet 192.168.72.30 netmask ffffff00 broadcast 192.168.72.255

/var/adm/messages
clab2 in.mpathd[475]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g1 to NIC e1000g2

...and after six seconds...
clab2 in.mpathd[475]: [ID 594170 daemon.error] NIC failure detected on e1000g2 of group ipmp0
clab2 Cluster.PNM: [ID 890413 daemon.notice] ipmp0: state transition from OK to DOWN.

Where am I wrong?
[/LEFT]

DukeNuke2 · August 11, 2009, 10:53am

disable ipmp and use a normal ip configuration. set the first interface up and try to ping your default router. if successfull try the same with the second interface.

can you reach the router from both interfaces?

incredible · August 11, 2009, 10:56am

Besides the above error, what else is seen in the messages file?

gxmsgx · August 11, 2009, 11:42am

SOLVED

1) delete line "setprop local-mac-address? 'true'" from /boot/solaris/bootenv.rc
2) delete line "ether 0:c:29:67:16:ef" from /etc/hostname.e1000g2
3) reboot

# ifconfig -a
...cut...
e1000g1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3
        inet 192.168.72.33 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:ef 
e1000g1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet 192.168.72.40 netmask ffffff00 broadcast 192.168.72.255
e1000g2: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 4
        inet 192.168.72.34 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:f9 
e1000g2:1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
        inet 192.168.72.40 netmask ffffff00 broadcast 192.168.72.255

Detach test (e1000g2,e1000g1)

# if_mpadm -d e1000g2
/var/adm/messages
clab2 in.mpathd[470]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g2 to NIC e1000g1

# ifconfig -a
...cut...
e1000g1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3
        inet 192.168.72.33 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:ef 
e1000g1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet 192.168.72.40 netmask ffffff00 broadcast 192.168.72.255
e1000g1:2: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet 192.168.72.40 netmask ffffff00 broadcast 192.168.72.255
e1000g2: flags=89040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,OFFLINE> mtu 1500 index 4
        inet 192.168.72.34 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:f9 

# if_mpadm -r e1000g2
/var/adm/messages
clab2 in.mpathd[470]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g2

# if_mpadm -d e1000g1
/var/adm/messages
clab2 in.mpathd[470]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g1 to NIC e1000g2

# ifconfig -a
...cut...
e1000g1: flags=89040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,OFFLINE> mtu 1500 index 3
        inet 192.168.72.33 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:ef 
e1000g2: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 4
        inet 192.168.72.34 netmask ffffff00 broadcast 192.168.72.255
        groupname ipmp0
        ether 0:c:29:67:16:f9 
e1000g2:1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
        inet 192.168.72.40 netmask ffffff00 broadcast 192.168.72.255
e1000g2:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
        inet 192.168.72.40 netmask ffffff00 broadcast 192.168.72.255

# if_mpadm -r e1000g1
/var/adm/messages
clab2 in.mpathd[470]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g1

thanks to all