in.mpathd Cannot meet requested failure detection time

Hello World

I am facing following issue on machine

HW:
Sun Fire X4200 M2
OS:
Solaris 10/08 s10x_u6wos_07b X86

Errors:
Jun 28 08:11:46 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 24528 ms on (inet nge1) for group "prd"
Jun 28 08:11:46 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 12264 ms on (inet e1000g1) for group "prd"
Jun 28 08:11:47 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 10000 ms on (inet nge1) for group "prd"
Jun 28 10:45:33 backupsrv in.mpathd[197]: [ID 585766 daemon.error] Cannot meet requested failure detection time of 10000 ms on (inet nge1) new failure detection time for group "prd" is 41230 ms
Jun 28 10:46:33 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 20615 ms on (inet nge1) for group "prd"
Jun 28 10:46:33 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 10307 ms on (inet e1000g1) for group "prd"
Jun 28 10:46:35 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 10000 ms on (inet e1000g1) for group "prd"
Jun 28 15:01:27 backupsrv in.mpathd[197]: [ID 594170 daemon.error] NIC failure detected on nge1 of group prd
Jun 28 15:01:27 backupsrv in.mpathd[197]: [ID 832587 daemon.error] Successfully failed over from NIC nge1 to NIC e1000g1
Jun 28 15:01:29 backupsrv in.mpathd[197]: [ID 299542 daemon.error] NIC repair detected on nge1 of group prd
Jun 28 15:01:29 backupsrv in.mpathd[197]: [ID 620804 daemon.error] Successfully failed back to NIC nge1
Jun 28 15:02:27 backupsrv in.mpathd[197]: [ID 585766 daemon.error] Cannot meet requested failure detection time of 10000 ms on (inet e1000g1) new failure detection time for group "prd" is 153664 ms
Jun 28 15:03:27 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 76832 ms on (inet e1000g1) for group "prd"
Jun 28 15:03:27 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 38416 ms on (inet nge1) for group "prd"
Jun 28 15:03:28 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 19208 ms on (inet nge1) for group "prd"
Jun 28 15:03:29 backupsrv in.mpathd[197]: [ID 302819 daemon.error] Improved failure detection time 10000 ms on (inet e1000g1) for group "prd"

I have checked there is no issue on switch on which these interfaces are connected.
No crc errors.
These interfaces are full duplex and 1000Mbps autoneg on on both machine and switch
I don't know why such errors pop up everyday

:confused:
Below is network config:

root@backupsrv# uname -a
SunOS backupsrv 5.10 Generic_138889-03 i86pc i386 i86pc
root@backupsrv# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000 
e1000g0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 2
        inet 192.168.255.2 netmask ffffff00 broadcast 192.168.255.255
        ether 0:21:28:10:63:6c 
e1000g1: flags=269040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY,INACTIVE,CoS> mtu 1500 index 3
        inet 172.18.190.26 netmask ffffffe0 broadcast 172.18.190.31
        groupname prd
        ether 0:21:28:10:63:6d 
nge0: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 4
        inet 10.20.30.90 netmask ffffff00 broadcast 10.20.30.255
        ether 0:21:28:10:63:6a 
nge1: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 5
        inet 172.18.190.27 netmask ffffffe0 broadcast 172.18.190.31
        groupname prd
        ether 0:21:28:10:63:6b 
nge1:1: flags=209040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,CoS> mtu 1500 index 5
        inet 172.18.190.25 netmask ffffffe0 broadcast 172.18.190.31
nxge0: flags=1201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS,FIXEDMTU> mtu 9000 index 6
        inet 192.168.254.2 netmask ffffff00 broadcast 192.168.254.255
        ether 0:21:28:1e:90:50 
root@backupsrv# netstat -in
Name  Mtu  Net/Dest      Address        Ipkts  Ierrs Opkts  Oerrs Collis Queue 
lo0   8232 127.0.0.0     127.0.0.1      29572877 0     29572877 0     0      0     
e1000g0 1500 192.168.255.0 192.168.255.2  823122402 0     331691531 0     0      0     
e1000g1 1500 172.18.190.0  172.18.190.26  312615 0     301785 0     0      0     
nge0  1500 10.20.30.0    10.20.30.90    501750 0     130462 0     0      0     
nge1  1500 172.18.190.0  172.18.190.27  38355391 0     47160563 0     0      0     
nxge0 9000 192.168.254.0 192.168.254.2  126187049 0     64678723 0     0      0     
 
root@backupsrv# netsat -nr
bash: netsat: command not found
root@backupsrv# netstat -nr
 
Routing Table: IPv4
  Destination           Gateway           Flags  Ref     Use     Interface 
-------------------- -------------------- ----- ----- ---------- --------- 
default              172.18.190.14        UG        1        783           
10.20.30.0           10.20.30.90          U         1        360 nge0      
172.18.190.0         172.18.190.27        U         1        323 nge1      
172.18.190.0         172.18.190.25        U         1          0 nge1:1    
172.18.190.0         172.18.190.26        U         1        197 e1000g1   
192.168.254.0        192.168.254.2        U         1        205 nxge0     
192.168.255.0        192.168.255.2        U         1       1135 e1000g0   
224.0.0.0            172.18.190.27        U         1          0 nge1      
127.0.0.1            127.0.0.1            UH      863    4619931 lo0     

Man Page for in.mpathd (All Section 1m) - The UNIX and Linux Forums

Cannot meet requested failure detection time of time ms on (inet[6]
interface_name) new failure detection time for group group_name is time
ms
Description:

 The round trip time for ICMP probes is higher than necessary to
 maintain the current failure detection time. The network is proba-
 bly congested or the probe targets are loaded. in.mpathd automati-
 cally increases the failure detection time to whatever it can
 achieve under these conditions.

Improved failure detection time time ms on (inet[6] interface_name) for
group group_name
Description:

 The round trip time for ICMP probes has now decreased and in.mpathd
 has lowered the failure detection time correspondingly.

Congestion and too aggressive failover configuration? If an ethernet fails over, the old will be in Windows arp cache 5 minutes as I recall, soooooooo . . . .

I have faced the same sometime back.
If no hardware problem on your side its better check with network admins to see if theres any firewall or network/traffic problem..

This is fairly normal for ipmp ..note:
Improved failure detection time 245*** .. the detection time is getting shorter..

There should not be anything wrong there ..