Lesser bandwidth while copying files from 1 RHEL server to abother.

I am facing an issue of getting lesser band width while copying file from RHEL server to another RHEL machine.In our system , there are 4 no of RHEL 5.9 Enterprise server and 2 network interface in each servers are bonded.
Out of 4, 2 machines (Server A and Server B) are with in same network and other 2s (Server C and Server D) are in different network connected via WAN link in some other location.These 2 server in other location is also inside the same network.
Now when I am copying file through SCP from Server A to Server C or Server A to Server D, it copies files at speed of 15 MB/Seconds.
But when I am copying files from Server B to Server C and server B to Server D, I am getting 2 MB / Second. Bonding conf. files are identical for all the 4 server and all 4 servers Ethernet interfaces are showing 1 GBPs 9sseen through ethtool). I am not able to find out the way how to troubleshoot this issue.Can somebody can give some clue to troubleshoot the issue.

Just a few things that come to mind:

  • Are the files the same files and size, are they identical (servers A and B)?
  • What filesystem are they being copied from, are they identical (servers A and B)?
  • Are there low level errors (ethernet packet drops) or higher level errors (tcp/ip retransmits) (server B)?
  • Is LACP configured well at the switch level (server B)?
  • Are all ethernet cables well connected (server B)?
  • Are all interfaces at the right speed (server B)?
  • Do servers A and B have the same default route ?

Other things to consider:-

  • Is the network port settings correct, i.e. matching your NIC settings? (preferred and actual) Have a look at the output from ethstat or ethtool or whatever it is for your OS.
  • Is there is any packet shaping/smoothing being done? There could be all sorts of network rules in play that would be invisible to you.

I hope that these help,
Robin