AIX networking performance, during copying files

UNIX AIX
1

Hello, AIX-gurus! We have some AIX networking (?) issue, for test I created 2 LPARs (AIX 6.1) on 2 different blade IBM power servers (p702 with POWER7 on board), located in same BladeCenter H.

Networking bandwith between them is 1Gbit/s

# iperf -c 172.16.17.170
------------------------------------------------------------
Client connecting to 172.16.17.170, TCP port 5001
TCP window size: 63.6 KByte (default)
------------------------------------------------------------
[  3] local 172.16.17.53 port 37113 connected with 172.16.17.170 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.09 GBytes   935 Mbits/sec

But when I copy files with scp, even with fastest available cipher arcfour, it shows about 30 Mb/s (about 240 MBit/s bandwidth),

scp -o Cipher=arcfour root@172.16.17.170:/u01/exp/bigfile.dmp /u01/backup/2.dmp
The authenticity of host '172.16.17.170 (172.16.17.170)' can't be established.
RSA key fingerprint is SHA256:EuxnUV1Ru1nzf9RI0j6ui+tcC1vFGXrU0fLnVzJw0Ds.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.16.17.170' (RSA) to the list of known hosts.
root@172.16.17.170's password: 
bigfile.dmp                                                                3% 2009MB  28.8MB/s   28:44 ETAK

I even tried rsync-daemon, which is not using encryption, unlike regular rsync, it gives about 36 Mb/s (same bandwidth as scp)

rsync -aHAxv --numeric-ids --progress rsync://172.16.17.170:/big-archive/bigfile.dmp /u01/backup/1.dmp
receiving incremental file list
bigfile.dmp
  2,212,724,736   4%   36.28MB/s    0:23:19

I think it's AIX issue, because we have some x86_64 machines in this network, and when I use scp on them, with same cipher I recieve about 80 Mb/s-100Mb/s download speed.

2

I would be curious about cpio's results have you tried yet?

3 
time ssh -o Cipher=arcfour oracle@172.16.17.170 "ls /u01/exp/3.dmp | cpio -ov" | cpio -imvd
oracle@172.16.17.170's password: 
/u01/exp/3.dmp
2097153 blocks 
/u01/exp/3.dmp
2097153 blocks 

real    0m38.634s
user    0m18.829s
sys     0m5.865s
bash-4.4# du -sg 3.dmp 
1.00    3.dmp

So, it's 1 Gb transferred by 38s, which is equal ~27 MB/s average download speed, about 216 Mbit/s bandwidth, if I made calculations right

4

ssh, scp et al have high CPU overhead from encryption and aren't representative of down-to-the-wire performance.

You also have your disks to consider. How fast would that happen if you wrote to /dev/null instead of ssh? Also, caching. Does it speed up when you run it twice?

1 Like
5 

du -sg 3.dmp 
1.00    3.dmp
bash-4.4# time ls ./3.dmp | cpio -ovB | cpio -imvdB /dev/null
3.dmp
20971602097160 blocks 
 blocks 

real    0m3.405s
user    0m2.064s
sys     0m0.934s

1 Gb for 3,4s, or ~300 MB/s (~30MB/s with ssh in comparsion).

No, I don't observe such bechavior.

6

Easiest utility I know to throw unencrypted content down the wire ASAP is udpcast. Of course, it's wide-open and limited to your local subnet. It can have more than one receiver with no bandwidth penalty however.

# sender
udp-sender < filename

# receiver
udp-receiver > /dev/null