Copy huge files system

Hi Mr. AIX,

We'll find you a good command to copy your data but I'm a little concerned you're going to fill up your file system or something!

Let's make sure the basics are OK before we lump 250GB around.

Can you show the output of:

df -k /sfsapp
df -k /tgtapp
lsattr -El sys0 | grep realmem

The command you posted doesn't match the one the one juredd1 posted. This will do a test run of an rsync:

rsync -na /sfsapp/ /tgtapp

This is the output .

 
 
root>df -k /sfsapp
/dev/sfsapplv   212129280  36861824   80% 15982043    43% /sfsapp
 
root>df -k /tgtapp
/dev/tgtapplv   212129280 158892800   26%  6195440    49% /tgtapp
 
 
root>lsattr -El sys0 | grep realmem
realmem         10485760           Amount of usable physical memory in       Kbytes        False
 
 
 
root>rsync -na /sfsapp/ /tgtapp/  
 
ERROR: out of memory in make_file [sender]
rsync error: error allocating core memory buffers (code 22) at util.c(117) [sender=3.0.6]
 
 
 
 
 

OK, you convinced me your data is huge. :slight_smile: You must have many files/folders/links. I have a file system like this. I have to rsync subfolders to keep it from failing.

If /tgtapp is being used for other things:

  • You may not have enough free space.
  • You probably have some mess to clean up from your prior attempts.

If /tgtapp is a new file system just for this purpose, clean it up. This deletes everything so make sure you mean it.

rm -fr /tgtapp/*

I would expect the commands from bakunin and methyl to work.

cd /sfsapp; tar cpf - . | (cd /tgtapp; tar xpvf -)
cd /sfsapp; find . -xdev -print | cpio -pdumv /tgtapp

You didn't mention any errors from the tar/cpio you tried. Did it complete silently?

I put the 'v' (verbose) option on the last commands. For now it's probably worth seeing what it's doing. Shrink down your window to speed up the transfer.

as long as both filesystems are on the same box - mount the filesystems both with nolog and noatime and a simple cp -hpr will do the trick for you probably as good as any of the other commands ... make sure that the VG is a scalable one ...

Regards
zxmaus

/tgtapp/ has the same space of /sfsapp/

I have removed the /tgtapp/ then I re create it again now it's empty it has the same space of /sfsapp

now I'm using this command

 
cd /sfsapp; tar cpf - . | (cd /tgtapp; tar xpvf -)

it's running after some time got hanged and it's seems is not running.

and now I'm using this command

 
cd /sfsapp; find . -xdev -print | cpio -pdumv /tgtapp

it's running .. it seems will get hang also .. let's see ..

---------- Post updated at 02:36 AM ---------- Previous update was at 02:36 AM ----------

both of the below commands got hanged ..

 
cd /sfsapp; tar cpf - . | (cd /tgtapp; tar xpvf -) 
 
cd /sfsapp; find . -xdev -print | cpio -pdumv /tgtapp

I'm looking for strong command to copy the data from /sfsapp to /tgtapp

Pls help ..

I'm wondering if the source file system has errors. Can you unmount it to perform an fsck or is it in production use?

The copies should have shown you which files they were on.

Did they hang on the same file / folder? Similar size transferred? If so, is there anything special about that folder / file?

What made you decide they were hung? (i.e. no disk growth, no cpu usage)

How about the copy (cp) command.

Try:

# cp -rph source_directory destination_directory.

I'm using your command , it seems good command ! :slight_smile:

I'll give the update about it !

If you would read your own threads than you would have seen this was suggested earlier - together with a few mount options that would make your copy even faster ... see #24

Regards
zxmaus