Dear Guy�s
By using dd command or any strong command, I�d like to copy huge data from file system to another file system
Sours File system: /sfsapp
File system has 250 GB of data
Target File system: /tgtapp
I�d like to copy all these files and directories from /sfsapp to /tgtapp as file system copy by using strong command
Pls your advice..
I suppose you want to preserve absolutely everything: files, ownerships, filemodes, ...
I do that (out of habit - presumably there are other equally good ways to do it) using two tar's and a pipeline:
# cd /path/to/sourcedir
# tar -cf - . | (cd /path/to/targetdir ; tar -xf - )
The first tar packs everything to <stdout>, the second one unpacks from <stdin>, this is what the "-" stands for. If you include a "-v" to one tars options you can even watch the progress (at a small expense of performance of course).
I hope this helps.
bakunin
So my command should to be like this ...
cd /sfsapp
tar -cf - . | (cd /tgtapp ; tar -xf - )
---------- Post updated at 07:17 AM ---------- Previous update was at 07:10 AM ----------
But the number of the files is very huge .. Are you sure this is will success ?
Yes, this will work, regardless of the number of files.
You could use "dd" too, but i would prefer to use an "high-level" approach when possible instead of a "low-level" approach such as "dd". With "dd" it is easily possible to overwrite not only files, but devices, volume group information, things quite vital to your system.
With "tar" you deal with files, directories and the like - things which are easily recognized by you. With "dd" you ultimately deal with devices, which is much more dangerous. If you mistype "/tgtpaath" in the above mentioned "tar" command you will run into a "disk full"-error, have the chance to correct it and start over. If you confuse hdisk127 and hdisk126 in a dd command it will work well but probably destroy data you didn't want to lose.
That doesn't mean at all that "dd" is a bad tool - it is very powerful, but the power comes at the price of being dangerous too. Use it when you need it, not when it is possible, is my principle. Maybe i'm feeble, but i have been working as a Sysadmin for a long time and i am still alive. ;-))
I hope this helps.
bakunin
OK .. can you please explain exactly how to can I use dd command to move the files from /sfsapp to /tgtapp
How about using the backup and restore commands?
Are /sfsapp and /tgtapp mountpoints?
Is there anything in /tgtapp already?
Though you don't mention the version of AIX or anything about the hardware of the disc system, do you have an AIX version which can deal with files larger than 2 Gb ?
The usual solution is to use "find" piped to "cpio -p" .
e.g.
cd /sfsapp
find . -xdev -print | cpio -pdumv /tgtapp
I'm talking about dd command, can you please explain about dd command
my AIX version is : AIX 6.1
there is nothing in /tgtapp and it's empty mount point files system has the same size of /sfsapp
/sfsapp and /tgtapp are mount points and each one mointed on it own disk
Quite frankly, i don't understand your obsession with "dd", given that you have been offered several good alternatives. You will not gain anything by using "dd", not speed, not ease of use and definitely not safety. To me that starts to sound like homework.
Here is your solution, use at your own risk, i haven't tested it.
First find out the devices which are mounted at the respective mountpoints with the "mount" command, then use these devices as infile and outfile of "dd".
Example: copy "/ap101" to "/ap102".
# mount
node mounted mounted over vfs date options
-------- --------------- --------------- ------ ------------ ---------------
<...SNIP...>
/dev/lvap101 /ap101 jfs2 Nov 29 11:33 rw,log=/dev/logapdev1vg
/dev/lvap102 /ap102 jfs2 Nov 29 11:33 rw,nodev,nosuid,log=/dev/logapdev1vg
<...SNIP...>
# dd if=/dev/lvap101 of=/dev/lvap102 bs=512
I hope this helps.
bakunin
I have used the below command..
dd if=/dev/sfsapplv of=/dev/tgtapplv bs=512
but it's taking long time more than 30 min's without coping any thing to /tgtapp
After I canceled that , I got the below result
2352183+0 records in.
2352183+0 records out.
Can any one explain.. ?
I have no idea about your disks performance, but a quick estimation tells me: if the target disk can write at 20MB/s continuously then it will take (250.000/20=) 12500 seconds to write 250GB - 12500/3600 ~ 3.5 hours. I doubt your disk (LUN, whatever) can write at the ~140 MB/s (continuous!) necessary to finish the job in 30 minutes.
Could you be bothered to read the man page of "dd", after insisting on this tool, before asking here? It tells you that it has - until you cancelled it - 2352183 pieces of the size given in the command in option "bs" read and written as many. As you issued
dd if=/dev/sfsapplv of=/dev/tgtapplv bs=512
it read 2352183x512=1204317696 bytes and wrote as many.
Again, i don't think it is advisable to use "dd" for that task (even more it is dangerous in the hand of someone not completely knowing what he is doing) and i won't answer questions easily answered by looking at the man page again.
I hope this helps.
bakunin
it's huge data , I have tried to use your suggested command but it got hanged after some time and it did not copy whole the data
do you have any recommended command will be strong command to copy whole the files from /sfsapp to the new mount point /tgtapp
Have you tried the three other suggestions that were posted? If not start there.
You haven't said so until now. If so, what exactly was the error? The method i described (tar) has worked for me countless times, also on such amounts of data and even more.
Using "backup" and "restore", "cpio" or "savevg" will equally work and probably at roughly the same speed as "tar".
One thing though: you can't cancel a job after some time and expect it to have finished the task - if you have to cancel it that means it was still running and if it was running it means it hasn't finished what it was doing.
I hope this helps.
bakunin
I don't know AIX, but with the OS's I know unmounting the destination prior to running the dd is a must. I would unmount the source as well if I could. This is one of several reason why a dd in this case is not a great idea.
The belwo command did not work with me
dd if=/dev/sfsapplv of=/dev/tgtapplv bs=512
and
The below command worked but after 3 hours finished but the data is not completed.
whole data in the source mount porint is 80% after finish the copying data came in the target mount porint as 30%
cd /sfsapp
tar -cf - . | (cd /tgtapp ; tar -xf - )
guy's
I'm looking for strong command dd or cp ... etc whatever I want to copy these huage data to the anothere mount porint ..
Pls advice ..
Compare number of files and directories instead of looking at df output.
If things still don't look right you might try installing the rsync rpm.
IBM AIX Toolbox Download Page - By Date
Syntax
rsync -a /sfsapp/ /tgtapp/
You can throw a -P in there which will show progress and a -n will give you a dry run to so you can see what it would look like before a real run. Also a -v if you want verbose output.
Please give me more information about
rsync -a /sfsapp/ /tgtapp/
let me know , what will do it exactly ?
---------- Post updated at 02:58 AM ---------- Previous update was at 02:48 AM ----------
can anyone provide list of all the dd copy command options
This sounds quite suspicious. Could you please post the AIX version you are using (and, if applies, if you are using a 32-bit or 64-bit version?). This sounds like an older version not capable of dealing with files larger than 8G (ustar) or maybe hitting a 2G file size limit.
While you are at it: output of "ulimit -a" might also be interesting.
bakunin
AIX vertion is : AIX 6.1
64-bit
---------- Post updated at 03:06 AM ---------- Previous update was at 02:54 AM ----------
I'm using now this command till now it's running ... let's see
rsync -a /sfsapp and /tgtapp
---------- Post updated at 03:59 AM ---------- Previous update was at 03:06 AM ----------
got failed ..! Pls advice ...
rsync: writefd_unbuffered failed to write 4092 bytes to socket [sender]: Broken pipe (32)
rsync: connection unexpectedly closed (14471473 bytes received so far) [sender]
rsync error: error allocating core memory buffers (code 22) at io.c(600) [sender=3.0.6]