Faster way: SAN hd to SAN hd copying

hi! i got a rhel 6.3 host that already have an xfs filesystem mounted from a SAN (let's call it SAN-1) whose size is 9TB.

i will be receiving another SAN (let's call it SAN-2) storage of 15TB size. this new addition is physically on another SAN storage. SAN-1 is on a Pillar storage while the new one is on a Dell SAN storage.

i supposed brand does not matter here. my initial reaction to move the directories and files from SAN-1 to SAN-2 is either using "tar" or "dd". in both cases, i need to do it online as there is continuous data being duplicated from SAN-1 to tape storage.

is using "tar" or "dd" the only online option? are there better ways?

Why not cp (if you're trying to make copies of the files on the old SAN so they also appear on the new SAN) or mv (if you're trying to move files from one SAN to the other SAN)?

coz i want to keep the links, permissions, owners, etc.

old school way. so i was wondering if nowadays there are better ways specially if a SAN storage is involved (as opposed to physical hd to hd).

Don't you think it would have been a good idea to tell us
Note that cp -Rp source destination can be used to copy filesystems maintaining dates, permissions, and ownership (but may lose links) as long as you're running with the appropriate privileges.

You can use cpio , pax , or tar to maintain links, owners, and permissions as long as you're running with the appropriate privileges.

If you use dd to copy individual files, you will lose links, owners, and permissions.

If you use dd to copy an entire filesystem, you can't change the filesystem type or size with the copy and you will destroy any files that were on the SAN you are overwriting with the dd . And, the filesystem cannot be active (i.e., should not be mounted (or, if it is mounted, mount it read-only)) during the dd . After you copy the filesystem, depending on the filesystem type, you may be able to grow the filesystem in the new SAN to occupy the additional space available there depending on the tools available on the new SAN to do such things. Note that some SAN vendors use proprietary filesystem and/or RAID designs that might not work on another SAN vendor's hardware. Be very sure that you know what you're doing before trying to use dd to replace one SAN filesystem with another SAN filesystem.

I don't know if the cpio command is available on Linux but that is what I use in the Solaris world... AFAIK the faster'ish way to copy whole file systems. Read a bit here:

Copying Directories Between File Systems (cpio Command) - Oracle Solaris Administration: Devices and File Systems

When you say 'a faster way', are you trying to move the data from one SAN to another, so that you can detach the original and you need to make the offline time as short as possible?

If your disk is LVM, then you might be better to proceed like this:-

  • Get the new LUN assigned and re-scan to pick it up if necessary
  • Make the whole new LUN into an LVM Physical Volume with pvcreate
  • Add the PV into the volume group containing the data you want to move
  • Alter the logical volume holding your filesystem/data to have two copies, specifying the new PV as the target. This will take a long time, but the applications remain available because the data is still accessible.
  • Verify that the logical volume has no errors reported.
  • Alter the logical volume to have just a single copy, specifying the old SAN device to be removed
  • Repeat the mirror/drop for as many logical volumes as you need to move to completely free the old PV
  • Remove the old PV from the volume group
  • Detach the old LUN from the OS
  • Detach the old LUN from the SAN side

Using the tools in the OS rather than trying to do the work yourself should be a lot neater. The logical volume is moved, but the filesystem has no idea about it, so all the files remain exactly as they were and your applications, backups or whatever can happily carry on.

I prefer to add a mirror to the new device rather than just execute a move for large logical volumes just in case there are any issues in the process. You don't destroy the original so you can always drop the new copy and start again.

Have I missed the point and waffled on about something irrelevant? My apologies if I have, but I hope that this helps. If you are looking to make another copy to be used separately, then we might be able to use LVM to do that too, but that would be a copy at a fixed point in time that would then go out of date with respect to the original as either is changed (you suggest that there are many concurrent processes working on it)

Anyway, I hope that this helps. Please feel free to correct me if I'm way off target.

Kind regards,
Robin

There's nothing particularly slow about tar, either.

In the end it's 15 tb of data, transferring it will take time.