Looking for a script or command to -
Move a very large directory with tens of thousands of files and sub-directories recursively (filenames can include spaces) and replace with symlinks pointing to the new location at the same time so there is no downtime
Looking for speed + safety :o
In other words, there will be activity going on in that directory while hoping to move...
Well unless your OS has special filesystem facilities theres very little chance to achieve what you want, the only thing I can think of is snapshots...
Then in command line or scripting the fact filenames can include spaces makes thing more hazardous, not impossible but that means needs monitoring to see if all is going well, so the task is not that easy if you hope to do it online
Offline the best tool to use in such case is cpio
Good luck
Only read activity - no changes or writes being made to files, or new files / directories being added - it's like an archive
Can you tell us a bit more about why you ned to do this? Is it a space issue or are you moving from using one SAN to another perhaps? That might colour our thinking rather than us guessing and giving (perhaps wildly inaccurate) guesses about what is best.
If it is read-only, then you would be better to copy the data to the new location. Then at an agreed time you can rename the top level directory and create the symbolic link. There will be the time between the rename and the creation of the link that things might not work but that should be a pretty small interval.
Perhaps an option (as root):
cd /original
tar -cvf - . | (cd /target ; tar -xvpf - ) # Will run for quite a while
# When you are ready to switch
mv /original /original.old ; ln -s /target /original # Rename the original and almost immediately create the link
When you are happy, remove /original.old but have a search through to make sure that there are no symbolic links inside it that a find ..... or rm -rf might follow. Take extra care at this point.
If you are moving SAN and assuming you have a volume manager, I would recommend that you attach the new LUNs and add them in to the old volume group. Then you can use the volume manager to firstly mirror the logical volumes then remove the old ones. You could use a volume manager to move the logical volume but if they are large I would always recommend a mirror and delete. It takes a little longer but you have less to worry about if there is an interruption. At worst you might have to drop the partial new copy and start again.
Like I suggested, very different options depending on what you are trying to achieve. Can you also confirm your OS and version with the output from uname -a (pasted in CODE tags) so we use the correct tools.
You also might have to consider SE-Linux, other security packages and Access Control Lists. Do you know if you have any of those?
I hope that this helps or we can continue the discussion.
Robin
Thanks - that was exactly what i was after - especially the tar move, makes it very efficient
Always check the exit status of a cd command!
cd /original &&
tar -cvf - . | (cd /target && tar -xvpf - )
An generic version is
(cd /original && tar -cvf - .) | (cd /target && tar -xvpf - )
where both the sending and the receiving tar work in certain directories in a subshell, while the current directory in the main shell is unchanged.