Hi
I need to copy a huge directory with thousands of files onto another directory but without *.WMV files (and without *.wmv - perhaps we need to use *.[wW][mM][vV] ).
Pls advise how can I do that.
Thanks
Use "find":
cd /path/to/sourcedir
find . ! -name "*.[wW][mM][vV]" -exec cp {} /path/to/targetdir \;
Note that you have to change into the directory first, because "find" will find ./file , which can used in the cp -command directly, while with an absolute path you'd get the absolute path /path/to/sourcedir/file which you'd need to modify first.
I hope this helps.
bakunin
1 Like
Should you use a recent bash with extended pattern matching, you could try:
shopt -s extglob
ls !(*.wmv|*.WMV)
2 Likes
Thanks bakunin and RudiC for your replies.
Your solutions are correct as per the requirement i mentioned in my post. My bad, I missed to mention that the directory has sub-directories (and further sub-directories) too which have *.wmv files. The solution you offered working only for non-recursive directories. I tried cp -r with find but it copies all wmv as its parent directory doesn't wmv.
Please suggest a solution for the scenario where I have *.wmv files in sub-directories (any depth) too. Thanks a lot.
Unless there is something else you haven't told us, bakunin's script does exactly what you have requested: find all files in or under a source directory with names not ending in *.wmv or *.WMV and copy them into a single destination directory. (I would have used + instead of \; to terminate the -exec to make it run much faster when copying thousands of files, but that only affects speed; not which files will be copied.)
Oops, I take it back, he missed a key point. His script copied directories as well as regular files. And, you can't use -exec ... + when {} is not the last argument. Try:
cd /path/to/sourcedir
find . -type f ! -name "*.[wW][mM][vV]" -exec cp {} /path/to/targetdir \;
You should also have a look at some of the options available to the cp command in the manual.
You will need to make some decisions on what you want to do with any file links, file/directory permissions/ACLs, date/time stamps owners and groups for directories and files in the tree.
gnutar has quite a sensible set of defaults for all this stuff and the following has a good chance of doing what you want:
cd /path/to/sourcedir
tar -c -f - --ignore-case --exclude=*.wmv . | (cd /path/to/targetdir ; tar xf - )
1 Like
True - my bad. My only excuse is one shouldn't answer threads before the second coffee.
cd /path/to/sourcedir
find . -type f ! -name "*.[wW][mM][vV]" -exec cp {} /path/to/targetdir \;
This will probably not work either, because now the directory hierarchy will not become copied and some files (the one in sub-subdirectories) will have no proper targets. I suggest to do it in a two-pass way:
cd /path/to/sourcedir
find . -type d -exec mkdir -p /path/to/targetdir{} \;
find . -type f ! -name "*.[wW][mM][vV]" -exec cp {} /path/to/targetdir +
The first pass creates a copy of the directory structure, the second copies all the files. If you need to copy filemodes, ownerships, etc. too, you need to modify the "cp"-command accordingly: "cp -p ...".
@Chubler_XL: using "tar" for that was my first impulse, but it would limit the solution to a system where the GNU-tar is available.
I hope this helps.
bakunin
We both needed something before we replied to this thread. I noticed that you hadn't included -type f and then posted a fix that had its own problems. And fixed it, and fixed it again, and fixed it again in a period of about 10 minutes. And, unfortunately, I can't use lack of coffee as an excuse. (I have a caffeine allergy, so I never drink coffee.)
And, I don't think:
cd /path/to/sourcedir
find . -type d -exec mkdir -p /path/to/targetdir{} \;
find . -type f ! -name "*.[wW][mM][vV]" -exec cp {} /path/to/targetdir +
does what you were thinking either. The first find does copy the directory hierarchy under sourcedir to targetdir , but the second find still copies all of the non-.wmv files to the targetdir directory; not to their corresponding places in the newly created hierarchy. And, find -exec command + can only be used when there is no more than one {} in the command, and, if there is one, it has to be the last operand in that command.
The OP's desired destination for the copied files seems ambiguous to me. I don't know if the intent is to duplicate the file hierarchy or if the intent is to flatten the file hierarchy into the single directory named as targetdir .
If the intention is to flatten the hierarchy, I still believe:
cd /path/to/sourcedir
find . -type f ! -name "*.[wW][mM][vV]" -exec cp {} /path/to/targetdir \;
is sufficient.
The the intention is to preserve the hierarchy, I believe we would need something like:
cd /path/to/sourcedir
find . -type d -exec mkdir -p /path/to/targetdir{} \;
find . -type f ! -name "*.[wW][mM][vV]" -exec cp {} /path/to/targetdir{} \;
I believe this could be combined into a single find command to avoid walking the tree twice, but with the thousands of invocations of cp the additional complexity might overshadow any performance gain:
cd /path/to/sourcedir
find . \( -type d -exec mkdir -p /path/to/targetdir{} \; \) -o \
\( -type f ! -name '*.[wW][mM][vV]' -exec cp {} /path/to/targetdir{} \; \)
Are we having fun yet???
Like i can't in this case. Somehow this thread is bringing up the worst in us both. You are right in your assessment
Alas, this is not the case either, because "{}" can only be used once in a command.
Yes, we do. Here is my next suggestion. By now i am quite sure it will not do what i want, neither do what the thread-o/p wants and most probably be forbidden by law.
cd /path/to/sourcedir
find . -type d -exec mkdir -p /path/to/targetdir{} \;
find . -type f ! -name "*.[Ww][Mm][Vv]" -exec FNAME={} ; cp "$FNAME" "/path/to/targetdir$FNAME" \;
Aren't we a happy lot?
bakunin
On systems without GNU-tar I'd be tempted to use cpio.
cd /path/to/sourcedir
find . -depth ! -name "*.[Ww][Mm][Vv]" | cpio -pdm /path/to/target
3 Likes
Why not? It appears to work.
My man on find says
From that I would take it that supporting multipe {} could be a GNU extension.
1 Like
With -exec ... + , {} is only required to be replaced by the current pathname if it is the last argument before the + and only if it is a separate argument.
With -exec ... \; , {} is required to be replaced by the current pathname every time it appears as a separate argument. As an extension to the standards, many implementations replace {} with the current pathname no matter where it occurs in an argument (beginning, middle, end of an argument as well as when it is a stand-alone argument). The standards say the behavior is unspecified if {} appears anywhere other than as a stand-alone argument.
On OS/X, the command:
find . -type f ! -name "*.[wW][mM][vV]" -exec cp {} /path/to/targetdir/{} \;
works, but (obviously) it isn't portable.
True! find ... | cpio -pd ... would perhaps be the most portable and elegant solution to this. What a shame i haven't even thought of that. Thanks for reminding me that there are alternatives to tar .
bakunin
portable and elegant assuming people have cpio, that is. It's not actually a standard anymore.
Thanks all and there're too many solutions.
Sorry for the confusion, Let me clarify again -
- I need to copy a directory to another location
- Must keep the same hierarchy and structure etc.
- All files and directories must be copied incl. hidden files EXCEPT for *.WMV and *.wmv
After posting here, I tried tar chpvf - . -X /tmp/exclude_list | (cd /dest; tar xvf - ) and it worked but needed more tests. I added both *.WMV and *.wmv to /tmp/exclude_list
Any better and more accurate solution is appreciated.
Looks fine - One small thing is you have v (verbose) option on both ends. This will result in jumbled output as both tar commands will be writing to stderr at the same time.
I usually keep v on the extract side only.
Neither cpio nor tar are in the current standards. The standard does include pax . It could be used as follows:
(cd source_dir;pax -w .) | (cd destination_dir;pax -rvc '*.wmv' '*.WMV')
The v causes pax to list the files as they are being created in the destination directory; drop that character from the options if you want it to copy the hierarchy silently. The pax -c option says to copy all files found in the archive except those with filenames matching the patterns given on the command line (in this case, don't copy files ending in .wmv nor .WMV ).