Running rename command on large files and make it faster

Shell Programming and Scripting
1

Hi All,

I have some 80,000 files in a directory which I need to rename. Below is the command which I am currently running and it seems, it is taking fore ever to run this command. This command seems too slow. Is there any way to speed up the command. I have have GNU Parallel installed on my system but I am not sure how to run this command using GNU Parallel.

What this script is doing is to rename all .xml.dat files to .xml

for f in *.xml.dat; do f2=`basename $f .dat`; echo "$f -> $f2"; mv $f $f2; done

I am using Linux with BASH.

2

Look for "rename" command on your system and its man page.

1 Like
3

yes, first I tried with this:

rename .xml.dat .xml *.xml.dat

but I got the message Argument list too long.

Then I came up with this command of renaming. Though it is working but the only problem is that its too slow.

4

Try:

find DIR -name '*.xml.dat' | xargs rename '.xml.dat$' .xml
1 Like
5 
find DIR -name '*.xml.dat' | xargs rename '.xml.dat$' .xml

The command is not working; neither it is giving any messages (file name extensions are still the same), but I am trying my best to modify it and make it work. I replaced the DIR with the working directory (just to clear any doubts!) :frowning:

If I could make it work, I'll post my command here.

---------- Post updated at 04:18 PM ---------- Previous update was at 04:06 PM ----------

Great. it worked just tweaked your command:

find . -name '*.xml.dat' | xargs rename '.xml.dat' .xml

Thanks for your help. :slight_smile:

6

This is probably what you are looking for:

parallel -j0 mv {} {.} ::: *.xml.dat
parallel -j0 echo {} -\\\> {.}\;mv {} {.} ::: *.xml.dat
1 Like
7

If you can install mmv, you can do mmv '*.xml' '#1.dat' which will be faster than the version with parallel, because it doesn't need to run 80,000 separate instances of mv.

I don't think renaming 80,000 files is ever going to be fast, though. That's 80,000 file tree operations.

1 Like