Hi Friends,
I'm trying to create a script that allows me to recursively resize, crop (holding the center of the image) and optimize images jpg, jpeg, png for a specific folder and subfolder with the ability to exclude certain folder and its subdirectory.
Again, I should to do with this script:
1) exclude images under a certain KB size
2) exclude images previous a certain date
3) exclude images in specific folders or sub-folders
4) resize images to precise height (400px for example)
5) crop images (gravity center) transforming into the square image with each side by 400px
6) optimize the image compression ratio
for f in `find . -name "*.jpg"`
do
convert $f -resize x400\> $f.resized.jpg
done
then crop (gravity center) and optimize the result images
for f in `find . -name "*.jpg"`
do
convert $f -gravity center -crop 400x400+0+0 -quality 50% $f.resized.jpg
done
But still I don't know how exclude file for size or folder and eventually how to merge the whole in a single script
I'm not a scripting expert, so I ask you if you can help me.
The trick in programming is to use a step-by-step approach. Start dissecting the problem into smaller sub-problems and these into ever smaller sub-sub-problems until at the lowest level there is no problem any more. You haven't told us anything about your environment, so i assume you can use Korn shell and the standard utilities. So let us start:
First problem: how to treat a single file? You already answered that and we make a function to call for that purpose. Functions are like building blocks and since we will treat any file the same way it should be done by executing the same code over and over. The question is: what does that function need to know - we must provide this as parameter(s) to the function - and what must it return? The usual return code is 0 for success and any other value for various ways of failure.
So the first "script" we write will contain only this function and a call to that function with a single test file. We just want to see if everything goes right before we proceed:
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1" # here lands our parameter, the pathname of a file "/some/where/file"
typeset fTmp="/tmp/${fIn##*/}" # we extract the filename from the pathname: "/some/where/file" -> "/tmp/file"
convert "$fIn" -resize x400\> "$fTmp"
convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg"
rm "$fTmp"
return 0
}
pConvertPicture "/some/example/picture"
exit 0
As you see the function will give back always 0, regardless of the "convert"-program being successful or not. We don't want that but a reasonable return code, telling us if it succeeded and, if not, where it went wrong. Let us fix that before we proceed:
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1" # here lands our parameter, the pathname of a file "/some/where/file"
typeset fTmp="/tmp/${fIn##*/}" # we extract the filename from the pathname: "/some/where/file" -> "/tmp/file"
if ! convert "$fIn" -resize x400\> "$fTmp" ; then
print -u2 - "Error resizing ${fIn}, return code was $?"
return 1
fi
if ! convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg" ; then
print -u2 - "Error cropping ${fIn}, return code was $?"
return 2
fi
rm "$fTmp"
return 0
}
# --------- main program starts here ------
typeset -i iRetVal=0
# here we process our test file:
pConvertPicture "/some/example/picture" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting
exit 1
elif [ $iRetVal -gt 2 ] ; then
print -u2 - "miscellaneous error, exiting
exit 1
fi
exit 0
Notice that we remove the file in /tmp only if everything goes well, otherwise we leave it there because it may help to debug the problem. You won't maybe need all the diagnostic messages in the end but at the start is good to be generous with them - you can always remove them later. In general expect every program to fail at every single possible step - this in itself is not bad. As long as you are able to find out where it failed and why you can do something about it, so build into it everything that will help you analyse where it failed, why and in which way.
Start by trying this script under various conditions: change the file name with which the function is called to some non-existing file and see if the script fails accordingly (it should, otherwise some more logic is needed), try with a non-picture file (the convert utility should fail and the function accordingly too), etc.. Only if you are satisfied with the outcome, proceed:
Now, the next problem: we need to generate a list of files to feed into our function because the single test file business is growing old rapidly. So let us do that - with find , as you did, but in a slightly different way: first, we want full pathnames instead of relative ones, so we need to provide our script a starting directory as parameter. Let us set the problem of how to pass parameters aside for the moment and let us use a single path which we define in the script for now. I suggest that you copy one or two files along with several directories and subdirectories to this path, something like this:
Now we will implement the logic to traverse this tree: Notice that we need to pass only filenames to our function so we need to filter out only the files and only the ones ending in "jpg":
Satisfied? I am! So let us incorporate this into our script:
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1" # here lands our parameter, the pathname of a file "/some/where/file"
typeset fTmp="/tmp/${fIn##*/}" # we extract the filename from the pathname: "/some/where/file" -> "/tmp/file"
if ! convert "$fIn" -resize x400\> "$fTmp" ; then
print -u2 - "Error resizing ${fIn}, return code was $?"
return 1
fi
if ! convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg" ; then
print -u2 - "Error cropping ${fIn}, return code was $?"
return 2
fi
rm "$fTmp"
return 0
}
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir="/path/to/start" # this is fixed, but only for now
# we create a list of files and run these through our function:
for fToProcess in $( find "$fStartDir" -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
Next problem: we want to exclude some directories: again, let us agree on some fixed list, postponing the problem of how to pass it to the script to later. Always one step at a time. Removing certain subdirectories from the search is done like this (modifying the last find-example):
You see this takes away everything in /path/to/start/subdir2 from the result set. Alas, for several subdirectories to exclude this needs a further little twist (it makes little sense to first remove the subdirectory and the parent directory altogether, but it shows the mechanism and is syntactically correct).
Now we build that into our script, here is just the main portion, the rest is unchanged. Notice that we do not know in advance how many subdirectories to exclude, so we create an array for this type of information:
[snip]
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir="/path/to/start" # this is fixed, but only for now
typeset afExcludeDir[1]="/path/to/start/subdir1/subdir1_1" # you may want to put a file there to see if its really excluded
typeset afExcludeDir[2]="/path/to/start/subdir2/subdir2_1" # this should exclude file3.jpg but not file2.jpg
typeset chExclusion=""
typeset -i iCnt=0 # our array counter to loop through it
# First we construct the exclude-list from the array of dirs to exclude if there are
# any array elements at all: first a "\(", then "-path" and the first array element.
# for any further array element we append "-o -path", followed by that element.
# After the last element we append "\)" and "-prune -o".
# If there are no array elements we skip this step altogether.
if [ ${#afExcludeDir[*]} -gt 0 ] ; then # if there are any elements in the array
chExclusion="\\( -path \"${afExcludeDir[1]}\" "
(( iCnt = 2 ))
while [ $iCnt -le ${#afExcludeDir[*]} ] ; do
chExclusion="$chExclusion -o -path \"${afExcludeDir[$iCnt]}\""
(( iCnt += 1 ))
done
chExclusion="$chExclusion \\) -prune -o"
fi
# we create a list of files excluding some directories and run these through our function:
for fToProcess in $( find "$fStartDir" $chExclusion -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
Now, if this works satisfyingly we have a last problem to solve: how to pass parameters to the script in an organised way. Luckily, there is an utiity which does exactly help us with that: getopts . It can decode options that get no arguments and flags that do and still leaves the rest of the commandline so that the script can process it. I suggest you go over the man page of getopts carefully because i am going to skp over some of your requirements once i have shown the process to implement them:
To use getopts we need first to design our interface: which options should the script understand and how should they be formed?
We need:
a path to start with (exactly one such path, so it needs not to be an option but a commandline argument)
none, one or several directories to exclude. These will be options with one argument (a path). Since we are eXcluding we can use an "-x" to introduce each of them. We will also check these directories to see if they exist, are accessible, etc.. Feel free (in fact you are very welcome) to expand on the checks if you find a possible error condition i oversaw.
Now, i don't know how many files you have and how time-consuming the conversion is but probably you want the possibility for a "dry run"? How about
an (optional) option "-S" (simulate) which doesn't really execute the the conversion but only prints the commands that would take place?
Now, here we are: the script will take one commandline parameter ("/path/to/start" in the above example). Again, the function is left out as it isn't changed:
[snip]
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir=""
typeset afExcludeDir[1]="/path/to/start/subdir1/subdir1_1" # you may want to put a file there to see if its really excluded
typeset afExcludeDir[2]="/path/to/start/subdir2/subdir2_1" # this should exclude file3.jpg but not file2.jpg
typeset chExclusion=""
typeset -i iCnt=0 # our array counter to loop through it
if [ -d "$1" ] ; then # if the first commandline argument is a directory
fStartDir="$1" # then make it the starting dir
shift
else # otherwise complain and exit
print -u2 "Error: specify a starting point for the operation."
exit 1
fi
if [ ${#afExcludeDir[*]} -gt 0 ] ; then # if there are any elements in the array
chExclusion="\\( -path \"${afExcludeDir[1]}\" "
(( iCnt = 2 ))
while [ $iCnt -le ${#afExcludeDir[*]} ] ; do
chExclusion="$chExclusion -o -path \"${afExcludeDir[$iCnt]}\""
(( iCnt += 1 ))
done
chExclusion="$chExclusion \\) -prune -o"
fi
for fToProcess in $( find "$fStartDir" $chExclusion -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
Now for the getopts: we need to implement 2 options: -S takes no arguments, "-x" takes one. "-x" can be specified multiple times. getopts will be called as often as there are options to process and every time it is called it cuts off one option (plus argument if any) from the commandline. We start with "-S":
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1"
typeset fTmp="/tmp/${fIn##*/}"
if ! $SIMULATE convert "$fIn" -resize x400\> "$fTmp" ; then
print -u2 - "Error resizing ${fIn}, return code was $?"
return 1
fi
if ! $SIMULATE convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg" ; then
print -u2 - "Error cropping ${fIn}, return code was $?"
return 2
fi
$SIMULATE rm -f "$fTmp"
return 0
}
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir=""
typeset afExcludeDir[1]="/path/to/start/subdir1/subdir1_1" # you may want to put a file there to see if its really excluded
typeset afExcludeDir[2]="/path/to/start/subdir2/subdir2_1" # this should exclude file3.jpg but not file2.jpg
typeset chExclusion=""
typeset -i iCnt=0 # our array counter to loop through it
typeset chOpt="" # our buffer for commandline processing
typeset SIMULATE=""
while getopts ":S" chOpt ; do # commandline
case $chOpt in
S) # simulation flag
SIMULATE=' print - '
;;
"?")
if [ "$chOpt" == "?" -a "$OPTARG" == "?" ] ; then
print -u2 "a help option is not implemented yet"
else
print -u2 "unknown option -${OPTARG}"
fi
exit 1
;;
esac
done
shift $(( OPTIND -1 )) # get everything already processed off the command line
if [ -d "$1" ] ; then # if the (now) first commandline argument is a directory
fStartDir="$1" # then make it the starting dir
shift
else # otherwise complain and exit
print -u2 "Error: specify a starting point for the operation."
exit 1
fi
if [ ${#afExcludeDir[*]} -gt 0 ] ; then # if there are any elements in the array
chExclusion="\\( -path \"${afExcludeDir[1]}\" "
(( iCnt = 2 ))
while [ $iCnt -le ${#afExcludeDir[*]} ] ; do
chExclusion="$chExclusion -o -path \"${afExcludeDir[$iCnt]}\""
(( iCnt += 1 ))
done
chExclusion="$chExclusion \\) -prune -o"
fi
for fToProcess in $( find "$fStartDir" $chExclusion -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
Next, we implement "-x": we need to construct the array afExcludeDir with each new "-x" we encounter. This involves some array arithmentic:
#! /usr/bin/ksh
pConvertPicture ()
{
typeset fIn="$1"
typeset fTmp="/tmp/${fIn##*/}"
if ! $SIMULATE convert "$fIn" -resize x400\> "$fTmp" ; then
print -u2 - "Error resizing ${fIn}, return code was $?"
return 1
fi
if ! $SIMULATE convert "$fTmp" -gravity center -crop 400x400+0+0 -quality 50% "${fIn}.resized.jpg" ; then
print -u2 - "Error cropping ${fIn}, return code was $?"
return 2
fi
$SIMULATE rm -f "$fTmp"
return 0
}
# --------- main program starts here ------
typeset -i iRetVal=0
typeset fToProcess=""
typeset fStartDir=""
# typeset afExcludeDir[]
typeset chExclusion=""
typeset -i iCnt=0 # our array counter to loop through it
typeset chOpt="" # our buffer for commandline processing
typeset SIMULATE=""
while getopts ":Sx:" chOpt ; do # commandline
case $chOpt in
S) # simulation flag
SIMULATE=' print - '
;;
x) # dir to exclude
if [ -d "$OPTARG" ] ; then
typeset afExcludeDir[$((${#afExcludeDir[@]}+1))]="$OPTARG"
else
"print -u2 - "Error: exclusion directory $OPTARG does not exist"
fi
;;
"?")
if [ "$chOpt" == "?" -a "$OPTARG" == "?" ] ; then
print -u2 "a help option is not implemented yet"
else
print -u2 "unknown option -${OPTARG}"
fi
exit 1
;;
esac
done
shift $(( OPTIND -1 )) # get everything already processed off the command line
if [ -d "$1" ] ; then # if the (now) first commandline argument is a directory
fStartDir="$1" # then make it the starting dir
shift
else # otherwise complain and exit
print -u2 "Error: specify a starting point for the operation."
exit 1
fi
if [ ${#afExcludeDir[*]} -gt 0 ] ; then # if there are any elements in the array
chExclusion="\\( -path \"${afExcludeDir[1]}\" "
(( iCnt = 2 ))
while [ $iCnt -le ${#afExcludeDir[*]} ] ; do
chExclusion="$chExclusion -o -path \"${afExcludeDir[$iCnt]}\""
(( iCnt += 1 ))
done
chExclusion="$chExclusion \\) -prune -o"
fi
for fToProcess in $( find "$fStartDir" $chExclusion -type f -name "*jpg" -print ) ; do
print - "processing now: $fToProcess"
pConvertPicture "$fToProcess" ; iRetVal=$?
if [ $iRetVal -eq 1 ] ; then # here we analyse the various return codes of pConvertPicture()
print -u2 - "pConvertPicture() failed to resize, exiting"
exit 1
elif [ $iRetVal -eq 2 ] ; then
print -u2 - "pConvertPicture() failed to crop, exiting"
exit 1
elif [ $iRetVal -gt 2 ] ; then # this is just a "catch-all" if the functions gets extended
print -u2 - "miscellaneous error, exiting"
exit 1
fi
done
exit 0
We are done and you should be able to implement the rest of your requirements easily now. For instance: a minimum size for files - goes to the process function itself, just return if the file is too small. You may also want to skip over already processed files (tip: put a condition into the processing function to look for a file named like the target file but with ".resized.jpg" at the end).
Some finishing touches you should implement, though: an instructive help message (see the program code for pointers where to put it), some more checks if files are really in JPG format (the extension actually says nothing, i could rename any file to "file.jpg") for which i recommend the file utility.
WAW! A real lesson in code development!
I would never have thought of such a complete and authoritative answer!
I'm really very grateful! I'll show your answer to some friends I think they'll be amazed!
Regarding what you suggest, it is believed that the utility "file" could be very useful.
Also find the way to not process files already modified by the script.
Unfortunately I do not know if I will succeed in the enterprise, but I try, but give me "some time" to process everything (I'm still under shok! :-))...
Regarding the environment in use you're absolutely right, I forgot to specify it, I'm in a Debian Jessie (Bourne shell), can the script work?
I'll try!
If I can complete it I would like to publish it (not for personal glory, but only to help other people like me who are looking for something like this. You must to know that my search started for a Joomla plugin that had such features, but I did not find it, and I thought it could help GNU Linux!) on GitLab, what do you think about?