Anyone have better way to remove ls from this function?

Shell Programming and Scripting
1

all,

i have this script which grabs the number of files to process and displays the percent done as the loop works. i'm never sure how many files will need to be processed.

let k=1
tot=$( ls -l *.${src_ext} | wc -l)
TIMEFORMAT='%3lR'
for i in `ls *.${src_ext} | sort -k2 -t_ -n`
     do
    //lots of stuff that takes a while here,
   // example 1. variable file size could be anywhere from 10mb to 1gb, *.mp4
   // example 2 fairly consistent filesizes +- few percent with a huge list of image files *.tiff or *.dng
SECONDS=0
     // initialize with ffprobe $i, dump results into array
    //  ffmpeg -i $i  $command[0] \
   //  $command[1] \
TIME2FINISH=$(($SECONDS * ($tot - $i )))
percentDone=$(echo "scale=3; $i/$tot*100.0" | bc -l)
echo -e "percentDone $percentDone")
echo -e "Time to finish $(($TIME2FINISH / 3600))hrs $((($TIME2FINISH / 60) % 60))min $(($TIME2FINISH % 60))sec" 
let k=$((k+1))
done

i'm fairly certain using 'ls' is not a good way to do things, can anyone suggest better way?

anyone have any functions that could estimate amount of time remaining based on the wallclock?

for the timing is this a better way to do it

time_1=echo `time ffprobe -i $i command1 command2`
time_2=echo `time ffmpeg -i $i commandA command B`

thanks

2

ls -l is overdone here - you don't need the extra fields in a long listing.

tot=$(echo *.${src_ext} | wc -w)

may save you a few CPU cycles.

Unless you tell people if and how "lots of stuff that takes a while here" depend on file count or sizes or even other parameters, I'm afraid no one will be able to lend you a hand.

1 Like
3

If you happy with Percent complete being accurate to within 1% you can also save spawning bc like replace this:

tot=$(printf "%s\n" *.${src_ext} | wc -l)
for i in `ls *.${src_ext} | sort -k2 -t_ -n`
do

    //lots of stuff that takes a while here

    printf "percentDone %d\n" $((100*++k / tot))
done

or, using an array when can also get rid of wc -l :

F=( *.${src_ext} )
tot=${#F[@]}
for i in "${F[@]}"
do

    //lots of stuff that takes a while here

    printf "percentDone %d\n" $((100*++k / tot))
done
1 Like
4

Thanks for the help.

Ok, if it's an video file *.mp4, the video gets shoved into a long ffmpeg command line to do some heavy duty processing, crop, noise reduction, pixel conversion, etc. I probe the file intially, get some parameters and let it crunch.

5

If those actions are somehow file size dependent, and you supply the accurate dependency algorithm, people might be able to help you develop a formula to calculate a "time percentage" or "time to finish" value.

1 Like