How to read max of 10 file at a time?

Shell Programming and Scripting
1

Hi All,

Please advise . Welcome more suggestions.

For examples, I have 1000 file with prefix x??? In fact, I want to convert them to x???.txt with max 10 files at a time. As such, I will need to call another script to read from those 10 *txt files and sleep 5000 to convert the next 10 again.

xaaa
;
;
xbbz

Hope to hear from.

Regards,

2

You can do something like that:

count=0
limit=10
ls x??? | \
while read file
do
   mv $file $file.txt
   count=`expr $count+1`
   if [ $count -ge $limit ]
   then
      count=0
      /path/to/another_script &
      sleep 5000  # Wait more than an hour (5000 secs)
   fi
done
3

Aigles, about your solution:
1) It does not work if the number of files is not multiple of 10.
2) The user is asking to work with 10 files at a time.
You are working with one at a time.

Here is one possible solution to rename the files:

for mFile in x???
do
  mv $mFile $mFile".txt"
done

Here is the solution to work with groups of 10 files at a time:

set -- x???.txt
printf "%s %s %s %s %s %s %s %s %s %s\n" $@ | \
while read m10Files
do
  echo "m10Files = "${m10Files}
done
4

1) Exact. The following new version fix it.
2) The script is called every ten files. I add a variable containing the file names (can be used as a parameter for the script)

count=0
file_list=
limit=10
ls x??? | \
while read file
do
   mv $file $file.txt
   count=`expr $count+1`
   file_list="$file_list $file.txt"
   if [ $count -ge $limit ]
   then
      /path/to/another_script $file_list &
      sleep 5000  # Wait more than an hour (5000 secs)
      count=0
      file_list=
   fi
done
[ $count -gt 0 ] && /path/to/another_script $file_list &

I"m not sure that running the script in background is required. If not remove the &

5

Hi aigles and Shell_Life,

Thanks for yours input. I will give a try.
Cheers

6 
count=0
limit=10
mydate=`date '+%Y%m%d%H%M%S'`

cd /var/tmp/VAS

for mFile in x????
do
     mv $mFile $mFile".txt"
     count=`expr $count+1`
     if [ $count -le $limit ]
     then
          sh up.sh 2>LOG/update.sh.$mydate.log &
          sleep 5000
          mv *out LOG/
          count=0
  fi
done

The counter doesn't increase at at and file are mv at all and. It won't stop at limit 10th...

Something wrong... Please advise

7

Your if test in incorrect, replace -le by -ge :

     if [ $count -ge $limit ]
8

hi aigles,

I think it won't help. I noticed mv didn't work at all.

9

The required spaces are missing from the expr.

count=`expr $count + 1`
10

It still make no difference.. :frowning:

11

Execute your script with the debug option (-x) and post the debug trace displayed (until the sleep command).

set -x
count=0
limit=10
mydate=`date '+%Y%m%d%H%M%S'`

cd /var/tmp/VAS

for mFile in x????
do
     mv $mFile $mFile".txt"
     count=`expr $count + 1`
     if [ $count -ge $limit ]
     then
          sh up.sh 2>LOG/update.sh.$mydate.log &
          sleep 5000
          mv *out LOG/
          count=0
     fi
done