Ordering batch number

Hi,

Could some one please help to order the batch number in sequence.

I will be getting bunch of files with batch number in folder1 which are not in sequence.

I need to move all files from folder1 to folder2 with batch number in sequence.

Header record looks like

 PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000013 

example.

/tmp/folder1
1
2
4
3
5
6
8
9
7
10
/tmp/folder2
1
2
3
4
5
6
7
8
9
10

I need to run this for every few hours.

The following does what you want, if you need to compare the DATE field as well it will obviously require modification.

#!/usr/bin/perl

use strict;
use warnings;
use File::Copy;
my %batch;
opendir(my $src, "/tmp/folder1")|| die "Could not open /tmp/folder1: $!\n";
while (my $f=readdir($src)){
    if ($f !~/^\.+/){
          open (my $transaction , "<", "/tmp/folder1/$f")|| die "Could not open file \"/tmp/folder1/$f: $!\n";
          my $head=readline($transaction);
          close($transaction);
          $batch{$f}=$1 if ($head=~/BATCHNO=(\d+)/);
    }
}
my $index=0;
for my $file (sort {$batch{$a}<=>$batch{$b}} keys %batch){
    copy("/tmp/folder1/$file", "/tmp/folder2/$index");
    $index++;
}

Thanks for the perl script. But I am looking for shell programming.

You could start with

grep BATCHNO /tmp/folder1/* | sort -t"=" -nk 4

and then pick the filename in this ordered list to move to folder2.

Was in a Perl frame of mind, something straightforward like this?

 for j in $(for i in /tmp//folder1/* ; do echo $i,$(head -1 $i| sed "s/.*BATCHNO=//");done | sort -k2 -t, | cut -d, -f1) ; do cp $j /tmp/folder2/$k; k=$(( $k + 1 )); done

Thanks both. Above command correctly searching for BATCHNO and sorting in order, but when coping the files to folder2 the files are not copied on sequence. When writting to folder2 we should get following result

/tmp/folder2 $ ls -tr| more 
1
2
3
4
5
6
7
8
9
10

This is what i came with. If i correctly understood your question.

You have a file like :
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000014
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000009
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000017

And you need to order it by the BATCHNO as in :
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000009
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000014
PROCESS1 DATE=130813 TIME=130503 BATCHNO=0000000017

Hope this will help you. Feel free to modify the script to put the file in another path.

#!/bin/bash
#Date :
#Author
#Purpose : Sort batch file
#Usage : ./sortbatch.sh <batchfile>

BATCHFILE=$1
ORDEREDBATCH=`pwd`/ordered_`date +%F_%H:%M`_$BATCHFILE

#Verify if a file was given
if [ $# -lt 1 ]; then
        echo "Usage : ./sortbatch.sh <batchfile>"
        exit 1
fi

#Verify if the file exists
if [ ! -e $BATCHFILE ]; then
        echo "File does not exist"
        exit 1
fi

#Read batch numbers
BATCHNRS=`awk '{print $4}' $BATCHFILE | sort | uniq`

#Copy to another file the lines in order of the batch number
for i in $BATCHNRS; do
        awk -v batch="$i" '{if ($4 == batch ) print $0}' $BATCHFILE >> $ORDEREDBATCH
done

#End

Is ls -tr listing them in numerical/alphabetical sequence?
Is their creation time identical and so sequence is the default sort order?
If you insert a sleep 1 after the cp command does the time order sort work as expected?

Try

awk -F= 'FNR==1 {fn="/tmp/folder2/"$NF} {print > fn}' /tmp/folder1/*