Extract batch based on condition

Shell Programming and Scripting
1

HI,
I have a file as mentioned below. Here one batch is for one user id.Batch starts from |T row and ends at .T row. I want to create a new file by reading this file. The condition is for record 10(position 1-2), if position 3 to position 17 is 0 then delete the entire batch and write into the new file.Here i want to delete entire batch for 12345678 and 4589722 because 10 record is 0 for position 3 to 17. The new file will have only last batch.

|T 20150315
01456200080043000
02453980000000000
03682902950092200
05342901230000000
07980341985040000
09103530000000000
10000000000000000
.T 12345678
|T 20150315
01000000000000000
02000000000000000
03000000000000000
07000000000000000
09000000000000000
10000000000000000
.T 4589722
|T 20150315
01345893000000000
05000000000000000
09000000000000000
10000034578690980
.T 8726345
2

I don't understand what you're trying to do here.

You say that if your "10" record contains enough zeros, you want to delete the entire batch and write into the new file. And, you say that the new file will contain only the last batch.

You don't show any desired sample output, and the requirements above are in direct conflict with each other.

3

Hi Don,
The new will have only below entries.
|T 20150315 01345893000000000 05000000000000000 09000000000000000 10000034578690980 .T 8726345

This is because you can see the row "10" have records > 0 from position 3 to 17. Position 3 to 17 is transaction amount. I want to create a new file which will have only those batches whose transaction amount is >0.

4

If Perl is an option, then here's a program that reads the file line-by-line, stores each line of a record in an array and then either skips the record or prints it as per the rule.

$
$ cat -n f27
     1  |T 20150315
     2  01456200080043000
     3  02453980000000000
     4  03682902950092200
     5  05342901230000000
     6  07980341985040000
     7  09103530000000000
     8  10000000000000000
     9  .T 12345678
    10  |T 20150315
    11  01000000000000000
    12  02000000000000000
    13  03000000000000000
    14  07000000000000000
    15  09000000000000000
    16  10000000000000000
    17  .T 4589722
    18  |T 20150315
    19  01345893000000000
    20  05000000000000000
    21  09000000000000000
    22  10000034578690980
    23  .T 8726345
$
$
$ perl -lne 'if (/^\|T/) {
                 $in = 1;
                 $skip = 0;
                 push @x, $_;
             } elsif ($in and /^10000000000000000/) {
                 $skip = 1;
                 @x = ();
             } elsif ($skip) {
                 next;
             } elsif ($in and not $skip and !/^\.T/) {
                 push @x, $_;
             } elsif (not $skip and /^\.T/) {
                 push @x, $_;
                 foreach $i (@x) {print $i}
                 @x = ();
                 $in = 0;
                 $skip = 0;
             }
            ' f27
|T 20150315
01345893000000000
05000000000000000
09000000000000000
10000034578690980
.T 8726345
$
$

Alternatively, here's a Perl program that reads an entire record (that starts at the "|T" line and ends at ".T" line) as a chunk and then either skips it or prints it as per the rule.

$
$ perl -lne 'BEGIN {undef $/}
             while(/^(\|T.*?\.T \d+)/msg) {
                 $rec = $1;
                 next if $rec =~ m/10000000000000000/;
                 print $rec;
             }
            ' f27
|T 20150315
01345893000000000
05000000000000000
09000000000000000
10000034578690980
.T 8726345
$
$
5

Is this possible to write in a shell script or any unix tool?

6

Yes, I'm sure it is possible.

7

Sure - give it a try.
Any particular reason you don't consider perl being a "unix tool"?

8

Thanks for your reply...but i am not sure if i can put this perl code in my script which is a shell script.

9

I use perl in shell scripting all the time, you should be fine. Are you getting errors?

10

If for some reason, you really need a way to do this only using shell built-ins, you could try something like the following with any POSIX-conforming shell:

#!/bin/ksh
IAm="${0##*/}"
Usage() {
	case "$1" in
	(2)	printf "%s: Can't read input_file: %s\n" "$IAm" "$2";;
	(4)	printf '%s: Invalid date in "|T" line: %s\n' "$IAm" "$2";;
	(5)	printf '%s: Invalid ID in ".T" line: %s\n' "$IAm" "$2";;
	(6)	printf '%s: Invalid tag: %s\n' "$IAM" "$2";;
	(7)	printf '%s: Invalid line length: %s\n' "$IAm" "$2";;
	esac >&2
	printf 'Usage: %s input_file output_file\n' "$IAm" >&2
	exit $1
}

[ $# -ne 2 ] && Usage 1
[ ! -r "$1" ] && Usage 2 "$1"
> "$2" || Usage 3 "$2"
while read -r tag data
do	case ${#tag} in
	(2)	case "$tag" in
		("|T")	[ ${#data} -ne 8 ] && Usage 4 "$tag $data"
			out="$tag $data"
			copy=0;;
		(".T")	[ "$data" = '' ] && Usage 5 "$tag"
			if [ $copy -eq 1 ]
			then	printf '%s\n%s %s\n' "$out" "$tag" "$data" > "$2"
			fi;;
		(*)	Usage 6 "$tag $data";;
		esac;;
	(17)	case "$tag" in
		(10000000000000000)
			;;
		(10*)	copy=1;;
		esac
		out=$(printf '%s\n%s' "$out" "$tag");;
	(*)	Usage 7 "$tag $data";;
	esac
done < "$1"