Delete lines above and below specific line of text

I'm trying to remove a specific number of lines, above and below a specific line of text, highlighted in red:

<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151205000001[-3:GMT]
<TRNAMT>10
<FITID>667800001
<CHECKNUM>667800001
<MEMO>BALANCE
</STMTTRN>
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151207000001[-3:GMT]
<TRNAMT>-10
<FITID>667800002
<CHECKNUM>667800002
<MEMO>PAY
</STMTTRN>

That's 6 lines above and 9 below. The content of the file extends beyond those lines, so I have to specify.

I know the sed command can do this but I don't know how to proceed in this case.

Save as filter.pl
Run as perl filter.pl bomsom.file

Substitute the highlighted for any regex you want.

#!/usr/bin/env perl
use strict;
use warnings;

my @buffer;

while( <> )
{
    push @buffer, $_;
    if( @buffer == 16 )
    {
        if( $buffer[6] =~ /<MEMO>BALANCE/ )
        {
            @buffer = ();
            next;
        }
        print shift @buffer;
    }
}
print @buffer if @buffer;

Sometimes, good old ed is simpler:

#!/bin/ksh
ed -s file <<-"EOF"
	g/<MEMO>BALANCE/.-6,.+9d
	1,$p
EOF

Although written and tested using the Korn shell, this will work with any shell that uses basic Bourne shell syntax. It will also work with ex instead of ed .

In case you want a throw away command line:

perl -ne 'push @b, $_; (@b==16) and $b[6]=~/<MEMO>BALANCE/?@b=():print shift @b; END{print @b if @b}' bomsom.file

However, the full script is more flexible since you could add to the work it can do; for example if you were to combine the requirement of this thread and the requirement of your other thread

cat bomsom.file
STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151127000001[-3:GMT]
<TRNAMT>-8.77
<FITID>667800007
<CHECKNUM>667800007
<MEMO>PAYPAL
</STMTTRN>

<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151130000001[-3:GMT]
<TRNAMT>6.00
<FITID>667800008
<CHECKNUM>667800008
<MEMO>ENAPARK
</STMTTRN>
<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151205000001[-3:GMT]
<TRNAMT>10
<FITID>667800001
<CHECKNUM>667800001
<MEMO>BALANCE
</STMTTRN>
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151207000001[-3:GMT]
<TRNAMT>-10
<FITID>667800002
<CHECKNUM>667800002
<MEMO>PAY
</STMTTRN>
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151127000001[-3:GMT]
<TRNAMT>-8.77
<FITID>667800007
<CHECKNUM>667800007
<MEMO>PAYPAL
</STMTTRN>

<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151130000001[-3:GMT]
<TRNAMT>6.00
<FITID>667800008
<CHECKNUM>667800008
<MEMO>ENAPARK
</STMTTRN>

Highlighted in blue is the modification.

cat filter.pl
#!/usr/bin/env perl
use strict;
use warnings;

my @buffer;

while( <> )
{
    push @buffer, $_;
    if( @buffer == 16 )
    {
        if( $buffer[6] =~ /<MEMO>BALANCE/)
        {
            @buffer = ();
            next;
        }
        print reverse_sign( shift @buffer );
    }
}
print reverse_sign( @buffer ) if @buffer;

sub reverse_sign
{
    my $lines = \@_;
    for my $line ( @{$lines} ){
        $line =~ s/^(<TRNAMT>)(-?\d+\.\d+)$/sprintf "%s%.2f", $1, -$2/e;
    }
    return @{$lines};
}
perl filter.pl bomsom.file
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151127000001[-3:GMT]
<TRNAMT>8.77
<FITID>667800007
<CHECKNUM>667800007
<MEMO>PAYPAL
</STMTTRN>

<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151130000001[-3:GMT]
<TRNAMT>-6.00
<FITID>667800008
<CHECKNUM>667800008
<MEMO>ENAPARK
</STMTTRN>
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151127000001[-3:GMT]
<TRNAMT>8.77
<FITID>667800007
<CHECKNUM>667800007
<MEMO>PAYPAL
</STMTTRN>

<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151130000001[-3:GMT]
<TRNAMT>-6.00
<FITID>667800008
<CHECKNUM>667800008
<MEMO>ENAPARK
</STMTTRN>

Thanks a lot guys.

Aia, I ran the Perl .pl file in Terminal and it works very well combining both requirements in one script like you said. I changed the file name of course, like this:

perl filter.pl cc3.ofx

However, when I added the variable to write the changes in the file (-i) the new file is missing the last 15 lines, and just those exact lines are printed in the terminal window. I ran the command like this:

perl -i.bak filter.pl cc3.ofx 

What is wrong?

Simple awk ? Try

awk '
NR == 1         {Above++
                 Below++
                }

                {RB[NR%Above] = $0
                }

$0 ~ Pat        {Above = NR + Below
                 B = 1
                }

NR >= Above     {print B?$0:RB[(NR+1)%Above]
                }

' Above=3 Below=5 Pat="BALANCE" file
 1 <STMTTRN>
 2 <TRNTYPE>CREDIT
 3 <DTPOSTED>20151205000001[-3:GMT]
13 <FITID>667800002
14 <CHECKNUM>667800002
15 <MEMO>PAY
16 </STMTTRN>

The flag -i works only with the <> operator. Mostly, in throw away command line executions. I am using the <> in the script trying to be flexible from where it is getting the input (from stdin, and from file given). However, it is intended to show at stdout and thus it closes $ARGV (where <> is reading) and then it finishes the job.

If you are going to use it as such, a modification must be done. The highlighted part represents the modification. It has been tested, minimally. Please, try and let me know if you need something else.

#!/usr/bin/env perl
use strict;
use warnings;

my @buffer;

while(<>) {
    push @buffer, $_;
    if(@buffer == 16) {
        if($buffer[6] =~ /<MEMO>BALANCE/) {
            @buffer = ();
            next;
        }
        print reverse_sign(shift @buffer);
    }
    if(eof) {
        print reverse_sign(@buffer) if @buffer;
    }
}

sub reverse_sign {
    my $lines = \@_;
    for my $line (@{$lines}) {
        $line =~ s/^(<TRNAMT>)(-?\d+\.\d+)$/sprintf "%s%.2f", $1, -$2/e;
    }
    return @{$lines};
}
1 Like

If you'd add two lines to the ed script I suggested in post #3 in this thread, it will also handle the changes requested in both threads:

#!/bin/ksh
ed -s bomsom.file <<-"EOF"
	g/<MEMO>BALANCE/.-6,.+9d
	g/<TRNAMT>/s//&-/
	g/\(<TRNAMT>\)--/s//\1/
	1,$p
EOF

and, using bomsom.file from post #4 in this thread, it produces the same output as Aia's perl script also shown in post #4.

Aia, that's awesome, it works beautifully. Thanks a lot.

@Don: Thanks Don, I'm yet to make it work but I'm sure doing something wrong. I'm still in the very very basics, try and error (a lot).