extract multiple sections of file

rgentis · March 18, 2008, 7:09pm

I have a file that I need to parse multiple sections from the file.

The file contains multiple lines that start with ST (Abunch of data)
Then the file contains multiple lines that start with SE (Abunch of data)

SE*300001
ST*8100002

I need all of the lines between and including these.
They are invoices.
The invoice starts with the ST line and ends with the SE line.

I need to break out all of the invoices into separate files.

Can someone please help me. I know Grep, sed, or AWK can do this, but not sure how.
Thank you

Here is an example:
ST*8100001
BIG*20080315*1220680417**SUPPLYDI
N1*SFMCLANE HIGH PLAINS*9246120004
N1*STSWC 7-11 #57134*91571315
N32712 E 8TH ST
N4*ODESSA*TX79761
REF*ST000134
ITD*05*3*****7*****NET 7
IT1**1*CA*20.09**CB*649251*PI*093*UP*099299711018*RANA
TXI*ZZ*1.532
CTP**RES*0***CSR1
PID*F7-11 T-SHIRT BAG 1/7 BBL
PO41000
IT1**1*EA*33.72**CB*834861*PI*093*UP*012253022401*RANA
TXI*ZZ*2.572
CTP**RES*0***CSR1
PID*FKIT CONCRETE CHAMP
PO41
IT1**1*EA*0.03**CB*192849*PI*093*UP*000000192842*RANA
CTP**RES*0***CSR1
PID*FSCS 711 BK 200
PO41
IT1**30*EA*2.59**CB*001511*PI*093*UP*025215102776*RANA
CTP**RES*0***CSR1
PID*FMAXELL T-160 PLUS VIDEO
PO41
TDS18454
SAC*C*G740***5300*******06SERVICE
CTT4
SE*30*0001

shamrock · March 18, 2008, 7:20pm

awk '/^ST/,/^SE/' file

rgentis · March 18, 2008, 7:48pm

Thank you for your prompt response.

It did what I wanted. However the three sections need to be parsed to to different files.

So you have
ST
data
SE
This should be taken to file 1
ST
data
SE
This should be taken to file 2

ETC.....

Also I noticed that the ST and SE are numbered.

ST*810*0004
Then
SE*(Number)*0004
Thank you

summer_cherry · March 18, 2008, 10:48pm

nawk 'BEGIN{n=1}
$0 ~ /^ST/ {f=1}
$0 ~ /^SE/ {invoice[n]=sprintf("%s\n%s",invoice[n],$0);f=0;n=n+1}
{
if (f==1)
invoice[n]=sprintf("%s\n%s",invoice[n],$0)
}
END{
for (i in invoice)
print invoice [i]>> i
close(i)
}' filename

drl · March 18, 2008, 11:29pm

Hi.

An alternate awk solution:

#!/usr/bin/env sh

# @(#) s1       Demonstrate extraction of range to separate files.

#  ____
# /
# |   Infrastructure BEGIN

echo
set -o nounset

debug=":"
debug="echo"

## The shebang using "env" line is designed for portability. For
#  higher security, use:
#
#  #!/bin/sh -

## Use local command version for the commands in this demonstration.

set +o nounset
echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>&1 && version =o $(_eat $0 $1) awk my-nl
set -o nounset

# Use nawk or /usr/xpg4/bin/awk on Solaris.

echo

FILE=${1-data1}
echo " Input file $FILE:"
cat $FILE

# |   Infrastructure END
# \
#  ---

echo
echo " Results from processing:"
awk '
BEGIN   { i = 0 }
/ST/            { i++ ; name = "file" i }
/ST/,/SE/       { print > name }
' $FILE

my-nl file?

exit 0

Producing:

% ./s1

(Versions displayed with local utility "version")
Linux 2.6.11-x1
GNU bash, version 2.05b.0(1)-release (i386-pc-linux-gnu)
GNU Awk 3.1.4
my-nl (local) 296

 Input file data1:
ST
first invoice
SE
ST
second invoice
SE
ST
third invoice
SE

 Results from processing:

==> file1 <==

  1 ST
  2 first invoice
  3 SE

==> file2 <==

  1 ST
  2 second invoice
  3 SE

==> file3 <==

  1 ST
  2 third invoice
  3 SE

Choose the base file name you wish in variable "name" ... cheers, drl

faltooweb · March 19, 2008, 12:47am

#-- Use ST values as output filename.
awk -v out="/dev/null" '
/^ST/ {gsub("\\*","-",$0); out=$0".txt"}
/^SE/ { close(out) }
{ printf "%s\n",$0 >> out }
' $INFILE

Output will be
ST-810-0001.txt
so on ...

-Ramesh

rgentis · March 24, 2008, 11:51pm

I wanted to thank all of you for your response.

One issue, I am porting the awk utility to windows. So I do not think all of the functionality is there.
For instance when I used Ramesh's example, I received numerous errors.
Here is the code:

c:\tools\gnuwin32\bin\awk -v '/^ST/ {gsub("\\*","-",$0); out=$0".txt"}
/^SE/ { close(out) }
{ printf "%s\n",$0 >> out }
' %input%edifile.dat

Here is the result:
awk: `/ST/' argument to `-v' not in `var=value' form

Usage: awk [POSIX or GNU style options] -f progfile [--] file .
Usage: awk [POSIX or GNU style options] [--] 'program' file ...
POSIX options: GNU long options:
-f progfile --file=progfile
-F fs --field-separator=fs
-v var=val --assign=var=val
-m[fr] val
-W compat --compat
-W copyleft --copyleft
-W copyright --copyright
-W dump-variables[=file] --dump-variables[=file]
-W exec=file --exec=file
-W gen-po --gen-po
-W help --help
-W lint[=fatal] --lint[=fatal]
-W lint-old --lint-old
-W non-decimal-data --non-decimal-data
-W profile[=file] --profile[=file]
-W posix --posix
-W re-interval --re-interval
-W source=program-text --source=program-text
-W traditional --traditional
-W usage --usage
-W use-lc-numeric --use-lc-numeric
-W version --version

To report bugs, see node `Bugs' in `gawk.info', which is
section `Reporting Problems and Bugs' in the printed version.

gawk is a pattern scanning and processing language.
By default it reads standard input and writes standard output.

Examples:
gawk '{ sum += $1 }; END { print sum }' file
gawk -F: '{ print $1 }' /etc/passwd
'/SE/' is not recognized as an internal or external command,
operable program or batch file.
'{' is not recognized as an internal or external command,
operable program or batch file.
''' is not recognized as an internal or external command,
operable program or batch file.
C:\tools>edi
awk: `'/ST/' argument to `-v' not in `var=value' form

Usage: awk [POSIX or GNU style options] -f progfile [--] file .
Usage: awk [POSIX or GNU style options] [--] 'program' file ...
POSIX options: GNU long options:
-f progfile --file=progfile
-F fs --field-separator=fs
-v var=val --assign=var=val
-m[fr] val
-W compat --compat
-W copyleft --copyleft
-W copyright --copyright
-W dump-variables[=file] --dump-variables[=file]
-W exec=file --exec=file
-W gen-po --gen-po
-W help --help
-W lint[=fatal] --lint[=fatal]
-W lint-old --lint-old
-W non-decimal-data --non-decimal-data
-W profile[=file] --profile[=file]
-W posix --posix
-W re-interval --re-interval
-W source=program-text --source=program-text
-W traditional --traditional
-W usage --usage
-W use-lc-numeric --use-lc-numeric
-W version --version

To report bugs, see node `Bugs' in `gawk.info', which is
section `Reporting Problems and Bugs' in the printed version.

gawk is a pattern scanning and processing language.
By default it reads standard input and writes standard output.

Examples:
gawk '{ sum += $1 }; END { print sum }' file
gawk -F: '{ print $1 }' /etc/passwd
'/SE/' is not recognized as an internal or external command,
operable program or batch file.
'{' is not recognized as an internal or external command,
operable program or batch file.
''' is not recognized as an internal or external command,
operable program or batch file.

Thank you again for your help.