It looks like ShriniShoo has given you code that will work fine as long as:
- you always want to store the output in files named ABCD_part?.xml (no matter what the input file name is),
- your input file always has a number of records that is a positive integral multiple of the number of output files you want to create,
- you only have one input file to process,
- and you want to read your input files twice.
If you want code that:
- produces output files based on the input file name,
- handles input files with zero or more records,
- can process multiple input files,
- only reads your input files once,
- verifies that each input file has the number of input lines indicated by the TotalRecord tag,
- prints status information for each input file processed, and
- returns a non-zero exit status if one or more of the input files is malformed,
you could try something like:
#!/bin/ksh
awk '
function eofcheck( e, i) {
# Close output files for previous input file.
for(i = 1; i <= nf; i++)
close(of)
# Perform end-of-file error checks...
if(tlp == ntl) return
e = 0
for(i = 1; i <= nf; i++)
if(c > 0) {
printf("\t*** Missing %d+%d records for part %d.\n",
int(c / lpr), (c % lpr) > 0, i)
e = 1
}
if(e) ec = 2
else { printf("\t*** Expected %d trailer line%s; found %d.\n", ntl,
ntl == 1 ? "" : "s", tlp)
ec = 3
}
}
BEGIN { if(lpr == 0) lpr = 15 # lines per record (default 15)
if(nf == 0) nf = 4 # # of output files (default 4)
if(nhl == 0) nhl = 7 # # of header lines (default 7)
if(ntl == 0) ntl = 1 # # of trailer lines (default 1)
ec = 0 # final exit code
}
FNR == 1 {
# If this is not the first input file, perform EOF checks on lsat file.
if(NR > 1) eofcheck()
# Generate output filenames...
for(i = 1; i <= nf; i++)
of = substr(FILENAME, 1, length(FILENAME) - 4) "_part" i \
substr(FILENAME, length(FILENAME) - 3)
# Set temporary value for ftl (it will be recalcuated when we process
# the TotalRecord tag.
ftl = 1
# Clear number of trailer lines printed for current file.
tlp = 0
}
FNR <= nhl || FNR >= ftl {
# Look for input record count.
if(split($0, rc, /<\/*TotalRecord>/) != 3 || rc[2] !~ /^[0-9]+$/) {
# Copy other header lines and the trailer to all output files...
for(i = 1; i <= nf; i++)
print > of
# Count number of trailer lines printed.
if(FNR >= ftl) tlp++
next
}
# We have the header line that defines the number of records present.
irc = rc[2] # input record count
rpf = int(irc / nf) # base output records / file
rem = irc % nf # records left over after even split among files
printf("Found TotalRecord header in %s, %d input records.\n", FILENAME,
irc)
for(i = 1; i <= nf; i++) {
# Calculate # of records for each output file.
c = rpf + (rem >= i)
# Print TotalRecord tag header lines.
printf("%s<TotalRecord>%d</TotalRecord>%s\n", rc[1], c,
rc[3]) > of
printf("\tPreparing to write %d records to %s\n", c, of)
# Convert count for each file from records to lines.
c *= lpr
}
# Calculate First Trailer Line number and initialize output file number.
ftl = nhl + 1 + lpr * irc # line # of 1st trailer line
ofn = 1 # output file number
tlp = 0 # # of trailer lines printed
next
}
ftl == 1 {
# TotalRecord tag not found.
printf("TotalRecord tag not found in %s headers; aborting.\n", FILENAME)
exit 99
}
{ # Copy data lines to appropriate output file.
while(c[ofn]-- <= 0)
if(++ofn > nf) {
printf("Internal error: FNR=%d, ftl=%d, ofn=%d\n",
FNR, ftl, ofn)
exit 98
}
print > of[ofn]
}
END { eofcheck()
exit ec
}' "$@"
As always, if you want to run this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk .
But, of course, this doesn't meet the conflicting requirements you have posted in this thread: You said that the TotalRecord tags are on line 7 in your headers, but your sample header has it on line 6. This code looks for a TotalRecord tag on any line in the headers. It will give you an error if no tag is found. It will produce multiple TotalRecord tags if more than one appears and use data from the last one found. If more than one set of TotalRecord tags appears on a single line, all of the tags on that line will be silently ignored. (Producing an error in these cases is left as an exercise for the reader.)
You said you wanted the same number of records in the first three files and any additional records added to the last file. This code spreads any extra records out such that if there is one extra record, it will go into the first output file; if there are two extra records, one will go into each of the first two output files; and if there are three extra records, one will go into each of the first three output files. (This made error checking simpler in cases where there are fewer records in the input file than there are output files. And, I think it make more sense to do it this way. If you disagree, feel free to modify the code to partition output records the way you want it.)
The awk script is fully parameterized to accept any positive number of header lines, any positive number of trailer lines, any positive number of lines per record, and any positive number of output files/input file (up to your system's awk's limit on the number of open files), but adding getopts code to parse options to this script to override the defaults is left as an exercise for the reader.
If you save the above code in a file named splitter and make it executable ( chmod +x splitter ), you can invoke it as:
./splitter ABCD.xml
to split ABCD.xml into four files named ABCD_part1.xml through ABCD_part4.xml. If you give it additional file operands it will split all of the give files.
This code assumes that it is working on XML files, but doesn't enforce any naming convention. Note, however, that this code assumes that the input file pathames end with a period followed by a three character filename extension (such as .xml or .XML ). If an input pathname contains less than four characters, the results are unspecified. Adding checks for this situation is left as an exercise for the reader.