Help needed - Split large file into smaller files based on pattern match

Shell Programming and Scripting
1

Help needed urgently please.

I have a large file - a few hundred thousand lines.

Sample

CP START ACCOUNT
1234556
name 1
CP END ACCOUNT
CP START ACCOUNT
2224444
name 1
CP END ACCOUNT
CP START ACCOUNT
333344444
name 1
CP END ACCOUNT

I need to split this file each time "CP START ACCOUNT" is matched.

Preferably I would split it every 20 times this is matched and output to smaller files.

I was trying something like the below but could do with help obviously

awk '/START/{x="F"++i;}{print > x;}' inputfile
awk: too many output files 10
 record number 610

Also need to replace START above with CP START ACCOUNT

Can anyone help urgently?

2 
awk '/START/{close(x);x="F"++i;}{print > x;}' inputfile
3 
awk '/START/{close(x);x="F"++i;}{print > x;}' inputfile 
awk: too many output files 10
 record number 610

Hi - thanks for responding quickly

Tried that but same error. It creates output files from 0 to 9 with some output but fails with error above.

Any ideas?

---------- Post updated at 04:40 PM ---------- Previous update was at 04:30 PM ----------

Hi - Found some error - as I was using solaris version of awk - so now using the POSIX awk

This works

$ /usr/xpg4/bin/awk '/CP START/{close(x);x="F"++i;}{print > x;}' inputfile

however this fails:

$ /usr/xpg4/bin/awk '/CP START ACCOUNT/{close(x);x="F"++i;}{print > x;}' inputfile 
/usr/xpg4/bin/awk: line 0 (NR=1): output file "": No such file or directory

How can I get the above to succeed
Also - can I get the split to only split the file after say 20 matches of "CP START ACCOUNT" ?

4 
awk '/START/{close(x);x=("F" ++i)}{print > x;}' inputfile

if on Solaris, using nawk instead of awk

5

Thanks, can you help on the previous comment as well ?

This works

$ /usr/xpg4/bin/awk '/CP START/{close(x);x="F"++i;}{print > x;}' inputfile

however this fails:

$ /usr/xpg4/bin/awk '/CP START ACCOUNT/{close(x);x="F"++i;}{print > x;}' inputfile 
/usr/xpg4/bin/awk: line 0 (NR=1): output file "": No such file or directory

How can I get the above to succeed
Also - can I get the split to only split the file after say 20 matches of "CP START ACCOUNT" ?

6

Another approach is using a BASH script, but this is gonna run slower than awk:

#!/bin/bash

c=0
while read line
do
        if [[ "$line" =~ "^CP START ACCOUNT" ]]
        then
                c=$(( c + 1 ))
                echo "$line" >> F${c}.txt
        else
                echo "$line" >> F${c}.txt
        fi
done < inputfile
7

Give this a try, extending vgersh99's proposal

awk     '/^CP START ACCOUNT/ {if (!(n%20)) {close (fn); fn=("F" ++i)}; n++}
         {print > fn;}
        ' file
8

make sure that the FIRST line of the file starts with CP START ACCOUNT - no blank lines, no nothing else. Otherwise your fn will be an empty string resulting in what you've quoted/observed above.
Or to accommodate and skip any lines BEFORE the first CP START ACCOUNT :

awk     '/^CP START ACCOUNT/ {if (!(n%20)) {close (fn); fn=("F" ++i)}; n++}
         fn {print > fn;}
        ' file