Extract line between two patterns

Shell Programming and Scripting
1

Hi All,
I need a script to extract a lines between two patterns.I have done this using grep,cut,tail and head.But its very slow, because my input file contain more than a lakh.


COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:aaaaaaaaaaaaaaa;11111
222222
333333
SEARCHPATTERN
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:bbbbb SEARCHPATTERN bbbbbbbb;11111
222222
333333
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:cccccccccccc;11111
222222
333333
SEARCHPATTERN
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:dddddddddddd;111111
222222
333333
44444444444444
55555555
66666

From the above sample,

Extract single COMMAND and search for SEARCHPATTERN, if SEARCHPATTERN available print the REQUESTSTRING line before ";" .

Note:
The search pattern may present in separate line or in REQUESTSTRING line itself.

Expected output is:

REQUESTSTRING:aaaaaaaaaaaaaaa
REQUESTSTRING:bbbbb SEARCHPATTERN bbbbbbbb
REQUESTSTRING:ccccccccccccccc

My sample code using grep..

grep -n -e "COMMAND" input.txt > tmp.txt
startline=0
while read line
do
	if [ "$startline" = "0" ]
	then
		read line
	fi
	endline=`echo $line |cut -d ":" -f 1`
	endline=`expr $endline - 1`
#	echo "Startline:$startline,Endline:$endline"
	noofline=`expr $endline - $startline`
	head -$endline input.txt > t1.txt
	tail -$noofline t1.txt > t2.txt 
	grep "SEARCHPATTERN " t2.txt > /dev/dull
	if [ "$?" = "0" ] 
	then
		grep "REQUESTSTRING " t2.txt | cut -d ";" -f 1 
	fi
	startline=$endline
done < tmp.txt
2

With assumptions:

awk '/SEARCHPATTERN/ && NF==2{split($2,a,/;/);print FS a[1]}' FS='REQUESTSTRING:' RS= file
3

I believe this does what you want:

awk -F';' '/^COMMAND:/{
        found=1
        if(saved) print saved
}
/^REQUESTSTRING:/{
        if(found) saved = $1
}' input

----------------------------
Oops. I looked too closely at the expected output and missed the part about needing SEARCHPATTERN. The above script happens to work with the sample input, but is not correct in the general case.

What I should have said is something more like:

awk -F';' '/^COMMAND:/{
        if(fc && fs && saved) print saved
        fc = 1
        saved = ""
        fs = 0
}
/^REQUESTSTRING:/{
        if(fc) saved = $1
}
/SEARCHPATTERN/{
        fs = 1
}
END {   if(fc && fs && saved) print saved}' input

but elixir_sinari's proposal is shorter and should work fine as long as the input file doesn't violate the assumptions made. And, given the sample input, the assumptions look perfectly reasonable to me.

1 Like
4

Through Sed..

 sed -n 'H;/^$/{x;/COMMAND.*SEARCHPATTERN/s/.*\(REQUESTSTRING[^;]*\);.*/\1/p}' inputfile
1 Like
5

Thanks a lot.. Don Cragun.. its working good..

---------- Post updated at 01:55 AM ---------- Previous update was at 01:53 AM ----------

Thanks a lot michaelrozar17.
Its very simple and working good..