Extract lines between patterns

Shell Programming and Scripting
1

I have a list in the format below, how do I read through the list and extract the lines between the ##START## and ##END## , so i can check for specific values between each ##START## & ##END## pattern

##START##
RANDOMTEXT
DFGSD
SDFSDF
##END##
##START##
morestuff
sdfggfg
sdfsdf
##END##

EG.

X="morestuff sdfggfg sdfsdf"
2

What tool/command (shell? awk ? sed ?)?

---------- Post updated at 18:26 ---------- Previous update was at 18:04 ----------

shell ( bash ):

while read T
    do  case $T in
        ("##START##")   continue;;
        ("##END##")     echo $X
                        unset X   
                        continue;;
        (*)             X+=$T" ";;
        esac   
   done < file
RANDOMTEXT DFGSD SDFSDF
morestuff sdfggfg sdfsdf
3

Hello squrcles,

Could you please try following and let me know if this helps.

awk '/##START##/{A=1;if(Q){print Q;Q=""};next} /##END##/{A=0} A{Q=Q?Q OFS $0:$0} END{print Q}'  Input_file

Output will be as follows.

RANDOMTEXT  DFGSD  SDFSDF 
morestuff  sdfggfg  sdfsdf

Thanks,
R. Singh

4

And expanding on RavinderSingh13's solution to put it into your variable 'x':-

#!/bin/bash
# test.sh
echo '##START##
RANDOMTEXT
DFGSD
SDFSDF
##END##
##START##
morestuff
sdfggfg
sdfsdf
##END##' > /tmp/text
x=$(awk '/##START##/{A=1;if(Q){print Q;Q=""};next} /##END##/{A=0} A{Q=Q?Q OFS $0:$0} END{print Q}' < /tmp/text)
n=1
while read line
do
	echo "$n = $line"
	n=$(($n+1))
done <<< "$x"

Results on OSX 10.7.5, default bash terminal...

Last login: Fri Jan 15 19:35:11 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> chmod 755 test.sh
AMIGA:barrywalker~/Desktop/Code/Shell> ./test.sh
1 = RANDOMTEXT DFGSD SDFSDF
2 = morestuff sdfggfg sdfsdf
AMIGA:barrywalker~/Desktop/Code/Shell> _
5

I was having some fun with doing this in sed to produce a script to pipe to shell. The idea is that each START / END segment is numbered and the text is piped to a shell function with the segment number (first one being 1).

Might not be worth anything...
Place the following into test.sed

1i\
seg=0
/##START##/b a
b
:a
s/.*$/seg=`expr $seg + 1`/p
n
s/^/echo '/p
:b
n
/##END##/b e
p
b b
:e
s/.*$/' | call_func $seg/p

Then you can get an idea of where this could go by doing:

sed -f test.sed my-input-file.txt
6

Slight variation with a trailing space:

awk '/##END##/{p=0; print s; s=x} p{s=s $0 FS}; /##START##/{p=1}' file

Another sed, testing if the hold buffer is non-empty ...

sed -n '/##START##/{n;h;d;}; /##END##/{g;s/\n/ /g;p;s/.*//;h;d;}; x;/./{x;H;}'

Shorter, but less efficient, using more memory:

sed -n '/##START##/{n;h;d;}; /##END##/{g;s/\n/ /g;p;d;}; H' file
7

Thanks for all you suggestions,
i'm my haste I did not make it clear what my end required result was.

whilst reading through the list I need to capture the text between each individual ##START## and ##END## pair to run a check against the contents.

8

Hi squrcles...

The contents of what?
Another random file?
A sourced set of variables?

Could you give us a much better description along with as much info as possible with real data if possible.

OS.
Shell type and version.
Real data input.
Required data output.
AND now where your contents are to check against!
LBNL where is/are your attempts at solving!

ESP is not my forte hence asking these questions...

9

While incomplete, my sed example outputs a shell script that could be piped into "sh". Just needs the function filled in which receives the block of text and the "segment" number. I was hoping it might put you on the right track without offering a more "complete" solution.

10

Still not clear. Perhaps squrcles want

awk '/##END##/ {p=0} (p==1); /##START##/ {p=1}' file

in order to grep it in a further step.?

11

Didn't he want the strings concatenated in one line?