retrieving data between two strings

NareshN · July 20, 2012, 6:34am

I have input file like

AAA
AAA
CCC
CCC
CCC
EEE
EEE
EEE
EEE
FFF
FFF
GGG
GGG

i was trying to retrieve data between two strings using sed.

sed -n /CCC/,/FFF/p input_file

Am getting output like

CCC
CCC
CCC
EEE
EEE
EEE
EEE
FFF

The above command retrieves the data till to the first occurence of the destination string (i.e., till 1st occurence off FFF record)
I need to search the records till last occurence.
Expected output is

CCC
CCC
CCC
EEE
EEE
EEE
EEE
FFF
FFF

Plz help on this query.

yogendra.barode · July 20, 2012, 6:52am

You can try following command

sed -n '/CCC/,/GGG/p' FILE_NAME |sed '$d'

NareshN · July 20, 2012, 6:56am

The problem is i dont know the next record to delete in the file (may be GGG or ZZZ). In that case how can we use destination string.

PikK45 · July 20, 2012, 6:57am

@Yogi: what would you do.. If the next string after the last FFF is HHH and the next time it changes to MMM and it keep changing?

Scrutinizer · July 20, 2012, 7:23am

Try:

awk '/FFF/ || /CCC/,/FFF/' infile

elixir_sinari · July 20, 2012, 7:41am

cat test
AAA
AAA
CCC
CCC
CCC
EEE
EEE
EEE
EEE
FFF
FFF
GGG
GGG
dfffffffff
dfdf
FFF
dfdf
dfdf
dfd

awk '/FFF/ || /CCC/,/FFF/' test
CCC
CCC
CCC
EEE
EEE
EEE
EEE
FFF
FFF
FFF

Missed records:

GGG
GGG
dfffffffff
dfdf

Scrutinizer · July 20, 2012, 7:48am

Good point, but the OP's requirements appeared to be limited to consecutive patterns...

NareshN · July 20, 2012, 8:14am

Scrutinizer,
Could you plz explain the command. I didn't understand.
Is it will work for huge data file.
Is it possible, by using grep

---------- Post updated at 05:44 PM ---------- Previous update was at 05:29 PM ----------

awk command is working file.
Thanks scrutinizer...

I would like to pass SourceString DestString at runtime.
How can i do this in while using awk.
Here $1 and $2 treats like columns in awk.

example:Executing script
filename CCC FFF

NareshN · July 22, 2012, 10:58pm

Could you plz help how to pass the parameters to awk

Scrutinizer · July 23, 2012, 3:07am

Try:

awk '$0~s || $0~r,$0~s' r="CCC" s="FFF" infile

NareshN · July 24, 2012, 5:23am

Scrutinizer,

It's getting some error if i execute the above command.

awk: syntax error near line 1
awk: bailing out near line 1

Scrutinizer · July 24, 2012, 6:28am

If you are using Solaris, use /usr/xpg4/bin/awk instead of awk

Irishboy24 · July 29, 2012, 1:04pm

Another way to do it is to get the line numbers where the pattern occur and then print the content between the lines:

$ cat linepat
AAA
AAA
CCC
CCC
CCC
EEE
EEE
EEE
EEE
FFF
FFF
GGG
GGG

using awk to get the line number for the first occurence of CCC

$ awk '/CCC/{print NR; exit}' linepat
3

using grep to get the line number for the last occurrence of FFF

grep -n "FFF" linepat | tail -1 | awk -F: '{print $1}'
11

Finally, using awk to read between the lines :

$ awk '(NR>=3) && (NR<=11)' linepat
CCC
CCC
CCC
EEE
EEE
EEE
EEE
FFF
FFF

NareshN · September 6, 2012, 3:58am

Something requirement related to above query,

i have a data like this

Thu Sep 6 02:15:54 2012->ABCServ(603):$Id: ABCServ.c,v 1.3 2006/08/24 20:52:17 rkaps 
Thu Sep 6 02:20:12 2012->ABCServ(603):$Id: ABCServ.c,v 1.3 2006/08/24 22:52:17 rkaps 
'
'
'
'
'
Thu Sep 6 23:15:54 2012->ABCChild(610):child attempting to open c-tree 02:24:20 rkaps

I want to retrieve the data(full record/all fields) between 02:00:00 to 03:00:00 based on the 4th column.
I am using the code like

awk '/0[2-2]:[0-2][0-9]:[0-9][0-9]/ || /[0-0][2-2]:[0-9][0-9]:[0-9][0-9]/,/0[2-2]:[0-2][0-9]:[0-9][0-9]/' filename

For the above query, its should not display the last record

Thu Sep 6 23:15:54 2012->ABCChild(610):child attempting to open c-tree 02:24:20 rkaps

How can i put the check for the above query.

pamu · September 6, 2012, 4:32am

Check this

NareshN · September 6, 2012, 5:31am

i just want to apply a filter based on the 4th column.

pamu · September 6, 2012, 5:48am

Use below command...

START="02:00:00"
NOW="03:00:00"
awk -v ST="$START" -v NW="$NOW" '{ if ($4 >= ST && $4 <= NW ) print }' file

NareshN · September 6, 2012, 6:27am

thanks pamu,
but its not working.
i need the code something like

awk '{print $4}' filename | awk '/0[2-2]:[0-2][0-9]:[0-9][0-9]/ || /[0-0][2-2]:[0-9][0-9]:[0-9][0-9]/,/0[2-2]:[0-2][0-9]:[0-9][0-9]/' filename |  awk 'print $0'

Based on the 1st awk result, i need to search the data by using 2nd awk and to display all the records by using 3rd awk command.

Above code is just an example.

this

awk '/0[2-2]:[0-2][0-9]:[0-9][0-9]/ || /[0-0][2-2]:[0-9][0-9]:[0-9][0-9]/,/0[2-2]:[0-2][0-9]:[0-9][0-9]/'

is working fine.I just need to apply a filter.

pamu · September 6, 2012, 6:41am

I am not getting what is problem

I am getting perfect here..

START="02:00:00"
NOW="03:00:00"
$ awk -v ST="$START" -v NW="$NOW" '{ if ($4 >= ST && $4 <= NW ) print }' file
Thu Sep 6 02:15:54 2012->ABCServ(603):$Id: ABCServ.c,v 1.3 2006/08/24 20:52:17 rkaps
Thu Sep 6 02:20:12 2012->ABCServ(603):$Id: ABCServ.c,v 1.3 2006/08/24 22:52:17 rkaps

If you want to change the filter just change the values for START and NOW.

Why you are forcing to use your solution..

is this a homework..?

NareshN · September 6, 2012, 7:12am

this code is already working in production env. I just want to enhance this.

awk '/0[2-2]:[0-2][0-9]:[0-9][0-9]/ || /[0-0][2-2]:[0-9][0-9]:[0-9][0-9]/,/0[2-2]:[0-2][0-9]:[0-9][0-9]/

this is just one example. Like this, there is a lot of scenairos to fetch the data between start and end date.