differentiating two sets

skyineyes · December 15, 2008, 4:49am

Hi

Suppose i have a set of files like this

set1

a.cpp@@main/5
b.cpp@@main/6

set 2

m.cpp@@main/51
n.hpp@@main/51
a.cpp@@main/15
b.cpp@@main/2

there may be files with same name in 2 sets. i need to list the files in set1 which have last numeric field less than the same file in the other set?

how can this be achieved?

radoulov · December 15, 2008, 5:27am

Use nawk or /usr/xpg4/bin/awk on Solaris:

awk -F'\57' 'NR == FNR {
 _[$1] = $2
 next
 }
$1 in _ && _[$1] > $2
' set2 set1

$ head set*
==> set1 <==
a.cpp@@main/5
b.cpp@@main/6

==> set2 <==
m.cpp@@main/51
n.hpp@@main/51
a.cpp@@main/15
b.cpp@@main/2
$ awk -F'\57' 'NR == FNR {
 _[$1] = $2
 next
 }
$1 in _ && _[$1] > $2
' set2 set1 
a.cpp@@main/5

skyineyes · December 15, 2008, 5:46am

Hi

Thanks. But my requirement is refined as follows
Both the sets will come as the output a list of filenames i.e each set will be having list of filenames.
from these two sets of filenames we have to have do the operation which you have performed earlier.
btw what is the 57 in your code?

radoulov:

Use nawk or /usr/xpg4/bin/awk on Solaris:

awk -F'\57' 'NR == FNR {
 _[$1] = $2
 next
 }
$1 in _ && _[$1] > $2
' set2 set1

$ head set*
==> set1 <==
a.cpp@@main/5
b.cpp@@main/6

==> set2 <==
m.cpp@@main/51
n.hpp@@main/51
a.cpp@@main/15
b.cpp@@main/2
$ awk -F'\57' 'NR == FNR {
 _[$1] = $2
 next
 }
$1 in _ && _[$1] > $2
' set2 set1 
a.cpp@@main/5

radoulov · December 15, 2008, 5:51am

Could you provide an example, please?

ASCII octal for /.

skyineyes · December 15, 2008, 6:08am

Hi

Each of the two sets will have following filename format

/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOWONHOLD.trg@@/main/3
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOW.trg@@/main/3
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSDIARY.trg@@/main/4
/vobs/odyssey/sm/sybase/ddl/ods/SM_SMVALIDCASHFLOW.trg@@/main/5

so we just need to see what filenames as present in both and then compare their last fields which is a number?

Can you help in this regard?

radoulov · December 15, 2008, 6:22am

Something like this may work with GNU Awk:

awk 'BEGIN {
  FS = OFS = "\57"
  }
NR == FNR {
  t = $NF; NF = NF - 1
 _[$0] = t; next
 }
{
  t = NF; NF = NF - 1
  if ($0 in _ && _[$0] > t)
    print $0, t
}' set2 set1

For other awks try this:

awk 'BEGIN {
  FS = OFS = "\57"
  }
NR == FNR {
  t = $NF; sub(/\/[^\/]*$/, "")
 _[$0] = t; next
 }
{
  t = $NF; sub(/\/[^\/]*$/, "")
  if ($0 in _ && _[$0] > t)
    print $0, t
}' set2 set1

skyineyes · December 15, 2008, 6:43am

Hi

Getting error. Please have a look

Set 1

/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOWONHOLD.trg@@/main/3
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOW.trg@@/main/3
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSDIARY.trg@@/main/4
/vobs/odyssey/sm/sybase/ddl/ods/SM_SMVALIDCASHFLOW.trg@@/main/5

set 2

/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOWONHOLD.trg@@/main/4
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOW.trg@@/main/21
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSDIARY.trg@@/main/14
/vobs/odyssey/sm/sybase/ddl/ods/SM_SMVALIDCASHFLOW.trg@@/main/15

The script is as follows:

#!/bin/sh
set1=$1
set2=$2
awk 'BEGIN {
FS = OFS = "\57"
}
NR == FNR {
t = $NF; NF = NF - 1
_[$0] = t; next
}
{
t = NF; NF = NF - 1
if ($0 in _ && _[$0] > t)
print $0, t
}' set2 set1

When run the script as follows:

/home/odyssey/f091018>./script.sh set1 set2
awk: syntax error near line 10
awk: illegal statement near line 10
*********************************

radoulov:

Something like this may work with GNU Awk:

awk 'BEGIN {
  FS = OFS = "\57"
  }
NR == FNR {
  t = $NF; NF = NF - 1
 _[$0] = t; next
 }
{
  t = NF; NF = NF - 1
  if ($0 in _ && _[$0] > t)
   print $0, t
}' set2 set1

For other awks try this:

awk 'BEGIN {
  FS = OFS = "\57"
  }
NR == FNR {
  t = $NF; sub(/\/[^\/]*$/, "")
 _[$0] = t; next
 }
{
  t = $NF; sub(/\/[^\/]*$/, "")
  if ($0 in _ && _[$0] > t)
   print $0, t
}' set2 set1

radoulov · December 15, 2008, 6:52am

Change the script like this:

#! /usr/bin/nawk -f

BEGIN {
  FS = OFS = "\57"
  }
NR == FNR {
  t = $NF; sub(/\/[^\/]*$/, "")
 _[$0] = t; next
 }
{
  t = $NF; sub(/\/[^\/]*$/, "")
  if ($0 in _ && _[$0] > t)
    print $0, t
}

And run like this (notice the order of the input files):

./script.awk set2 set1

skyineyes · December 15, 2008, 7:08am

I ran like the below. But no output at all.

Does the code needs any change? something related to cmd line arguments????
Ok can we modfiy it as follows:

Set 1 and Set2 are got from the o/p of some command.

radoulov:

Change the script like this:

#! /usr/bin/nawk -f

BEGIN {
  FS = OFS = "\57"
  }
NR == FNR {
  t = $NF; sub(/\/[^\/]*$/, "")
 _[$0] = t; next
 }
{
  t = $NF; sub(/\/[^\/]*$/, "")
  if ($0 in _ && _[$0] > t)
   print $0, t
}

And run like this (notice the order of the input files):

./script.awk set2 set1

radoulov · December 15, 2008, 7:13am

Try swapping the input files.

skyineyes · December 15, 2008, 7:27am

Seems its working. But still i'll paste what is needed

/home/odyssey/f091018>cat set1
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOWONHOLD.trg@@/main/3
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOW.trg@@/main/3
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSDIARY.trg@@/main/4
/vobs/odyssey/sm/sybase/ddl/ods/SM_SMVALIDCASHFLOW.trg@@/main/5

/home/odyssey/f091018>cat set2

/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOWONHOLD.trg@@/main/13
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSCASHFLOW.trg@@/main/23
/vobs/odyssey/sm/sybase/ddl/ods/SM_CFSDIARY.trg@@/main/4
/vobs/odyssey/sm/sybase/ddl/ods/SM_SMVALIDCASHFLOW.trg@@/main/15

The requirement is all the files in Set 1 should have last field bigger than the same file in set 2 (if present .. ). If set2 version is higher than we want to list that file name in the o/p from set 1.

But will the usage remains the same? i.e ./script.awk set2 set1 ????

radoulov · December 15, 2008, 7:42am

The script should be OK.
Yes, same usage.

skyineyes · December 15, 2008, 7:58am

Thanks Alot. I appreciate your help.

Can we modify the script a bit.

say
I'll be using following two lines inside the script itself. So set1 and set2 will populate inside awk script file only.

cleartool find . -version "lbtype($label1)" -print > /tmp/set1

cleartool find . -version "lbtype($label2)" -print > /tmp/set2

Now how can we use set1 and set2 inside the script itself and not as external arguments??????

what are the changes required in this case?????

radoulov · December 15, 2008, 8:41am

You may try this quick and dirty solution (you should use bash or /usr/dt/bin/dtksh for this script):

#! /bin/bash

nawk 'BEGIN {
  FS = OFS = "\57"
  }
NR == FNR {
  t = $NF; sub(/\/[^\/]*$/, "")
 _[$0] = t; next
 }
{
  t = $NF; sub(/\/[^\/]*$/, "")
  if ($0 in _ && _[$0] > t)
    print $0, t
}' <(cleartool find . -version "lbtype($label2)" -print) \
   <(cleartool find . -version "lbtype($label1)" -print)

I would write it all in AWK, but I don't have access to cleartool.

skyineyes · December 15, 2008, 11:31pm

Thanks alot !!!

radoulov:

You may try this quick and dirty solution (you should use bash or /usr/dt/bin/dtksh for this script):
#! /bin/bash

nawk 'BEGIN {
  FS = OFS = "\57"
  }
NR == FNR {
  t = $NF; sub(/\/[^\/]*$/, "")
 _[$0] = t; next
 }
{
  t = $NF; sub(/\/[^\/]*$/, "")
  if ($0 in _ && _[$0] > t)
   print $0, t
}' <(cleartool find . -version "lbtype($label2)" -print) \
   <(cleartool find . -version "lbtype($label1)" -print)
I would write it all in AWK, but I don't have access to cleartool.

summer_cherry · December 16, 2008, 1:13am

nawk -F"/" '{
if(NR==FNR)
	arr[$1]=$2
else
	if(exists arr[$1] && $2>arr[$1])
		print $1
}' file1 file2