log report script

Hello everyone,

By any chance, anyone has a script (shell) that I can modify for the following:

I have daily log files and they look like:

Accept aabbcc [02/Jun/2008:10:34:36 -0600] "111.11.111.111 uid=MyID, ou=People, o=helloworld.com" "webagent" [] []
Reject aabbcc [02/Jun/2008:10:34:36 -0600] "111.11.111.111 uid=MyID, ou=People, o=helloworld.com" "webagent" [] []
Validate ddeeff [02/Jun/2008:10:34:36 -0600] "111.11.111.111 uid=MyID, ou=People, o=helloworld.com" "webagent" [] []

The need is to run a script against it with some inputs, such as:

./script.sh uid start_date end_date all_dates

above may not contain the end_date; but eventually there should be sometimes.

The output should be something like this:

# Total Activities based on uid

uid=aabbcc
Accept: #
Reject: #
Validate: #

uid=ddeeff
Accept: #
Reject: #
Validate: #

so on... If anyone can help, greatly appreciated.

If you get errors use nawk, gawk or /usr/xpg4/bin/awk on Solaris.

awk '
!($1 in a){a[++i]=$1;a[$1]}
!($2 in b){b[++j]=$2;b[$2]}
END{
  for(k=1;k<=j;k++){
    print "uid=" b[k]
    for(l=1;l<=i;l++){
      print a[l] ": #"
    }
    print ""
  }
}' file

Regards

I appreciate for the response. I tried as you suggested on solaris but getting the error:
i used it as

awk -f a.awk a.txt > a.out

awk: syntax error near line 1
awk: bailing out near line 1

any suggestion please?

thnx much again

As mentioned, on Solaris:

/usr/xpg4/bin/awk '
!($1 in a){a[++i]=$1;a[$1]}
!($2 in b){b[++j]=$2;b[$2]}
END{
  for(k=1;k<=j;k++){
    print "uid=" b[k]
    for(l=1;l<=i;l++){
      print a[l] ": #"
    }
    print ""
  }
}' file

Regards

I appreciate for very fast response; I had tried the same as code above but it was the same error...I am unsure why it is bailing out at the 1st line...and syntax error..

awk: syntax error near line 1
awk: bailing out near line 1

It works well with the newer versions of awk, I made some changes, give this a try:

/usr/xpg4/bin/awk '
{if(!($1 in a)){a[++i]=$1;a[$1]}}
{if(!($2 in b)){b[++j]=$2;b[$2]}}
END{
  for(k=1;k<=j;k++){
    print "uid=" b[k]
    for(l=1;l<=i;l++){
      print a[l] ": #"
    }
    print ""
  }
}' file

Hi again, I added the following it works partially what I was looking for:

#!/usr/bin/awk -f
#!/usr/bin/nawk -f
#!/usr/bin/mawk -f
#!/usr/bin/gawk -f

output was:

uid=aabbcc
Accept: #
Reject: #
Validate: #

Is there a way to replace # with number of occurance and is there also a way to print the date stamp in the file below it ?

thanks much once again

You can use variables, assuming you have gawk:

#!/bin/sh

dat=`date`
number="5"

/usr/bin/gawk -v num="$number" -v d="$dat" '
{if(!($1 in a)){a[++i]=$1;a[$1]}}
{if(!($2 in b)){b[++j]=$2;b[$2]}}
END{
  for(k=1;k<=j;k++){
    print "uid=" b[k]
    for(l=1;l<=i;l++){
      print a[l] ": " num
    }
    print ""
  }
  print d
}' file

Save this in a file, make it executable and run it as a shell script.

I am sorry to keep asking questions ... I have put that in a script and ran. One thing is number of occurances did not print. and date was not working...i removed that part. Is there a way to print " [02/Jun/2008:10:34:36 -0600] " as an example for the corresponding uid in the same line of the input file ?

such as:

cat a.out

uid=aabbcc
Accept: 1
Reject: 1
Validate: 1
TimeStamp: [02/Jun/2008:10:34:36 -0600]

thanks much again

It should work with gawk or nawk. An example if you have the timestamp in a variable:

#!/bin/sh

number="5"
ts="TimeStamp: [02/Jun/2008:10:34:36 -0600]"

/usr/bin/gawk -v num="$number" -v d="$dat" '
{if(!($1 in a)){a[++i]=$1;a[$1]}}
{if(!($2 in b)){b[++j]=$2;b[$2]}}
END{
  for(k=1;k<=j;k++){
    print "uid=" b[k]
    for(l=1;l<=i;l++){
      print a[l] ": " num
    }
    print ts
  }
}' file

Hi Again,

I used nawk and got it working...I think gawk is not installed in my server. However, counts per variables like Accept, Reject did not work. I was also not bale to print the date stamp ...not sure ...tried modifying but getting the same results...any idea is appreciated. Thanks once again.

Sorry, I forgot to replace the variable in the code and to add a counter:

#!/bin/sh

ts="TimeStamp: [02/Jun/2008:10:34:36 -0600]"

/usr/bin/gawk -v num="$number" -v timestamp="$ts" '
{if(!($1 in a)){a[++i]=$1;a[$1]}}
{if(!($2 in b)){b[++j]=$2;b[$2]}}
END{
  for(k=1;k<=j;k++){
    c++
    print "uid=" b[k]
    for(l=1;l<=i;l++){
      print a[l] ": " c
    }
    print timestamp
  }
}' file