Script to count particular type of records

Shell Programming and Scripting
1

Hi,

I have a huge file containing thousands of records which are of following pattern:

TYPE1
{
originNodeType : "IVR"
originHostName : "AAIVR"
originTransactionID : "01310559"
originTimeStamp : "20110620192440+0530"
hostName : "hhhh"
voucher : '0'D
rProfileID : "ZZZZ"
Before
{
Flags : "10000000"
Bal : "123"
}
After
{
Flags : "10000000"
Bal : "123"
}
}
TYPE1
{
originNodeType : "UGW"
originHostName : "AAUGW"
originTransactionID : "01310560"
originTimeStamp : "20110620192441+0530"
hostName : "asdf" 
voucher : '1'D
Before
{
Flags : "00000000"
Bal : "123"
}
After
{
Flags : "10000000"
Bal : "124"
}
}
TYPE2
{
originNodeType : "IVR" 
originHostName : "AAIVR"
originTransactionID : "01462112"
originTimeStamp : "20110624001221+0530"
hostName : "aqws"
Counter : '1'D
errorCode : 'invalidNumber (19)' 
}
TYPE2
{
originNodeType : "UGW" 
originHostName : "AAUGW"
originTransactionID : "01462113"
originTimeStamp : "20110624001221+0530"
hostName : "aqws"
Counter : '2'D
errorCode : 'invalidAcc (18)' 
}
TYPE2
{
originNodeType : "IVR" 
originHostName : "AAIVR"
originTransactionID : "01462112"
originTimeStamp : "20110624001222+0530"
hostName : "aqws"
Counter : '1'D
errorCode : 'invaliddate (17)' 
}

I need to extract following data from this type of file:

  1. Number of TYPE1 records where
    originNodeType : "IVR" AND voucher : '0'D AND rProfileID : "ZZZZ"

  2. Count of different type of errorCode from TYPE2 records where originNodeType : "IVR". For data above output should be:

errorCode : 'invalidNumber (19)' = 1
errorCode : 'invaliddate (17)' = 1

Fileds hostname & rProfileID are optional so they may not be present in all records.

As number of records is large so would request if someone can help with perl based script.

Thanks,
Madhukar

2

Hi,

I can't understand what are you looking for.

1.- From your first condition, my count is 1.
2.- From your second condition, my count is 2.

What does your output mean?

Regards,
Birei

3

Based on your requirements, the solution for step (2) can be very complex as you did not specify how many different
error codes exists, thus I just display the error and you can use another tool to count the different ones.

Here is what I came up with:

#!/usr/bin/ksh
typeset -i mCnt1=0
mLOrig='originNodeType : "IVR"'
mLVouch="voucher : '0'D"
mLProf='rProfileID : "ZZZZ"'
mLONType='originNodeType : "IVR"'
mLErr="errorCode :"
while read mLine; do
  mP1to4=$(echo ${mLine} | cut -c1-4)
  if [[ "${mP1to4}" = "TYPE" ]]; then
    if [[ "${mLine}" = "TYPE1" ]]; then
      if [[ "${mOrig}" = "Y" && "${mVouch}" = "Y" && "${mProf}" = "Y" ]]; then
        mCnt1=${mCnt1}+1
      fi
      mType="1"
      mOrig="N"
      mVouch="N"
      mProf="N"
    else
      if [[ "${mLine}" = "TYPE2" ]]; then
        mType="2"
        mONType='N'
      else
        mType="?"
      fi
    fi
  fi
  if [[ "${mType}" = "1" ]]; then
    if [[ "${mLine}" = "${mLOrig}" ]]; then
      mOrig='Y'
    else
      if [[ "${mLine}" = "${mLVouch}" ]]; then
        mVouch='Y'
      else
        if [[ "${mLine}" = "${mLProf}" ]]; then
          mProf='Y'
        fi
      fi
    fi
  fi
  if [[ "${mType}" = "2" ]]; then
    if [[ "${mLine}" = "${mLONType}" ]]; then
      mONType='Y'
    else
      if [[ "${mONType}" = "Y" ]]; then
        mP1to11=$(echo ${mLine} | cut -c1-11)
        if [[ "${mP1to11}" = "errorCode :" ]]; then
          echo "TYPE2 IVR ${mLine}"
        fi
      fi
    fi
  fi
done < input_file
echo "Found ${mCnt1} condition in TYPE1"
4

Thanks shell_life

Is there someway to keep the pattern search mLOrig='originNodeType : "IVR"' to be space insensitive. Like we get match in this case and even if we have 'originNodeType : "IVR"' (multiple blank or tabs)?

Hi birei,

Yes in the sample file the output which you mentioned is ok. I need count.
In sample pattern below, count for TYPE2 record with sample pattern should be 2.

TYPE1
{
originNodeType : "IVR"
originHostName : "AAIVR"
originTransactionID : "01310559"
originTimeStamp : "20110620192440+0530"
hostName : "hhhh"
voucher : '0'D
rProfileID : "ZZZZ"
Before
{
Flags : "10000000"
Bal : "123"
}
After
{
Flags : "10000000"
Bal : "123"
}
}
TYPE1
{
originNodeType : "IVR"
originHostName : "AAIVR"
originTransactionID : "01310559"
originTimeStamp : "20110620192440+0530"
hostName : "hhhh"
rProfileID : "ZZZZ"
voucher : '0'D
Before
{
Flags : "10000000"
Bal : "123"
}
After
{
Flags : "10000000"
Bal : "123"
}
}
TYPE1
{
originNodeType : "UGW"
originHostName : "AAUGW"
originTransactionID : "01310560"
originTimeStamp : "20110620192441+0530"
hostName : "asdf"
voucher : '1'D
Before
{
Flags : "00000000"
Bal : "123"
}
After
{
Flags : "10000000"
Bal : "124"
}
}
TYPE2
{
originNodeType : "IVR"
originHostName : "AAIVR"
originTransactionID : "01462112"
originTimeStamp : "20110624001221+0530"
hostName : "aqws"
Counter : '1'D
errorCode : 'invalidNumber (19)'
}
TYPE2
{
originNodeType : "UGW"
originHostName : "AAUGW"
originTransactionID : "01462113"
originTimeStamp : "20110624001221+0530"
hostName : "aqws"
Counter : '2'D
errorCode : 'invalidAcc (18)'
}
TYPE2
{
originNodeType : "IVR"
originHostName : "AAIVR"
originTransactionID : "01462112"
originTimeStamp : "20110624001222+0530"
hostName : "aqws"
Counter : '1'D
errorCode : 'invaliddate (17)'
}

Thanks,

5

This code will work on mixed cases and multiple spaces:

#!/usr/bin/ksh
typeset -i mCnt1=0
mLOrig='originnodetype:"ivr"'
mLVouch="voucher:'0'd"
mLProf='rprofileid:"zzzz"'
mLONType='originnodetype:"ivr"'
mLErr="errorcode:"
tr -A '[:upper:]' '[:lower:]' < input_file | tr -d ' ' |
while read mLine; do
  mP1to4=$(echo ${mLine} | cut -c1-4)
  if [[ "${mP1to4}" = "type" ]]; then
    if [[ "${mLine}" = "type1" ]]; then
      if [[ "${mOrig}" = "Y" && "${mVouch}" = "Y" && "${mProf}" = "Y" ]]; then
        mCnt1=${mCnt1}+1
      fi
      mType="1"
      mOrig="N"
      mVouch="N"
      mProf="N"
    else
      if [[ "${mLine}" = "type2" ]]; then
        mType="2"
        mONType='N'
      else
        mType="?"
      fi
    fi
  fi
  if [[ "${mType}" = "1" ]]; then
    if [[ "${mLine}" = "${mLOrig}" ]]; then
      mOrig='Y'
    else
      if [[ "${mLine}" = "${mLVouch}" ]]; then
        mVouch='Y'
      else
        if [[ "${mLine}" = "${mLProf}" ]]; then
          mProf='Y'
        fi
      fi
    fi
    continue
  fi
  if [[ "${mType}" = "2" ]]; then
    if [[ "${mLine}" = "${mLONType}" ]]; then
      mONType='Y'
    else
      if [[ "${mONType}" = "Y" ]]; then
        mP1to10=$(echo ${mLine} | cut -c1-10)
        if [[ "${mP1to10}" = "${mLErr}" ]]; then
          echo "TYPE2 IVR ${mLine}"
        fi
      fi
    fi
  fi
done
echo "Found ${mCnt1} condition in TYPE1"
1 Like
6

Thanks:)