I have two log files of same name and structure which resides on different servers with ssh-keygen installed.
I want to search for a list of strings available in an array on last 15 minutes logs of both files and calculate the total count of occurence of each string.
Sample format of both file is same and as follows:
08-Sep-2010 07:58:40,044|8923629811|aclwireless_app|1282498070897|NotifySmsReception|CP URL to be called : http://203.122.58.164/uninorintegration/triggerdelivery?ShortCode=tel:567678888&Keyword=&sId=tel:8923629811&TransactionID=SDP-DM-31272772&ContentID=KKFUN%20#812#00209#405818#19500413#5853#102#0#
08-Sep-2010 07:58:41,672|9126657443|aclwireless_app|1282498070897|NotifySmsReception|CP URL to be called : http://203.122.58.164/uninorintegration/triggerdelivery?ShortCode=tel:567678888&Keyword=&sId=tel:9126657443&TransactionID=SDP-DM-31272777&ContentID=KKFUN%20#815#00606#405880#2060761#1445#102#0#
08-Sep-2010 09:47:52,778|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|ChargingInterceptor - subscriber details processed sucessfully- {arg0.referenceCode=channelType:SMS;transactionId:1283918818906926;pricePtAvl:true;PPL_FLAG:FALSE;CorrelatorId:BEA1-611DC04C995F;contentType:Text;senderMSISDN:52211;contentId:1096;Subscriber_ID:BJ#12101396;SPACCOUNT_ID:PhoneyTunes;methodName:sendSms;Sub_Profile:Pre-Paid;IMSI:405876120820301;Customer_Retrieved:true;contentName:PT_JOKES;cpID:Phoneytunes;origContentOwnerId:10225;serviceId:PT_JOKES;Circle_Name:BJ;CustomerID:B_12318157;Circle_ID:4;isRated:No;MSISDN:9122796686;productName:JKGEN001;basePrice:-1.0, arg0.endUserIdentifier=9122796686, arg0.charge.description=CP is Calling TO This API, arg0.charge.currency=INR, arg0.charge.code=code0, arg0.charge.amount=0}
08-Sep-2010 09:47:52,789|7398965380|subscription_app|-7932740d%3A12aedfb6d15%3A-26d3|ChargeAmount|GetBalance|PaymentPlugin-Response - Retrieved Balance Bucket: 1;20091003;20110306;122.50;|
08-Sep-2010 09:47:52,790|7398965380|subscription_app|-7932740d%3A12aedfb6d15%3A-26d3|ChargeAmount|Exception: [Message:null]
08-Sep-2010 09:47:52,810|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|DMGenerateLogInterceptor - InternalTransactionID:SDP-DM-31297182, ExternalTransactionID:1283918818906926, TransactionStatus:Sent For Delivery, INStatus:FAILED
08-Sep-2010 09:47:52,813|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|CustomCDRInterceptor - CDR Info[Optional_Field1:30.0,Subscription_Channel:,Optional_Field2:Subscribe,Transaction_ID:SDP-DM-31297182,Content_ID:1096,IMEI:,Product_Name:JKGEN001,PPL_FLAG:FALSE,Charge_Code:code0,Base_Price:0.0,CustomerID:B_12318157,Circle_Name:BJ,Sender_MSISDN:52211,IMSI:405876120820301,Content_Status:,Location:BJ,Circle_ID:4,Original_Content_Owner_ID:10225,CPNAME:PhoneyTunes,Content_Price:0,Zone:,Content_Name:PT_JOKES,Static_ID:BJ#12101396,External_Correlation_Id:1283918818906926,Subscription_Type:,MSISDN:9122796686,Transaction_Mode:SMS,Transaction_DateTime:2010-09-08 09:47:52 GMT+05:30,Content_Type:Text,Sub_Profile:Pre-Paid,CPID:Phoneytunes,Other_Info:]
08-Sep-2010 09:47:52,820|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|PaymentPlugin-Request - [amountToBeCharged-0 | balanceChargeDetails0;0;0;0;| | transactionId-null | profileType-Pre-Paid ]
08-Sep-2010 09:47:52,823|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|DMGenerateLogInterceptor - InternalTransactionID:SDP-DM-31297182, ExternalTransactionID:1283918818906926, TransactionStatus:Sent For Delivery, INStatus:Success
08-Sep-2010 09:47:52,823|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|Response Parameters:[Result:Success]
08-Sep-2010 09:47:52,970|BJ#8748783|phtunes_app||SendSms|Request Params:[Address:tel:BJ#8748783,Message:Joke: Jack: What are you going to be when you have finished studying and passing all your exams. Buck: Probably an old age pensioner. ,SenderName:52211,ChargingInformation:[Code:code0,Currency:INR,Description:CP is Calling TO This API,Amount:0],SimpleReference:[],X-Params:[[Key:transactionId,Value:1283918819184906],[Key:origContentOwnerId,Value:10225],[Key:channelType,Value:SMS],[Key:cpID,Value:Phoneytunes],[Key:serviceId,Value:PT_JOKES],[Key:PPL_FLAG,Value:FALSE],[Key:contentType,Value:Text],[Key:contentName,Value:PT_JOKES],[Key:senderMSISDN,Value:52211],[Key:isRated,Value:No],[Key:contentId,Value:1096]]]
08-Sep-2010 09:47:52,970|BJ#8748783|phtunes_app||SendSms|ValidateMSISDNInterceptor - sid to be validated BJ#8748783
In the above logs i want to search a list of strings which are available in an array as follows:
shortcodes=( "56882" "58585" "58888" "57575" "57677" );
Now I want to search the string contains "ShortCode=tel:"$shortcode from the last 15 minutes logs and calculate their count accordingly.For example
56882 : 12
58585 : 15
58888 : 18
57575 : 12
57677 : 98
The above output includes sum of count matching the string from both files on different servers.
The code what i have written is as follows and giving error while accessing file on other server:
#!/bin/bash
to=`date +"%d-%b-%Y %T"`
echo $to
let from_in_seconds=`date +%s`-90000
from=`date -d @$from_in_seconds +"%d-%b-%Y %T"`
echo $from
shortcodes=( "52211" );
for shortcode in ${shortcodes[@]}
do
count=0
sh_count=`awk '$0>=from && $0<=to && $0~"ShortCode=tel:" s' from="$from" to="$to" s="$shortcode" App_OP.log | wc -l`
count=`expr $count + $sh_count`
sh_count2=`ssh -n root@10.0.0.1 "awk '$0>=from && $0<=to && $0~"ShortCode=tel:" s' from="$from" to="$to" s="$shortcode" App_OP.log | wc -l"`
count=`expr $count + $sh_count2`
echo "${shortcode} : "$count
done
Please guide me for the above and if possible provide me more optimized way of doing above.
Thanks in advance