Parse file from remote server to calculate count of string existence in that file

Shell Programming and Scripting
1

Hi

I need to parse the file of same name which exist on different servers and calculate the count of string existed in both files.

Say a file abc.log exist on 2 servers.
I want to search for string "test" on both files and calculate the total count of search string's existence.

For example if file abc.log on server 1 has string "test" 2 times
and file abc.log on server 2 has string "test" 4 times.

then the output will be
StringName : Count
example
test : 6 times

Note : I have created the password less connectivity using ssh-keygen.

Please help.

Thanks in advance.

2

You can do something like:

Create a file with a list of the hosts you want to query:

$> vi hostlist
host1
host2
host3
..

Then take some code:

PATTERN=mypattern
while read LINE; do
     ssh -n $LINE "grep -c $PATTERN /path/to/file"
done < hostlist |\
awk -v pattern="$PATTERN" '
     {sum+=$1}
     END{
          print "Pattern:",pattern
          print "Sum:",sum
     }
'
3

Can we do it without using ssh?

4

You can do that with any remote access tool that is suitable. ssh might be the easiest and still secure.

5

Actually, I am willing to write a shell script that can fetch all the text in between certain tags (like <span class="someclass">Fetch This Text</span>) from an HTML file located at different web-server.

6

Maybe you open up a new thread for your request where you can go into details so we don't hijack this thread.

7

Hi zaxxon,

Thanks for the script but
I want to use awk command with ssh in the script mentioned and want to search from the logs of last 15 minute only of the file which includes the data of whole day.

What i have done till now is as follows and executes the code on same server itself which is working fine:

#!/bin/bash

to=`date +"%d-%b-%Y %T"`
echo $to
let from_in_seconds=`date +%s`-900
from=`date -d @$from_in_seconds +"%d-%b-%Y %T"`
echo $from

shortcodes=( "56882" "58585" "58888" "57575" "57677" );
for shortcode in ${shortcodes[@]}
do
    count=0
    sh_count=`awk '$0>=from && $0<=to' from="$from" to="$to" App_OP.log | grep "ShortCode=tel:${shortcode}" | wc -l`
    count=`expr $count + $sh_count`   
    echo "${shortcode} : "$count
done

In the above code i want to execute the above sh_count command on other server too from the same script itself and get count from the output and add to the total count to get final output.

Please help for the above situation
Thanks in advance.