Help on searching for a string on multiple files

Shell Programming and Scripting
1

Hi all,

I am sure some gurus will find a better way of doing this. FYI, I've manually modified some of the data 'coz they are somewhat confidential, so there may be some typo errors.

At the moment, I have 3 files that I am trying to search for. Sometime in the future, it may go beyond 3 files.

From these 3 files, I have to search for whether an application service is found in the files or not. If the file contains it, that means the application service is running on that server, otherwise it is not.

At the moment, I am reading each file and then awk'ing the field for the application service to a variable. I then read each file and grep for this string.

At the moment, the script is working as it is but it seems complicated. I thought maybe someone from the FORUM can make it better in some way.

Below is the main part that awk the field and search for the application string on each file

 
 
ls -1 *server* > tmp.00
server_count=`wc -l tmp.00 | awk '{ print $1}'`
#cat /dev/null > service_to_check.tmp.00
while read file1
do
   server=`echo $file1 | awk -F"." '{ print $1 }'`
   while read file2
   do
      service=`echo $file2 | awk '{ print $2 }'`
      service_count=0
      service_count=`grep -il ${service} *server* | wc -l | awk '{ print $1 }'`
      #echo "... Checking ${server} : Service = ${service} : ${service_count}"
      if [[ ${service_count} < ${server_count} ]] ; then
         grep -il ${service} *server* | awk -F"." '{ print $1 }' | sort | uniq > service_node.tmp.00
         service_node_server=""
         while read service_node
         do
            service_node_server="${service_node} :: ${service_node_server}"
         done < service_node.tmp.00
         echo "${service} :: FOUND ONLY in --> ${service_node_server} :: "
      fi
   done < ${file1}
done < tmp.00
  • Below is example of the files that I am comparing or trying to. Field2, test01_app is the one that I need to check if it exist on all the files. If it does not exist. For the sample files below, only test01_app exist on all files.
 
 
server1:
 
Service "test01_app.inq.com.ph" has 1 instance(s).
Service "test03_app.inq.com.ph" has 1 instance(s).
 
server2:
 
Service "test01_app.inq.com.ph" has 1 instance(s).
Service "test02_app.inq.com.ph" has 1 instance(s).
Service "test04_app.inq.com.ph" has 1 instance(s).
 
server3:
 
Service "test01_app.inq.com.ph" has 1 instance(s).
Service "test02_app.inq.com.ph" has 1 instance(s).
Service "test03_app.inq.com.ph" has 1 instance(s).
 
tmp.00
server1.inq.com.ph.txt
server2.inq.com.ph.txt
server3.inq.com.ph.txt

I believe the script is very beginner like. It works to some extent but I believe it can be better and there is one other thing that I can't figure out to do which is to also find where the string does not exist. There is a bit of 'cheating' involve as well I guess where the search is based on the grep count.

Note that there is two (2) while loops. This is because I won't know which of the files has all the services so I need to read them all using the outer while loop and the inner while loop is to check all files for the existence of the application string. tmp.00 contains the name of the files to compare, the files are named something like server1.inq1.com.ph.

On the code section below, while I am able to find the servers/files where the service string is not found, how do I print the servers/files where the service string is NOT FOUND? That is, instead of saying

echo "${service} :: FOUND ONLY in --> ${service_node_server} :: "

I would want to do

echo "${service} :: FOUND ONLY in --> ${service_node_server} :: NOT FOUND in --> ${service_node_server_not_found}"

. I can't find a grep option to do that. I've experimented with diff but can't get it to print in the format that I wanted it to.

Another useful thing to be able to do is to report where all files contain the application string, i.e. PASSED because it is on all server files. I believe that is just a matter of putting an else in the if-then.

 
      if [[ ${service_count} < ${server_count} ]] ; then
         grep -il ${service} *server* | awk -F"." '{ print $1 }' | sort | uniq > service_node.tmp.00
         service_node_server=""
         while read service_node
         do
            service_node_server="${service_node} :: ${service_node_server}"
         done < service_node.tmp.00
         echo "${service} :: FOUND ONLY in --> ${service_node_server} :: "
      fi

Any feedback will be much appreciated. Thanks in advance.

2

Why cant you try something like below

if [ `grep -il test01_app *server* | wc -l` -eq 3 ]
then 
echo "test01_app found in all servers"
else
echo "test01_app not found in all servers"
fi

obviously you need to replace hard-coded values by your original values in above script

3

you can also try like this for getting common from all three files .

awk -F "." '{if(FILENAME=="server1"){flag1[$1]=1;}else if(FILENAME=="server2"){flag2[$1]=1;}else{if(flag1[$1]==1 && flag2[$1]==1){print $0 >"common.txt"}}}' server1 server2 server3

here is the output files

> cat common.txt 
Service "test01_app.inq.com.ph" has 1 instance(s

)

or

for getting common and difference

cat server1 server2 server3 | sort | uniq > all_files #combining all three files. 
awk -F "." '{if(FILENAME=="server1"){flag1[$1]=1;}else if(FILENAME=="server2"){flag2[$1]=1;}else if(FILENAME=="server3"){flag3[$1]=1;}else{if(flag1[$1]==1 && flag2[$1]==1 && flag3[$1]==1){print $0 >"common.txt"} else{print $0 > "not_common"}}}' server1 server2 server3 all_files