compare realtime

Shell Programming and Scripting
1

I have two log files which keeps appending every sec...I want to extract a certain field from each file(using awk for extracting the data) and compare them in real time...

ex:
log1
122
234
567

log2
234
567

log3
122

i need a log3 which keeps appending the data found in log1 and not in log2..

2

Could you show us a sample of two log files and the expected content for log3? Which field in the log1 and log2 you want to extract?

3

#!/bin/bash
#assuming that there is only one field in file1 and file2

for i in $(cat file1)
do
if grep -q $i file2
then :
else
echo $i >>file3
fi
done

4

log1 which keeps appending every sec
234,abc
678,def
345,fgh

awk -F"," '{print $1}' log1

will give me
234
678
345

log2 which keeps appending every sec
345, ghi
678, jkl

awk -F"," '{print $1}' log2

will give me
345
678

now i want a log3 which compares the output of the above two commands and keeps appending... ex: i should get 234 in log3 as its there in log1 and not in log2

5 
#! /opt/third-party/bin/perl

my($i, %fileHash_a, %fileHash_b);

for( ;1; sleep 5 ) {

%fileHash_a = ();
%fileHash_b = ();

open(FILE, "< a") || die "file a <$!>\n";

while( <FILE> ) {
  chomp;
  $fileHash_a{$_} = $i++;
}

close(FILE);

$i=0;
open(FILE, "< b") || die "file b <$!>\n";
while( <FILE> ) {
  chomp;
  $fileHash_b{$_} = $i++;
}

close(FILE);

  open(FILE, "> update") || die "update <$!>\n";

  foreach my $k ( keys %fileHash_a ) {
    if ( exists $fileHash_b{$k} ) {
    }
    else {
      print FILE $k . "\n";
    }
  }
  close(FILE);
}


exit 0

All the differences would be captured in the file 'update'
This is a continuously running script, run it as a background process
Refresh rate can be altered through the sleep value given in for loop
Minimal time taken in comparison and generating what is not available in first file with respect to the second file

6 
$cat tmp1
234
678
345
$cat tmp2
345
678

$sort tmp1 > tmp1.sort
$sort tmp2 > tmp2.sort
$comm -23 tmp1.sort tmp2.sort
234

comm -23 prints only lines in the first file but not in the second
or

$grep -v -f tmp2 tmp1
234
7

Thanks....but this doesnt give me a realtime log...by the time i do cat and compare i would have lost many entries in the meantime(log gets appended with data every sec) .I want a log which keeps comparing in real time and gets appended every time it finds a diff in file...like in tail -f

8

With the concept of tail -f, only the script was provided, where you have the list updated in the file 'update' every 'n' seconds maintained in the for loop, if you want to refresh the list as soon as possible, decrement the sleep time.

Though I had not done extensive testing for the posted script. I just verified whether the update file is getting refreshed and it worked for me.

Am sure the required difference would be necessarily be captured.

Please post us with what is needed and whats happening! :slight_smile: