Hi, There are two files in UNIX system with some lines are exactly the same, some lines are not.
I want to compare these two files.The 2 files (both the files have data in Column format )should be compared row wise and any difference in data for a particular row should lead to storage of data of second file in another new third file.
i.e here we are assuming second file( file 2 ) to be updated version of first file (file 1) and any difference in row wise matching would lead to storage of the updated /modified data in another third file .
hi yogesh diff is NOT solving the problem ( i have tried that)..
bcoz the output apart from giving the difference in file contents is also displaying line numbers and other special kind of instructions ...
we got to pick up only the modified data from this third file .. so diff doesn't work out here properly
for eg below is one such display
field 1( f1) was the desired field which was to be picked up from the diff_file.txt file .. but again problem is that the diff_file.txt contained the line numbers in some places Also
(note: diff_file.txt has huge number of rows of data) ,so these unwanted line numbers which lie in same column as the field 1 data also gets picked up with the field1 data
eg line numbers like 4d3,5a5 etc
Note :file 1 n file 2 contains numbers as data
so i want final.txt file to have pure numbers only
Hey Vino thanks a lottt !!!!! ..but again ur suggestion is applicable only when both the files have certain contents which are unique to that particular file and NOT found in other one ...
for eg
two files file3 having contents
bat
mat
rat
and file4 having contents
bat
rat
hat
in such case grep -vf file3 file4
would display only "hat" and not rat because rat is present there is file 3
but our criteria is that
rat as well as hat should be displayed as we are matching the column contents row wise ONLY.. so here the values at second and third place of file 4 differ from those of file 3 so accordingly i want rat and hat to be displayed
any idea
well i tried that it is displaying
grep -vf file4 file3
mat
but criteria is that
i shud be getting rat and hat ( file 4 is assumed as updated version of file3 by the user concerned) as on comparing the 2nd and 3rd place values of file 4 do NOT match with the 2nd and 3rd place values of file3
the contents of both the files will be compared row wise( i.e. row 1 content of file1 to be compared with row 1 content of file2 and so on)
if there is any difference/change in values (it can be numbers or those words) of corresponding rows then the script will pick up the numbers of field one(f 1) from file2 NOT file1.
like here the script should pick up 3333 and 2222 ( 2nd and 3rd positions values of file2) from file2 and redirect it to another to another third file.
You could read in the files simultaneously by opening them in different file descriptors, then use "read" to read from these descriptors.
exec 3</path/to/file1
exec 4</path/to/file2
typeset chBuffer1=""
typeset chBuffer2=""
while read -u3 chBuffer1 ; do
read -u4 chBuffer2
if [ <put your tests using chBuffer1 and chBuffer2 here> ] ; then
....
else
....
fi
done
exec 3<&-
exec 4<&-
#!/usr/bin/env sh
# @(#) s2 Demonstrate perl line reading, comparison.
set -o nounset
echo
debug=":"
debug="echo"
## Use local command version for the commands in this demonstration.
echo "(Versions displayed with local utility \"version\")"
version >/dev/null 2>& 1 && version bash perl
echo
echo " Data files data1 data2:"
cat data1
echo
cat data2
echo
echo " perl script output, filtered:"
./p1 data1 data2 |
cut -d"|" -f1
exit 0
which calls a perl script:
#!/usr/bin/perl
# @(#) p1 Demonstrate display of line-by-line differences.
use warnings;
use strict;
my ($debug);
$debug = 1;
$debug = 0;
my ( $f1, $f2, $file1, $file2, $line1, $line2, $EOF1, $EOF2 );
my ($lines) = 0;
$EOF1 = $EOF2 = 0;
$file1 = shift || die " usage: $0 file_1 file2\n";
$file2 = shift || die " usage: $0 file_1 file2\n";
open( $f1, "<", $file1 ) || die " Cannot open file $f1\n";
open( $f2, "<", $file2 ) || die " Cannot open file $f2\n";
while (1) {
if ( not( $line1 = <$f1> ) ) {
$EOF1 = 1;
}
if ( not( $line2 = <$f2> ) ) {
$EOF2 = 1;
}
if ( $EOF1 + $EOF2 != 0 ) {
print STDERR " Checking EOF on both files.\n" if $debug;
if ( $EOF1 == 0 ) {
print STDERR " Note - file $file1 has extra lines, stopping.\n";
}
if ( $EOF2 == 0 ) {
print STDERR " Note - file $file2 has extra lines, stopping.\n";
}
# In any case, this is our last read
last;
}
$lines++;
if ( index( $line1, $line2 ) != 0 ) {
print $line2;
}
}
print STDERR " ( Complete pairs of lines read: $lines )\n";
exit(0);
Producing from your data on files data1 and data2:
% ./s2
(Versions displayed with local utility "version")
GNU bash 2.05b.0
perl 5.8.4
Data files data1 data2:
1111 | universe
2222 |good
3333 |good
4444 |good
1111 | universe
3333 |universe
2222 |good
4444 |good
perl script output, filtered:
( Complete pairs of lines read: 4 )
3333
2222
My impression is that this is a row-to-row comparison, so that if you run out of rows (lines) on either file prematurely, you just stop. It's not like diff, where one seeks back and forth vertically to see what was added, deleted, or changed.
Do you see it differently? [pun intended] ... cheers, drl
I'm just trying to verify that the original poster has thought the problem through. Your code will work correctly for the stated problem. But, if there are additions or deletions along with his updates, and the additions are not at the end of the file2, then he's back to a manual process of trying to figure out what changed between the files. In most situations, there is a key field to match on. But that doesn't seem to be the case here.
hey DRL thanks a ton!! for your help ..
but again as per the requirement of boss it has to be done using Unix shell script ( plsql if required)
the files file1 and file2 won't necessarily have the same number of rows .
file1 and file2 have column values of a specific oracle table exported and stored in Unix system on regular basis .
the idea is two compare the contents of the files of previous day and current day ( for e.g file1 is previous day's file and file2 is current day's file) i.e. we got to do row wise comparison of file and detect if there is change in any particular row value if any particular row in file2 (say for e.g row number 3 ) differs from row number 3 of file1 ... then the script should pick up the row num 3 value from file 2 and re-direct it to another third file.
but since we got to verify previous day's and current day's records .. so even if both the files do not have same number of lines ... it won't be a problem as our target is to compare whether a particular row which had some value until yesterday still has the same value today or has it been changed.. in such case the script can compare until it reaches the row end of file1 .
hope i m clear with my question
well i came across options ( thanks to this forum) but following code i tried
comp1=$(cat file1 | cut -d"|" -f1,2)
comp2=$(cat file2 | cut -d"|" -f1,2)
for str in ${comp2[]}
do
count=0
while (( $count < ${#comp1[]} ))
do
if [[ $str != ${comp1[count]} ]]
then
