UNIX utility to find difference in folder, file and contents of file against a base version

UNIX for Beginners Q & A
1

Hi,
I am trying to find out whether there are any Unix utilities that compares folders, files and contents within the file and provides a comprehensive report. The comparison can be against base version of a folder and file with content. Can you please let me know of such a utility?

Thanks, Sripathi

2

Hi.

I'd start with cmptree , from the following list of possibilities.

cmptree Compare directory trees recursively and report the differences. (what)
Path    : ~/bin/cmptree
Length  : 183 lines
Type    : Bourne-Again shell script, ASCII text executable
Shebang : #!/bin/bash

Possibilities:

Difference, similarity, compare

        1) diff, standard

        2) cmp, standard

        3) dwdiff, cdif (local), word differences

        4) cmptree (local), dirdiff (GUI), compare directory trees
           http://www.drdobbs.com/the-shell-corner-cmptree/199103123 (2016.11)

        5) docdiff, compare by character, word, line, numerous output formats

        6) meld (GUI), diff and merge files

        7) numdiff, my-ndiff (local), snd (local), compare numerically

        8) comm, standard, compare 2 sorted files, 3 column output

        9) comm-split (local), compare like comm, but split output to
           separate files.

        10) multi-comm (local), compare n files, need not be sorted

        11) contrast (local), like comm, needs no sort, symmetric and
        simple differences, intersection, union

        12) diff.pl, perl work-alike for diff
            https://metacpan.org/pod/distribution/PerlPowerTools/bin/diff

        13) multi-diff (local), common lines, multiple files, re-write omitting common lines

        14) psame (local), find similarity between 2 text files

        15) fdupes, rdfind, duff, find duplicate files

        16) dateutils.ddiff, find difference in dates, times

In this environment:

OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.6 (jessie)

Best wishes ... cheers, drl

3

You can also have a look at the diff command. Files only in one directory tree will be marked in Only in ..... messages, so you could select those from all your output perhaps:-

diff dir1 dir2 | egrep -v '^Only in'

An alternate might be to find all files in the directories and get the checksums into a pair of files, which you can then use diff against, something like:-

cd dir1
find . -type f -exec sum {} \; | sort > /tmp/dir1.sums

cd dir2
find . -type f -exec sum {} \; | sort > /tmp/dir2.sums

diff /tmp/dir[12].sums

This will list out different checksums and files in only one of the directories. It depends what you want.

Do either of these help?

Robin

1 Like
4

Hi.

See post #9 at thread find same size file for demonstrations of fdupes and rdfind .

Best wishes ... cheers, drl

1 Like
5

@drl, I tried following the links to cmptree, but it seems that it is no longer available. Can you post a listing?

6

It is no longer required by the standards, but many systems still have a dircmp utility that can be used to compare the file hierarchies rooted in two directories looking for added, missing, and common files and (for files that are present in both hierarchies) it can also provide a diff listing for differences between those files.

1 Like
7

Hi, jgt .

For The Shell Corner: cmptree | Dr Dobb's ,about half-way down the page, starts with:

#!/bin/bash
#
# cmptree: compare directory trees recursively and report the differences.
# Author: Ives Aerts
...

Once you get cmptree working, it can be a useful addition to one's toolbox because you can take it with you.

I do recall dircmp now that Don mentioned it. I found a source for dircmp at: Shell dircmp / Code /
[r1]
/trunk/dircmp.sh -- however I have not tried it.

As Robin noted, modern diff can compare directories.

Thanks @Robin @Don, I have modified my diff list accordingly.

Best wishes ... cheers, drl