Comparing Cluster Members

Does anyone know of a good utility that will do a binary compare of all the files on two cluster members? We are looking for something we can run on a monthly batch basis to monitor for configuration drift between AIX/HACMP and Solaris/VCS cluster member pairs, use as an ad hoc diagnostic tool when an app runs fine on one cluster node but not the other, or to compare test and prod systems to figure out why a problem cannot be reproduced in the test environment.

I realize tight system management procedures are the best preventative, but when you have eight sysadmins with root (sudo) access and another dozen people with varying access to app directories, some type of drift is inevitable. We need something that can at least help us prove that the machines are identical so we can direct diagnostic efforts back to the application.

2 things that come to mind:

  • md5sum and diff, both via SSH. If the checksum doesn't match, compare the files using diff.
  • CVS or Subversion as a central configuration repository. The valid configuration is fetched from there automatically at certain intervals, overwriting any local configuration drift.

Tripwire or similar.