script to monitor if ssh works.

Hi All,
I have a setup of around 100 servers with atleast 10 users on each box.The public key from one server has been created and updated on all other servers , so that passwordless login can be done from any use.

We recently had a problem that ssh keys on one of the system was changed and scripts using ssh failed.

I want to come up with a bash script to check if ssh is successful from one server to all servers using all users (All permutations and combinations)

Any ideas would be appreciated.

To cover all combinations, you could try using two loops,

the outer one will loop through the servers available(list of host names) and inner one will loop through the users available in the current server.

And execute simple ssh command for the user@host_name combination and check if it returns success, if yes then ssh connectivity is ok. otherwise report the combination tried in error log.

some thing like ssh_test.sh, copy this script to all your server or may be in a shared file system that can be accessed by all servers.

    for TO in server_list
    do
       for USER in current_server_user_list
       do
            # try ssh  $USER@$TO /shared_filesystem/ssh_test.sh & - as background process, and wait for a while to return
            # if that doesn't return (it may wait for password as the public has been changed)
            # then kill that process and make log to stdout that 
            # $HOSTNAME -> $USER@$TO Failed
            # if that process returns make a success log to stdout
            # in this case we would be having the remote server script's stdout log in this process.
            # $HOSTNAME -> $USER@$TO Success
          done
    done

By this we can collect the log message that was created by remote server in a single server that has started this script first.

I am not sure whether this would work or not.

But this is what i would have tried.

All the best.

Thanks! This is a good idea. I was also planning to use ssh -q so that everything happens in silent mode.