Check connectivity with multiple hosts - BASH script available here

Hi everyone!

Some time ago, I had to check connectivity with a big list of hosts, using different formats (protocol://server:port/path/, server:port, ....).
I developed a script that checks the connectivity using different commands (ping, telnet, nc, curl).
It worked for me so I'm sharing it now, hope you find it useful

Here's the code for connect.sh (v1.7 update: some fixes added, suggested by Scrutinizer):

#!/bin/bash

#SCRIPT FOR TESTING CONNECTIVITY WITH A LIST OF URLs by Fr3dY
#
#1.7: Misc. fixes suggested by 'Scrutinizer' (thanks!) at
#     http://www.unix.com/shell-programming-and-scripting/255925-check-connectivity-multiple-hosts-bash-script-available-here.html
#     IFS backup and restore
#     Modified curl parameters from array to string
#     Modified temp file location to /tmp
#     Fixed ignored errors detection
#
#1.6: Added support for PING
#
#1.5: Added support for NC
#
#1.4: Added error messages when using TELNET
#     Added error codes to be ignored when using CURL
#
#1.3: When using CURL, it shows a OK or ERROR message
#     The URL parser also admits the host:port format (and even host without port, using a default one)
#     Included the script version in the app messages
#     Misc. fixes
#
#1.2: Modified TELNET usage, no CTRL-C capture anymore (it connects automatically now)
#     Misc. fixes
#
#1.1: Added a URL text file as input parameter
#     Added CURL as alternate method
#     Added a parser to use complete connection strings (they're converted automatically when using TELNET)
#     Misc. fixes
#
#1.0: Initial version
#

#Version
version=1.7

#Number of parameters
numberParameters=$#

#Command for testing URLs (right now it accepts 'telnet' and 'curl')
command=$1

#URL file to load
file=$2

#List of errors to be ignored when using CURL
ignoredErrors=(52 403 404 500)

#Iterator
i=0

#PID of the child process to be killed when using CTRL-C (obsolete)
pid=999999999999

#Connection timeout
timeout=3

#Default Port if not specified in the host:port format
defaultPort=80

#CURL parameters
curlParameters="--connect-timeout $timeout --insecure -S -s -f -o /dev/null"

#List of host and ports to check (now they're loaded as a file)
#HOSTS_LIST is defined in f_checkCall, after verifying the syntax

#List size
#LIST_SIZE is defined in f_checkCall, after verifying the syntax

#Temporal file
randomNumber=$RANDOM
tempfile=/tmp/$randomNumber.temp

#Variable init
host=0
port=0
hostport=0
proto=http
IFSbackup=$IFS

#Function that process the list of hosts and execute the connectivity check (telnet, curl, ...)
#PID of each background process is kept, to kill it if remains active
f_processList () {
  if [ ${i} -lt ${LIST_SIZE} ]
  then
    #Format list if TELNET is used (sintax is 'telnet host port' and not 'telnet host:port')
    #Any other conversion functions could be added if necessary
    if [ $command = "telnet" ]
    then
      f_convertFormat
      f_executeTelnet
      f_killTimeout
    elif [ $command = "nc" ]
    then
      f_convertFormat
      f_executeNC
    elif [ $command = "ping" ]
    then
      f_convertFormat
      f_executePING
    else
      f_restoreIFS
      f_executeCurl
    fi
    i=$(($i + 1))
    f_processList
  fi
}

#Function to convert the URL format to HOST PORT (for using TELNET, NC...)
#Admits 'host:port' and 'protocol://host:port/path' --> they're converted to 'host port')
f_convertFormat () {

  COMPLETE_URL=${HOSTS_LIST[$i]}

  if [[ $COMPLETE_URL == */* ]]
  then
    format=url;
  else
    format=hostport;
  fi

  if [ $format = "url" ]
  then
    #Obtain the protocol
    proto="`echo $COMPLETE_URL | grep '://' | sed -e's,^\(.*://\).*,\1,g'`"
    #Get the URL after removing the protocol
    url=`echo $COMPLETE_URL | sed -e s,$proto,,g`
    #Extract user and password (if any)
    userpass="`echo $url | grep @ | cut -d@ -f1`"
    pass=`echo $userpass | grep : | cut -d: -f2`
    if [ -n "$pass" ]; then
        user=`echo $userpass | grep : | cut -d: -f1`
    else
        user=$userpass
    fi
    #Get the host. If no port is defined, the default one is used (attending to the protocol)
    hostport=`echo $url | sed -e s,$userpass@,,g | cut -d/ -f1`
    port=`echo $hostport | grep : | cut -d: -f2`
    if [ -n "$port" ]; then
        host=`echo $hostport | grep : | cut -d: -f1`
    else
        if [ $proto = "http://" ]
        then
           port=80
        elif [ $proto = "https://" ]
        then
           port=443
        elif [ $proto = "ftp://" ]
        then
           port=21
        elif [ $proto = "ftps://" ]
        then
           port=22
        fi
        host=$hostport
        hostport=$host:$port
    fi
  else
    host=`echo $COMPLETE_URL|cut -d ":" -f1`
    port=`echo $COMPLETE_URL|cut -d ":" -f2`
    #If no port is defined, the default one is used
    if [ $host = $port ]
    then
      port=$defaultPort
    fi
  fi
  #The $host and $port variables are set for their incoming usage by using TELNET
}

#Function that executes TELNET on the selected host
f_executeTelnet () {
  CURRENTHOST="$host $port"
  echo "Checking connection with $host:$port"
  $command $host $port > $tempfile 2>/dev/null &
  #Keep the child process PID, to stop it later if it remains active
  pid=$!
  sleep $timeout
  output=`tail -1 $tempfile`
  if [ -z $output ] || [ $output = "" ]
  then
    echo "ERROR WHEN CONNECTING WITH $CURRENTHOST !!!!!!!!"
  else
    if [ $output = "Escape character is '^]'." ]
    then
      echo "CONNECTION OK"
    else
      echo "ERROR WHEN CONNECTING WITH $CURRENTHOST !!!!!!!!"
    fi
  fi
  echo
  echo
}

#Function that executes NC on the selected host
f_executeNC () {
  CURRENTHOST="$host $port"
  echo "Checking connection with $host:$port"
  $command -v -w ${timeout} -z $host $port
  status=$?
  if [ $status = 0 ]
  then
    echo "CONNECTION OK"
  else
    echo "ERROR WHEN CONNECTING WITH $CURRENTHOST"
  fi
  echo
  echo
}

#Function that executes PING on the selected host
f_executePING () {
  CURRENTHOST="$host"
  echo "Checking connection with $host"
  $command -c 1 -W ${timeout} $host >/dev/null 2>/dev/null
  status=$?
  if [ $status = 0 ]
  then
    echo "CONNECTION OK"
  else
    echo "ERROR WHEN CONNECTING WITH $CURRENTHOST"
  fi
  echo
  echo
}

#Function that stops the TELNET processes that have been launched in background and are still active (didn't connect)
f_killTimeout () {
  #Check the process is still active and it's a 'telnet'
  process=`ps -ef|grep $pid|awk '{print $8}'|grep -v grep`
  if [ ! -z $process ]
  then
    if [ "$process" = "telnet" ]
    then
      #Remove this PID from BASH control before killing it, to avoid the 'Killed...' message
      disown $pid
      kill $pid
    fi
  fi
}

#Check if element $1 exists in array $2
f_contains () {
  for elem in "${@:2}"; do [[ "$elem" = "$1" ]] && return 0; done; return 1;
}

#Function that execute CURL on the selected host
f_executeCurl () {
  CURRENTHOST=${HOSTS_LIST[$i]}
  echo "Checking connection with $CURRENTHOST"
  $command ${curlParameters} $CURRENTHOST
  status=$?
  if [ $status = 0 ]
  then
    echo "CONNECTION OK"
  else
    #if [[ "${ignoredErrors[@]}" =~ "$status" ]]
    f_contains $status ${ignoredErrors[@]}
    contain=$?
    if [ $contain -eq 0 ]
    then
      echo "CONNECTION OK (ERROR $status IGNORED)"
    else
      echo "ERROR CONNECTING WITH $CURRENTHOST, ERROR CODE: $status !!!!!!!!"
    fi
  fi
  echo
  echo
}

#Delete temp files
f_deleteTemp () {
  rm -f $tempfile
}

#Check correct syntax
f_callCheck () {
  if [ ! $numberParameters -eq 2 ]
  then
    echo "*** Script for Connectivity Testing, version $version ***"
    echo ""
    echo "Syntax:"
    echo "$0 <command> <file>"
    echo ""
    echo "IMPORTANT: The file must be a complete URL list, each line must be like any of these:"
    echo "protocol://host/path"
    echo "protocol://host:port/path"
    echo "host:port"
    echo "host (when no port is defined, a default one will be used)"
    echo ""
    echo "Accepted commands are: telnet , curl, nc, ping"
    echo ""
    exit 0;
  fi

  if [ ! -f "$file" ]
  then
    echo "ERROR: Can't find the specified file: $file"
    exit 1;
  else
    IFS=$'\r\n' HOSTS_LIST=($(cat $file))
    LIST_SIZE=${#HOSTS_LIST[@]}
  fi

  if [ ! $command = "telnet" ] && [ ! $command = "curl" ] && [ ! $command = "nc" ] && [ ! $command = "ping" ]
  then
    echo "ERROR: Command not supported: $command"
    exit 1;
  fi

}

#Initial message
f_initialMessage () {
  echo
  echo "####################################################"
  echo "###                                              ###"
  echo "### Connectivity Tester, version $version             ###"
  if [ $command = "telnet" ]
  then
    echo "### Using TELNET for connection testing          ###"
  elif [ $command = "curl" ]
  then
    echo "### Using CURL for connection testing            ###"
  elif [ $command = "ping" ]
  then
    echo "### Using PING for connection testing            ###"
  else
    echo "### Using NC for connection testing              ###"
  fi
  echo "###                                              ###"
  echo "####################################################"
  echo
  echo
}

f_restoreIFS () {
  IFS=${IFSbackup}
}

################
## MAIN BLOCK ##
################
f_callCheck
f_initialMessage
f_processList
f_deleteTemp
f_restoreIFS

I have found a simple alternative to your telnet test:

timeout(){
to=$1; shift
perl -e "alarm $to; exec @ARGV" "$@"
}
f_telnet(){
# pass the terminator char to telnet, join its stderr to stdout, and grep for "Connection closed"
printf "%d\n" 0x1d | timeout 3 telnet $1 $2 2>&1 | grep "^Connection .*closed" >/dev/null
}

It returns 0 (the status of the grep) if localhost listens on port 25.
For example

if f_telnet localhost 25; then echo positive; else echo negative; fi

I'll take a look later, but I think I used the terminal output to experiment with it

Thanks!

Thanks for the script. Here are a couple of observations:

IFS=$'\r\n' HOSTS_LIST=($(cat $file))

Note: this permanently changes IFS throughout the script, not just local to the arrays assignment.

if [[ "${ignoredErrors[@]}" =~ "$status" ]]

This does not do what you want, check what happens when status equals 2 vs. when it equals 3.

curlParameters=(--connect-timeout $timeout --insecure -S -s -f -o /dev/null)

Why not just use a simple variable with a string?

curlParameters="--connect-timeout $timeout --insecure -S -s -f -o /dev/null"

tempfile=$randomNumber.temp

This means that in order for the script to run, it has to be able to write in the current directory. The script does not check if it can write to the file...

f_processList does not loop throught the process, but calls itself recursively, which is not efficient, while using global variables which is not good practise..

Consider using case statements rather than if then elif then elif constructs...

Thanks for the advice! I'll implement your suggestions ASAP

Hi,

I've implemented some of your suggestions, but have a couple doubts:

if [[ "${ignoredErrors[@]}" =~ "$status" ]]

What's the problem with that? I just check if the returned status exists in the ignoredErrors array

curlParameters=(--connect-timeout $timeout --insecure -S -s -f -o /dev/null)

Replacing ( ) by " " makes curl think the whole thing is a single parameter. I couldn't make it work that way (does it really matter?)

Thanks!

It does, but it does more than that. If status equals 2 for example, the result is also TRUE and it should not be..

Yes you would need to call it unquoted, just like you did when you called it as an array, so $curlParameters instead of $curlParameters[@]

But that would only be true if error code 2 was added to the ignore list, right?

Tried it but couldn't make it work that way, still take the whole thing as a single option (I think that's why I used the array mode)

No. Just try it out..

$ ignoredErrors=(52 403 404 500)
$ for status in {0..10}; do printf "var status: $status "; if [[ "${ignoredErrors[@]}" =~ "$status" ]]; then echo yes; else echo no; fi; done
var status: 0 yes
var status: 1 no
var status: 2 yes
var status: 3 yes
var status: 4 yes
var status: 5 yes
var status: 6 no
var status: 7 no
var status: 8 no
var status: 9 no
var status: 10 no

No it does not take it as a single option. It only does so if you put double quotes around it..

$ curlParameters=(--connect-timeout $timeout --insecure -S -s -f -o /dev/null)
$ printf "%s\n" ${curlParameters[@]}
--connect-timeout
--insecure
-S
-s
-f
-o
/dev/null
$ curlParameters="--connect-timeout $timeout --insecure -S -s -f -o /dev/null"
$ printf "%s\n" ${curlParameters}
--connect-timeout
--insecure
-S
-s
-f
-o
/dev/null

--edit--
But look what happens when you globally change IFS to $'\r\n' like you do in your script:

$ IFS=$'\r\n'
$ curlParameters="--connect-timeout $timeout --insecure -S -s -f -o /dev/null"
$ printf "%s\n" ${curlParameters}
--co

ect-timeout  --i
secu
e -S -s -f -o /dev/
ull

$ printf "%s\n" ${curlParameters[@]}
--co

ect-timeout
--i
secu
e
-S
-s
-f
-o
/dev/
ull
$ printf "%s\n" "${curlParameters[@]}"
--connect-timeout
--insecure
-S
-s
-f
-o
/dev/null

Thanks for your help, I've updated the first post with some fixes (more to come).