Bash Script to check Network Cards setup & install on Linux / RHEL

Hi All,

First time poster and bit of a newbie to boot so sorry if this is the wrong section for this.

I am currently writing a bash script for Linux / RHEL systems to check the status of each NIC the server has. The idea is to test the following for each NIC:

1. IP ping test
2. Gateway ping test
3. Reverse lookup test
4. Forward lookup test
5. Routing table check

I have written script to achieve this but I am looking to improve and possibly add more checks. So far this is my code:

#! /bin/bash

LOGFILE=/tmp/NIC-check.`date '+%Y-%m-%d-%H.%M'`.log

echo "Running Network Checks, pease wait..."

echo "" >> ${LOGFILE}

# > IP ping test

ip a | grep inet | grep -v 127.0.0.* | awk '{print $2}' | awk -F/ '{print $1}' >> /tmp/tempfile

while read line; do

echo "" >> ${LOGFILE}
echo "================================" >> ${LOGFILE}
echo "" >> ${LOGFILE}
echo ">>> PING TEST for $line" >> ${LOGFILE}
ping -c 1 "$line" > /dev/null
        if [ $? -eq 0 ]; then
        echo "NIC $line is UP"
        else
        echo "ALERT: NIC $line is NOT PINGING"
        fi
echo "" >> ${LOGFILE}

#  > Gateway ping test

echo ">>> GATEWAY PING for $line" >> ${LOGFILE}
GATEWAY=`echo $line | awk -F'.' -vOFS='.' '{$NF=1}1;'`
ping -b -c 1 "$GATEWAY" > /dev/null
        if [ $? -eq 0 ]; then
        echo "GATEWAY for $line is UP"
        else
        echo "ALERT: GATEWAY for $line is NOT PINGING"
        fi
echo "" >> ${LOGFILE}

#  > Reverse lookup test

echo ">>> REVERSE NSLOOKUP for $line" >> ${LOGFILE}
NSLOOKUP=`nslookup $line | grep "name =" | awk '{print $4}'`
nslookup $line > /dev/null
        if [ $? -eq 0 ]; then
        echo "IP $line is assigned to $NSLOOKUP"
        else
        echo "ALERT: REVERSE LOOKUP for $line is NOT WORKING"
        fi
echo "" >> ${LOGFILE}

#  > Forward lookup test

echo ">>> FORWARD NSLOOKUP for $line" >> ${LOGFILE}
NSLOOKUP=`nslookup $line | grep "name =" | awk '{print $4}'`
nslookup $NSLOOKUP | grep "Name:" > /dev/null
        if [ $? -eq 0 ]; then
        echo "FQDN $NSLOOKUP is assigned to $line"
        else
        echo "ALERT: FORWARD LOOKUP for $line is NOT WORKING"
        fi
echo "" >> ${LOGFILE}

done < /tmp/tempfile >> ${LOGFILE}

echo "...Done!"
echo "Running Routing Table lookup Check..."

#  > Routing table print out

echo "================================" >> ${LOGFILE}
echo "" >> ${LOGFILE}

echo ">>> ROUTING TABLE" >> ${LOGFILE}
netstat -nr | tee -a >> ${LOGFILE}

echo "" >> ${LOGFILE}
echo "================================" >> ${LOGFILE}

rm -rf /tmp/tempfile
echo "" >> ${LOGFILE}

echo "...Done!"

The above code works ok, see sample output below:

================================

>>> PING TEST for 192.168.1.104
NIC 192.168.1.104 is UP

>>> GATEWAY PING for 192.168.1.104
GATEWAY for 192.168.1.104 is UP

>>> REVERSE NSLOOKUP for 192.168.1.104
IP 192.168.1.104 is assigned to rhel-7-test.net

>>> FORWARD NSLOOKUP for 192.168.1.104
FQDN rhel-7-test.net is assigned to 192.168.1.104

================================

>>> ROUTING TABLE
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG        0 0          0 ens33
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 ens33

================================

I am looking for help in the following areas:

Gateway ping test - at the moment the script takes the primary IP and uses AWK to swap out the last characters after the full stop and replace them with a '1' - so if my IP was 192.168.0.999, it will swap .999 for .1
This is ok on most machines but runs into trouble when the gateway IP is different to the NICs IP. What i am looking for is for a way to identify the gateway for each NIC individually and then run a ping test to see if it is up. The above was run on a machine with only 1 NIC but i need to be able to run this script on servers with multiple NICs.

Routing table - at the moment i am only checking the kernel routing table, is there a way i can check the routing for each NIC individually?

I am also hoping for help improving the code, so is there a better approach for what i am trying to achieve? Should i use function command blocks instead of what i have used?

And finally, the purpose of this is to check the Network Cards have been setup and installed correctly for each server, do you guys know of any more checks i can add in to test this?

Sorry for the long essay!

Any help or advice will be appreciated

Thanks in advance!

Hi,

Welcome to the forum ! Hope you enjoy your time here.

Turning first to your question of how to determine the gateway: your best bet there to just to quite literally do exactly that. In other words, capture the current default gateway of the host. You could do this with a sequence of commands like:

/usr/sbin/ip route show | /bin/awk '$1 == "default" {print $3}'

This would take the output of ip route show, search for a line whose first field matches the pattern 'default', and then prints the third field of that line - which would be the default gateway. You could assign the output of the above to a variable, and then use that however you needed to in your script. This removes any and all guesswork about what the default gateway might be, since you'd be looking at what it actually is. This would fail on hosts with no default gateway or multiple default gateways, but for a "normal" host with one single default gateway, this approach should do the trick.

Looking next at how to get the routing table for each NIC: again, the best way to do this is to just, well, look at the routing table for each NIC. This can be done once more with a combination of either netstat or ip, and something like awk. In the example below I've gone with netstat, since that's what you've already used elsewhere in the script:

for nic in `/bin/netstat -rn | /usr/bin/sed 1,2d | /bin/awk '{print $NF}' | /bin/sort | /bin/uniq`
do
       echo Routing table for "$nic":
       /bin/netstat -rn | /bin/awk -v nic=$nic '$NF == nic {print $0}'
       echo --------------------
done

This should show you the routing table for each specific NIC that has a routing table entry on your system.

Anyway, hope this helps to get you started ! I'm sure others will chime in if they have other solutions or more information to share, but for now this will hopefully help you with those two problem areas at least. If anything about the solutions I've given above doesn't quite work for you, then if you let us know what isn't right, we can take things from there.

Hi @r34per,

here is a more compact, but not bullet proof version of your script. Just adapt it to your needs or pick the things that you want to use. Note that there may be other routing tables besides the standard local, main and default. And also not all addresses have to be pingable, this could be forbidden by firewall rules.

#! /bin/bash

LOGFILE=/tmp/NIC-check.$(date '+%Y-%m-%d-%H.%M').log

log() { echo "$*" >> $LOGFILE; }
err() { log "$*"; echo "$*" >&2; }

log "BEGIN $(date '+%F %T')"

read gw nic < <(ip r | awk '/default/ {print $3,$5}')
ping -c1 -w1 $gw &>/dev/null && log "OK GW $gw/$nic" || err "ERR GW $gw/$nic"
while read addr nic; do
    addr=${addr%/*}
    ping -c1 -w1 $addr &>/dev/null && log "OK $addr/$nic" || err "ERR $addr/$nic"
    rdns=$(host -t PTR $addr | awk '{print $5}')
    if [[ $rdns =~ NXDOMAIN ]]; then
        # this doesn't necessarily have to be an error,
        #   could also log "INFO NO RDNS $addr"
        err "ERR RDNS $addr"
    else
        log "OK RDNS $addr/$rdns"
        dns=$(host $rdns | awk '/has address/ {print $4}')
        [[ $dns != $addr ]] && log "INFO RDNS of $addr differs for $rdns: $dns"
    fi
done < <(ip -oneline -f inet a | awk '{print $4,$2}' | grep -v 127.0.0.1/8)

log "ROUTING TABLES"
while read tbl; do
    log "- TABLE $tbl"
    routes=$(ip r l table $tbl | grep -v kernel)
    [[ $routes ]] && log "$routes" || log "EMPTY"
done < <(ip rule | awk '{print $NF}')

log "END $(date '+%F %T')"

Each >> to a file is an open+append+write+close.
Less I/O stress is

LOGFILE=/tmp/NIC-check.$(date '+%Y-%m-%d-%H.%M').log
exec 3>> $LOGFILE # open+append
log() { echo "$*" >&3; } # write

Now every call of log is a simple write.
When the script ends the file is automatically closed. You can explicitely close it with

exec 3>&-

(Necessary if you want to read from the file within the script.)

Good point, but we write only a few tens of lines per script call in a small log file, so we cannot really speak of I/O stress here. Apart from that, it should be even sufficient to just redirect fd 1/stdout globally.
File descriptors are also particularly useful if you want to write different logging levels to separate files.