Strange exit of while loop

This code is used to check for duplicate ip and hostnames in an /etc/hosts file

CENTRAL is path to /etc/hosts
AWK =awk

 #check CENTRAL for duplicate ips or hostnames#
        grep -v "^#" $CENTRAL | $AWK '{ print $1, $2; }' | \
        while read ip hostname
        do
                if [ "$ip" != "" -a "$hostname" != "" ]
                then
                echo $ip $hostname
                        ret=`grep -v "^#" | grep -w "$ip" $CENTRAL | wc -l`
                        ret1=`grep -v "^#" | grep -w "$hostname" $CENTRAL | wc -l`
                        if [[ $ret > 1 ]]
                        then
                                echo "$ME:Duplicate IP in hosts.central: $ip $hostname"
                                error=true
                        elif [[ $ret1 > 1 ]]
                        then
                                echo "$ME:Duplicate hostname in hosts.central:$ip $hostname"
                                error=true
                        fi
                fi
        done

The result from this code is that it only iterates through 1 line

output:
127.0.0.1 localhost

strange enough when I comment out

ret=`grep -v "^#" | grep -w "$ip" $CENTRAL | wc -l`
ret1=`grep -v "^#" | grep -w "$hostname" $CENTRAL | wc -l`

it gives me my intended ouput

127.0.0.1 localhost
192.168.2.1 myip
192.168.2.3 myip

Note the $CENTRAL file contains the following entries

127.0.0.1 localhost
192.168.2.1 myip
192.168.2.3 myip

Any thoughts :wall: Thanks

---------- Post updated at 11:07 AM ---------- Previous update was at 10:55 AM ----------

I believe I just found my issue..
my grep form is off..

ret=`grep -v "^#" $CENTRAL  | grep -w "$ip"  | wc -l`
ret1=`grep -v "^#" $CENTRAL  | grep -w "$hostname" | wc -l`

how about doing it all inside of awk...

awk '$0 !~ "^#" {
  ip[$1]++
  h[$2]++
} END {
  for(i in ip) if(ip>1) print "dup ips "i
  for (j in h) if (h[j]>1) print "dup host "j
}' /etc/hosts

Do you need doing it through awk would make it faster? The files this will be in production with will have vary large files to go through

Dont how by how much precisely but it will be faster as your shellcode uses a lot more programs grep awk etc. to do the same thing which one instance of awk can do...mainly because forking processes is an expensive operation. How big do you expect your /etc/hosts file to be...dont you use DNS at your site.

There will be around 1500-2000 lines. Yeah but DNS is run by the Windows team......heaven forbid there be a zone transfer to a UNIX/Linux box

---------- Post updated at 02:32 PM ---------- Previous update was at 01:14 PM ----------

hey shamrock if i wanted to use your code from above and exclude blank lines as well as lines with # at the beginning how would i add that functionality?

---------- Post updated 07-08-11 at 08:03 AM ---------- Previous update was 07-07-11 at 02:32 PM ----------

For all those interested I ended up going with a code that looks like this:

I added the egrep to filter out any blank or commented lines. This way is so much faster than my original post. Thanks shamrock.

#check CENTRAL for duplicate ips or hostnames#
        centcheck=`egrep -v '^#|^$' $CENTRAL | $AWK -v me=$ME '{
                ip[$1]++
                h[$2]++
                } END {
                for(i in ip) if(ip>1) print me ":Duplicate IP in hosts.central " i
                for(j in h) if(h[j]>1) print me ":Duplicate hostname in hosts.central " j
                }'`
        if [ "$centcheck" != "" ]
        then
                echo "$centcheck"
                error=true
        fi

Throw away the egrep portion of your code and use the one below...

awk '$0 !~ "(^#|^$)" {
  ip[$1]++
  h[$2]++
} END {
  for(i in ip) if(ip>1) print "dup ips "i
  for (j in h) if (h[j]>1) print "dup host "j
}' /etc/hosts

$0 !~ "(^#|^$)" tells awk to discard blank lines ^$ and those that begin with a comment ^#.