Expect script help needed- script failing if router unavailable

Hey all. Sometimes I'm tasked to change some router configs for the entire network (over 3,000 Cisco routers). Most of the time its a global config parameter so its done with a loop and an IP list as its the same configuration change for all routers. This is working OK.

However, sometimes an expect script must be generated for ALL routers since the config changes are different for each router. The way we approach it, we divide the routers per regional domains (6 in total), and produce 1 expect script per regional domain. The problem I'm having is that when I schedule these type of expect scripts to run via crontab, they usually get hung anytime a router fails to respond (maintenance, outage, etc). How can I make the script go to the next router if a router cannot be reached? Here's a sample line of the 20K lines:

....

#
set timeout 10
spawn telnet 10.156.112.1
match_max 100000
expect Username:
sleep .1
send -s -- "username\r"
expect Password:
sleep .1
send -s --  "passwordstring\r"
expect *
sleep .1
send -s --  "conf t\r"
expect *
sleep .2
send -s -- "no ip flow-export destination 172.17.187.127 9995\r"
expect *
sleep .2
send -s -- "no ip flow-export destination 10.24.180.110 9995\r"
expect *
sleep .2
send -s -- "ip flow-export destination 172.17.187.188 9995\r"
expect *
sleep .2
send -s -- "ip flow-export destination 10.24.180.110 9995\r"
expect *
sleep .2
send -s -- "end\r"
expect *
sleep .4
send -s --  "wr mem\r"
expect *
sleep .2
send -s --  "exit\r"
expect eof
#
#
set timeout 10
spawn telnet 10.156.112.2
match_max 100000
expect Username:
sleep .1
send -s -- "username\r"
expect Password:
sleep .1
send -s --  "passwordstring\r"
expect *
sleep .1
send -s --  "conf t\r"
expect *
sleep .2
send -s -- "no ip flow-export destination 172.17.187.127 9995\r"
expect *
sleep .2
send -s -- "no ip flow-export destination 10.24.180.110 9995\r"
expect *
sleep .2
send -s -- "ip flow-export destination 172.17.187.188 9995\r"
expect *
sleep .2
send -s -- "ip flow-export destination 10.24.180.110 9995\r"
expect *
sleep .2
send -s -- "end\r"
expect *
sleep .4
send -s --  "wr mem\r"
expect *
sleep .2
send -s --  "exit\r"
expect eof
#
#

Thanks!
Marcos

You could:
make this into a file that accepts the IP as a parameter:

if {$argc!=1} {
   send_user "usage: $argv0 ip \n"
   exit
}

# script must be run by root user
set whoami [exec id -u]
if {$whoami!=0} {
   send_user "You must be a root user to run this script\n"
   exit
}
#
set timeout -1
match_max 100000

# stopre ip
set ip [lindex $argv 0]

set timeout 10
spawn telnet $ip
match_max 100000
expect Username:
expect *
...
sleep .2
send -s --  "exit\r"
expect eof

Then write a wrapper for the expect script.

#!/usr/bin/ksh
while read IP
do
   /your/expect/script $IP &
   sleep 30

done

Since it's run in the background the script won't wait for the next router. It does mean that if a router is hung, that individual script will hang.

I'm not sure I understand...that's the problem I'm running into. Because I'm touching over 450 routers per script, and because there are multiple flavors of configuration changes, it is one big long script. If a router is unavailable, it hangs.

My thoughts were that before spawn telnet ran, a ping would make sure that the IP address replied, and if it did it could proceed. If it did not reply, then go to the next spawn telnet, etc.

I'm obviously a router Cisco guy first, not a programmer, so I do apologize if I'm making no sense.

Marcos

With the above solution, you would run one script per router (so 450 scripts). If one router hangs, one script will hang, but 449 will succeed.

What kinda of Unix server do you have? Do you have a shell preference?