Need 'expect' help, ssh/telnet and trapping

So here is what I am trying to do. I have a large # of switches and routers I am trying to log into. Unfortunately some have ssh only, some have telnet only. and some i have never logged into with ssh. I first want it to SSH, if i have never logged into the box it will ask for adding the ssh key. I want to tell it yes. If i get connection refused. i want it to go and spawn a telnet session.
then run my commands.

Here is what i have that is definately not working.

#!/usr/bin/expect -f

set timeout 3
set verbose 0

log_file /home/username/portsfound

spawn ssh username@[lindex $argv 0]
   
if expect { "connecting (yes/no)?" {send "yes\r"} }
                        
if expect { "assword: " {send "password\r"} }

else expect { "*refused*" { spawn telnet [lindex $argv 0]
                          log_user 0
                          expect "ogin: "
                          send "username\r"
                          expect "assword: "
                          send "password\r"
                          expect "# "
                          log_user 1 }
						 
 log_user 1
   send "show clock\r"
   expect "# "
 log_user 0
   send -- "exit\r"