Forking a bunch of processes and filling up the process table

Shell Programming and Scripting
1

I have a bash script that has been used for months here at work for doing an SSH into other machines both Linux and Solaris and running a script on the remote machine. Recently I have started to noticed that things are being left being on the maching doing the SSH.

For example....

 tivoli  5701     1   0 01:19:00 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  3572     1   0 01:13:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 29389 29211   0 07:07:32 pts/1       0:00 grep gui
  tivoli  3045     1   0 01:13:31 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
    root  3516     1   0 01:13:53 ?           0:06 /lcl/prd/apps/Tivoli/netcool/guifoundation/asa/unix/dbsrv9 -o /lcl/prd/apps/Tiv
  tivoli  4514     1   0 01:15:01 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  4853     1   0 01:15:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  4123     1   0 01:14:32 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  4723     1   0 01:15:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  5128     1   0 01:16:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  4988     1   0 01:16:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  5428     1   0 01:17:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  5286     1   0 01:17:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  5546     1   0 01:18:30 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  6263     1   0 01:19:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  6066     1   0 01:19:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  6596     1   0 01:20:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  6436     1   0 01:20:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  6734     1   0 01:21:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  6856     1   0 01:21:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  6999     1   0 01:22:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  7125     1   0 01:22:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  7384     1   0 01:23:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  7261     1   0 01:23:30 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  7740     1   0 01:24:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  7946     1   0 01:24:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  8100     1   0 01:25:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  8477     1   0 01:26:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  8217     1   0 01:25:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  8355     1   0 01:26:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  8737     1   0 01:27:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  8609     1   0 01:27:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  8882     1   0 01:28:30 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  9005     1   0 01:28:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  9545     1   0 01:29:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  9311     1   0 01:29:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  9775     1   0 01:30:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli  9890     1   0 01:30:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 10157     1   0 01:31:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 10033     1   0 01:31:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 10422     1   0 01:32:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 10304     1   0 01:32:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 10562     1   0 01:33:30 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 10682     1   0 01:33:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 11260     1   0 01:34:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 11055     1   0 01:34:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 12259     1   0 01:38:30 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 11585     1   0 01:35:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 11432     1   0 01:35:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 12001     1   0 01:37:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 11714     1   0 01:36:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 11846     1   0 01:36:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 12117     1   0 01:37:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 12381     1   0 01:38:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 12921     1   0 01:39:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 12717     1   0 01:39:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 13186     1   0 01:40:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 13065     1   0 01:40:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 13326     1   0 01:41:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 13453     1   0 01:41:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 13592     1   0 01:42:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 13717     1   0 01:42:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 13980     1   0 01:43:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 13865     1   0 01:43:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 14524     1   0 01:44:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 14308     1   0 01:44:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 14748     1   0 01:45:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 14871     1   0 01:45:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 15009     1   0 01:46:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 15139     1   0 01:46:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 15398     1   0 01:47:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 15272     1   0 01:47:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 15536     1   0 01:48:30 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 15661     1   0 01:48:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 16543     1   0 01:50:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 16005     1   0 01:49:30 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 16215     1   0 01:49:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 16385     1   0 01:50:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 17071     1   0 01:52:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 16683     1   0 01:51:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 16806     1   0 01:51:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 16951     1   0 01:52:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 17644     1   0 01:54:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 17330     1   0 01:53:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 17207     1   0 01:53:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 17895     1   0 01:54:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 18159     1   0 01:55:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 18037     1   0 01:55:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 18422     1   0 01:56:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 18298     1   0 01:56:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 18680     1   0 01:57:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 18557     1   0 01:57:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 18948     1   0 01:58:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 18825     1   0 01:58:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 19486     1   0 01:59:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 19275     1   0 01:59:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 20334     1   0 02:00:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 20197     1   0 02:00:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 20478     1   0 02:01:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 20612     1   0 02:01:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 20869     1   0 02:02:59 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 20743     1   0 02:02:29 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh
  tivoli 21006     1   0 02:03:30 ?           0:00 /lcl/bin/ssh penguinshrimp /lcl/apps/Tivoli/omnibus_procedure_scripts/penguinsh

I have no idea why these are being left out here like they are.

Here is the code for it all...

#!/bin/sh
##############################################################################################################################
# This script is fired from the dba_filesystem_guard trigger from within Ominbus on bluezebra.
# The script first pings the target server. If the ping works it then remotes to the system
# with ssh or rsh and then executes the script named (servername)fire.sh on each machine which in return fires specific 
# scripts to monitor the file systems for the DBA's.
#
# Created by Larry Roberts 6/2/2009 
#
# Last update by Larry Roberts 6/5/09
#
##############################################################################################################################
##############################################################################################################################
# For Solaris machines you must use rsh.
# For Linux machines you must use /lcl/bin/ssh. 
#
# In order to prevent password prompting on each server using ssh which will create a failure, the server being connected to
# and the server attempting the connection must be listed in the .shosts file located in the home directory of the user 
# tivoli (/gfs/home/tivoli). Again this MUST be on both macines and only if using ssh for Linux. 
#
##############################################################################################################################
######################################################################################################
# Time check to make sure switch over is not taking place. If it is then this script should die.
# 
# Current switch over is set for between 5:50am and 6:30am
#
######################################################################################################
day=`date`
time=`echo $day | cut -d" " -f4`
hour=`echo $time | cut -d":" -f1`
minute=`echo $time | cut -d":" -f2`
dayname=`echo $day | cut -d" " -f1`
if [ $dayname = 'Sat' ]
then
if [ $dayname = 'Sat' -a $hour = 5 -a $minute -gt 50 ]
then
if [ $dayname = 'Sat' -a $hour = 6 -a $minute -lt 30 ]
then
exit
fi
fi
fi
######################################################################
# Server Delphin / OS: Solaris
######################################################################
x=1
/usr/sbin/ping delphin
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh delphin "/lcl/apps/Tivoli/omnibus_procedure_scripts/delphin_fire.sh" &
fi
######################################################################
# Server Angel / OS: Solaris
######################################################################
x=1
/usr/sbin/ping angel 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh angel "/lcl/apps/Tivoli/omnibus_procedure_scripts/angel_fire.sh" &
fi
######################################################################
# Server Kraken / OS: Solaris
######################################################################
x=1
/usr/sbin/ping kraken 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh kraken "/lcl/apps/Tivoli/omnibus_procedure_scripts/kraken_fire.sh" &
fi
######################################################################
# Server Bluemarron / OS: Linux 
######################################################################
x=1
/usr/sbin/ping bluemarron 
x=$?
echo $x
if [ $x = '0' ]; then
/lcl/bin/ssh bluemarron "/lcl/apps/Tivoli/omnibus_procedure_scripts/bluemarron_fire.sh" &
fi
######################################################################
# Server Penguinshrimp / OS: Linux 
######################################################################
x=1
/usr/sbin/ping penguinshrimp 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh penguinshrimp "/lcl/apps/Tivoli/omnibus_procedure_scripts/penguinshrimp_fire.sh" &
fi
######################################################################
# Server Leviathan / OS: Solaris
######################################################################
#x=1
#/usr/sbin/ping leviathan
#x=$?
#if [ $x = '0' ]; then
#/lcl/bin/ssh leviathan "/lcl/apps/Tivoli/omnibus_procedure_scripts/leviathan_fire.sh" &
#fi
######################################################################
# Server Roughgreen / OS: Linux
######################################################################
x=1
/usr/sbin/ping roughgreen 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh roughgreen "/lcl/apps/Tivoli/omnibus_procedure_scripts/roughgreen_fire.sh" &
fi
######################################################################
# Server Moccasin / OS: Linux
######################################################################
x=1
/usr/sbin/ping moccasin 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh moccasin "/lcl/apps/Tivoli/omnibus_procedure_scripts/moccasin_fire.sh" &
fi
######################################################################
# Server Bluegarter / OS: Linux
######################################################################
x=1
/usr/sbin/ping bluegarter 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh bluegarter "/lcl/apps/Tivoli/omnibus_procedure_scripts/bluegarter_fire.sh" &
fi
######################################################################
# Server Eastindigo / OS: Linux
######################################################################
x=1
/usr/sbin/ping eastindigo 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh eastindigo "/lcl/apps/Tivoli/omnibus_procedure_scripts/eastindigo_fire.sh" &
fi
######################################################################
# Server Banded / OS: Linux
######################################################################
x=1
/usr/sbin/ping banded 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh banded "/lcl/apps/Tivoli/omnibus_procedure_scripts/banded_fire.sh" &
fi
######################################################################
# Server Ringneck / OS: Linux
######################################################################
x=1
/usr/sbin/ping ringneck
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh ringneck "/lcl/apps/Tivoli/omnibus_procedure_scripts/ringneck_fire.sh" &
fi
######################################################################
# Server Penribbon / OS: Linux
######################################################################
x=1
/usr/sbin/ping penribbon 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh penribbon "/lcl/apps/Tivoli/omnibus_procedure_scripts/penribbon_fire.sh" &
fi
######################################################################
# Server blackracer / OS: Linux
######################################################################
x=1
/usr/sbin/ping blackracer 
x=$?
if [ $x = '0' ]; then
/lcl/bin/ssh blackracer "/lcl/apps/Tivoli/omnibus_procedure_scripts/blackracer_fire.sh" &
fi

Does anyone see something I am doing wrong as to why these would hang only once in a while? And that is the key word here... Once in a while. Most of the time this works perfectly. I am not sure if this is taking place because it can not reach the remote server or what. If that is the case is there a way to time out on each one or something?

Is this taking place because its waiting for the child to finish and its not? If so how can I just SSH, tell it to run the remote script and move on without waiting?

Thank you for the help

2

First off, it looks like you have problems with the box penguinshrimp since it's the only one being left. Does it ever hang on any other machine?

Next, I'd probably simplify the script by creating two variables, one for solaris boxes and the other for Linux.

linux_boxes="first_box second_box etc"

Then have two loops that iterate through those variables so you don't have to code for each box. Adding/removing a box is as simple as modifying the variable.

for box in $linux_boxes
do
  some stuff with $box
done

Also I'd probably change the test a bit to something like:

if ping -c 3 $box > /dev/null 2>&1
then
  ssh somewhere and do something
else
  echo $box is unpingable
fi

lastly, from getting it beat into my head in high-school programming, add some spacing/formatting in your code so it's a bit easier to read/modify/debug. This is especially true when your code is more complex.

Change this:

if [ $dayname = 'Sat' ]
then
if [ $dayname = 'Sat' -a $hour = 5 -a $minute -gt 50 ]
then
if [ $dayname = 'Sat' -a $hour = 6 -a $minute -lt 30 ]
then
exit
fi
fi
fi

to this:

if [ $dayname = 'Sat' ]
then
  if [ $dayname = 'Sat' -a $hour = 5 -a $minute -gt 50 ]
  then
    if [ $dayname = 'Sat' -a $hour = 6 -a $minute -lt 30 ]
    then
      exit
    fi
  fi
fi