Shell Script: Looking for Your Help

I appreciate the previous help on this issue; I really still have problems in executing this script remotely over a large scale network. For technical reasons this loop sometimes hang on an IP address from the loop-file "list.10". This can take up to unacceptable range of time, and I wish to force the loop if this thing happen to do next loop value if possible, and repeat the loop the value, at which it had the problem at the end of the file "list.10". This in case if the problem happened only at "loop = i" only, but if if even at "loop = i+1" then I would like to start outer loop "machine" from the beginning. I really look for your assistance.

while read machine; do echo "  "; echo "# Outer loop: $machine"; echo "==========="; 
ssh -n -i /~/~/~/~/~ ~@$machine 'cd iperf; while read loop; do echo "+ Inner loop: $loop"; TRIES=1; 
until ./iperf -c $loop -u -b 6M > $loop.iperf-udp-6 && grep -q "%" $loop.iperf-udp-6; do  if [ "$TRIES" -ge 5 ]; 
then echo "+ Inner loop: $loop [$TRIES]"; break; fi; echo "+ Inner loop: $loop [$TRIES]"; TRIES=`expr $TRIES + 1`; done; done < list.10'; done < list.9

I unwrapped your script and renamed a variable (in blue) to make it a little easier to read:

while read machine; do
  echo "  "
  echo "# Outer loop: $machine"
  echo "==========="

  ssh -n -i /~/~/~/~/~ ~@$machine '
    cd iperf;
    while read client; do
      echo "+ Inner loop: $client";
      TRIES=;
      until ./iperf -c $client -u -b 6M > $client.iperf-udp-6 && grep -q "%" $client.iperf-udp-6; do
        if [ "$TRIES" -ge 5 ]; then
          echo "+ Inner loop: $client [$TRIES]";
          break;
        fi;
        echo "+ Inner loop: $client [$TRIES]";
        TRIES=`expr $TRIES + 1`;
      done;
    done < list.10
  '
done < list.9

Correct me if I am wrong, but are trying to do iperf all the clients listed in list.10 first and then go back and try the ones that failed, instead of iperfing (!) each client five times?

If so, you could try something like:

MAXTRIES=5

while read machine; do
  echo "  "
  echo "# Outer loop: $machine"
  echo "==========="

  ssh -n -i /~/~/~/~/~ ~@$machine '
    cd iperf;
    cp list.10 list.10-try;
    
    TRIES=0;
    while [[ "$TRIES" -lt "$MAXTRIES" ]]; do
      TRIES=`expr $TRIES + 1`;

      mv list.10-try list.10-scan;

      while read client; do
        echo "+ Inner loop: $client";
        if ! ./iperf -c $client -u -b 6M > $client.iperf-udp-6 && grep -q "%" $client.iperf-udp-6; then
          echo "$client"
        fi;
      done < list.10-scan > list.10-try;
    done

    if [[ -s list.10-try ]]; then
      echo "! Failed:" `cat list.10-try`;
    fi
  '
done < list.9

If I understand you correctly, what you suggest is to iperf all clients in list.10, and at the end repeat any client that has no "%" sign in its file, correct? And for how many times you repeat failed clients? I need it at least 5 if possible.

If so, the original code actually doesn't iperf all clients 5 times, but only ones that fail. What you suggest is just another form from doing the same job BUT better for me than previous THANKS. However, the main problem is when a client or sender hangs on! This what I'm trying to solve "THE LOOP JUST STOP AS I SEE IT REMOTELY". I suggest to cases for this problem:

  • If server "from machine loop" hangs on, STOP the machine loop and repeat from beginning "NOT BREAK".
  • If a client, just move it to the end 100% AS YOU DID.

---------- Post updated at 11:47 AM ---------- Previous update was at 11:24 AM ----------

Also please, can you review your script because there is an error.

Why don't you review the script? If not, can you at least have the courtesy to say what the error is?

Although I added a comma after the second done, the script is just listing all IPs in a line without probing clients!

The script I wrote will try iperf-ing each client up to 5 times - but will not re-iperf a client once it succeeds. Yes, it is more-or-less equivalent to what you wrote, but instead of waiting for a client to timeout five times, my script moves to the next client and retries the failed client later, which is what is happening when you think:

So why don't you just ping the client before you iperf:

        echo "+ Inner loop: $client";

        if ! ping pingopts $client > /dev/null 2>&1; then
          echo "$client"
          continue
        fi;

        if ! ./iperf -c $client -u -b 6M > $client.iperf-udp-6 && grep -q "%" $client.iperf-udp-6; then
          echo "$client:" unable to iperf 1>&2
        fi;

perlopts will depend on your implementation of ping.

What error are you getting?