Selection from array

Hi,

I need some help taking a selection from a command and adding part of the output to an array.

I'd like to read the items into the array, have the user chose an option from the array and put the item from column 1 into a variable.

The command is:

awless -l list routetables --columns ID,NAME,VPC --filter VPC=vpc-0025xxxxxx94e52

The output from that command is:

|         ID           |          NAME           |          VPC          |
|-----------------------|-------------------------|-----------------------|
| rtb-0b5c74xxxxxxx | network-public  | vpc-0025xxxxxx94e52 |
| rtb-0d18xxxxxxxxx | network-private | vpc-0025xxxxxd94e52 |
| rtb-0e47xxxxxxxxx |                         | vpc-0025xxxxxx94e52 |

and i want rtb-xxxxx to be in my variable (for now just print the variable is fine)

As you can see one option has no name. I'd like the options to be displayed showing both ID and name:

1. rtb-0b5c74xxxxxxx | network-public
2. rtb-0d18xxxxxxxxx | network-private
3. rtb-0e47xxxxxxxxx | NO-NAME

Thank you for your time.

In general: to get qualified help with SHELL PROGRAMMING problems it usually helps to tell WHICH SHELL you are talking about. Sorry, but my crystal ball is in repair right now and mind reading is not part of the job description.

I suggest you read up on the select -command which many shells offer. It should do exactly what you have in mind. Have a look at the documentation of your shell and if it doesn't have it then report back - and tell us which shell that is.

I hope this helps.

bakunin

Hi Bakunin,

Thanks for the reply. I had a read up on the select command and it does look like something that could solve this.

I'm using bash but my programming skills are poor and can't get the result I'm looking for.

Could you confirm your belief that "it does look like something that could solve this", e.g. by trial and error? That were a possible (if not the usual) approach when encountering something new.

Howsoever, here's a small example how you could proceed. Unfortunately, I had to go through a TMP file, as piping the sed result directly into readarray failed, although its man page says: "Read lines from the standard input into the indexed array variable array"
Try and comment back

awless . . . | sed -r 's/( *)\| */\1/g; s/  / /g; s/[^ ]* *$//; > TMP
<TMP readarray -ts2 TARR
select OPT in "${TARR[@]}"
  do    echo "$REPLY: $REPLY, variable: $OPT, reduced var: ${OPT% * *}"
  done
1) rtb-0b5c74xxxxxxx network-public 
2) rtb-0d18xxxxxxxxx network-private 
3) rtb-0e47xxxxxxxxx 
#? 1
1: 1, variable: rtb-0b5c74xxxxxxx network-public , reduced var: rtb-0b5c74xxxxxxx
#? 2
2: 2, variable: rtb-0d18xxxxxxxxx network-private , reduced var: rtb-0d18xxxxxxxxx
#? 3
3: 3, variable: rtb-0e47xxxxxxxxx , reduced var: rtb-0e47xxxxxxxxx 
#? 

I can't get your script to work.

./test2
./test2: line 2: unexpected EOF while looking for matching `''
./test2: line 7: syntax error: unexpected end of file

#!/bin/bash
awless -l list routetables --columns ID,NAME,VPC  -filter VPC=vpc-00xxxxxxxxxxe52 | sed -r 's/( *)\| */\1/g; s/  / /g; s/[^ ]* *$//; > TMP
<TMP readarray -ts2 TARR
select OPT in "${TARR[@]}"
  do    echo "$REPLY: $REPLY, variable: $OPT, reduced var: ${OPT% * *}"
  done

------ Post updated at 09:43 AM ------

Just in case it helps here is my bad attempt that doesn't work. I thought posting my bad attempt might confuse the situation as I'm trying to find a simpler solution thats easier to replicate for other searches.

#!/bin/bash
q=1
for z in `awless -l list routetables --columns ID,NAME,VPC --filter VPC=vpc-00xxxxxxxxxxxe52  |\
 awk '
 /\| vpc/ && NF == 4 { print $2 ":No_Label" }
 /\| vpc/ && NF == 5 { print $2 ":" $4 }
 '`
 do
 array3[$q]=`echo $z | awk -F: ' { print $2 } '`
 array4[$q]=`echo $z | awk -F: ' { print $1 } '`
 echo "   $q. $z" ; q=$(($q + 1))
 done
 read selection2

 if [[ $selection2 =~ ^[0-9][0-9]*$ ]] ; then
  echo
 else
  echo "That selection was not valid"
  exit
 fi

 if [ ${array3[$selection2]} ] ; then
  arrayvar3="${array3[$selection2]}"
  arrayvar4="${array4[$selection2]}"
  echo "Thank you for your selection. I will use this for my script... $arrayvar3 $arrayvar4"
  echo ""
  echo "Processing....."
 else
  echo "That selection was not valid"
  exit
 fi

First off: thank you for posting your script. It is a lot easier (and perhaps much better for you) to explain to you what you did wrong than to just envision a solution which yu don't understand where it comes from. It will not confuse the situation but in fact help clarify it. We do not want to solve your problem - we want to enable you to solve it on yourself.

Let us first sum up the goal: you want to create a selection menu from the output of a program. The output is roughly a table and each line will be a possible selection, yes?

Now, let us examine the description of the selection -keyword. Here it is (taken from my system, a Linux):

       select name [ in word ] ; do list ; done
              The list of words following in is expanded, generating a list of
              items.   The  set  of  expanded words is printed on the standard
              error, each preceded by a number.  If the in  word  is  omitted,
              the  positional  parameters  are printed (see PARAMETERS below).
              The PS3 prompt is then displayed and a line read from the  stan-
              dard  input.   If the line consists of a number corresponding to
              one of the displayed words, then the value of  name  is  set  to
              that  word.  If the line is empty, the words and prompt are dis-
              played again.  If EOF is read, the command completes.  Any other
              value  read  causes  name  to  be set to null.  The line read is
              saved in the variable REPLY.  The list is  executed  after  each
              selection until a break command is executed.  The exit status of
              select is the exit status of the last command executed in  list,
              or zero if no commands were executed.

So we first have to create a list of words. You already did this by creating the array. Let us set aside your problem for a moment to get us acquainted with the select -keyword. Here it is:

#! /bin/bash

arr[1]="apples"
arr[2]="bears"
arr[3]="peaches"
arr[4]="oranges"

selection=""

select selection in ${arr[@]} ; do
    echo "You selected $selection"
done

echo "here is the end of the program"
exit 0

Try that out on your own system. (Really! Try it! Don't just take my word for it.)

Now what did you find? First, the selection works smoothly. As long as you enter "1", "2", "3" or "4" everything works fine. What happens when you enter "0" or "17" or "blabla"? And how is the selection loop left? You probably figured out that you can use Ctrl-C to exit, but that is not a nice way to leave it. We would rather have something like "Press 1-4 for select a fruit or x to exit", yes?

Also there is this ridiculous prompt: Shouldn't there be something like, "Press..." as i said above, rather than "#?"? Let us start with this. There was this part of the documentation:

So, we have to set the PS3 prompt (shell has 4 different prompts for certain situations. We usually see PS1, which it displays when waiting for the use to enter a command, but here PS3 is used) and we do that right before the start of the selection:

#! /bin/bash

arr[1]="apples"
arr[2]="bears"
arr[3]="peaches"
arr[4]="oranges"

selection=""

PS3="Enter 1-4 to select a fruit or x to exit => "
select selection in ${arr[@]} ; do
    echo "You selected $selection"
done

echo "here is the end of the program"
exit 0

Better, yes? But we should have a reaction of some sort when the user pressed a non-existing item, like "5" or "0" or "blabla". So we have to put some logic into the loop to react to that. Since we might need several cases we do not use if , but a case..esac :

#! /bin/bash

arr[1]="apples"
arr[2]="bears"
arr[3]="peaches"
arr[4]="oranges"

selection=""

PS3="Enter 1-4 to select a fruit or x to exit => "
select selection in ${arr[@]} ; do
     case "$selection" in
          "")
               echo "You selected a non-existing item"
               ;;

          *)
               echo "You selected $selection"
               ;;

     esac
done

echo "here is the end of the program"
exit 0

Why did this work? because the documentation said:

That means, enter 1-4 and the value of "$selection" is set to the first-fourth element of our array. Enter any other value and the value of selection is set to "" (the null string). This is what we checked in the first case-branch, any other value means some "legal" value was entered.

We still did not implement a means of leaving the loop so we tackle this as next: we can simply add a special item to the array which selection causes the loop to be left:

#! /bin/bash

arr[1]="apples"
arr[2]="bears"
arr[3]="peaches"
arr[4]="oranges"
arr[5]="EXIT"

selection=""

PS3="Enter 1-4 to select a fruit or 5 to exit => "
select selection in ${arr[@]} ; do
     case "$selection" in
          "")
               echo "You selected a non-existing item"
               ;;

          "EXIT")
               echo "You are leaving the loop."
               break
               ;;

          *)
               echo "You selected $selection"
               ;;

     esac
done

echo "here is the end of the program"
exit 0

Notice that now, for a change, the line i put in next to the last line "here is the end of the program" was executed for the first time. We really and correctly left the program. It is also time now to make the whole thing a bit more "dynamic": right now, if we add another item, we would have to change not only the array and the number of the "EXIT" item, but also the PS3-prompt. We will make these to adjust automatically:

The number of elements in an array is in a variable expansion: "${#<arrayname>[@]}". So, to automatically add the "EXIT" item as the last item of the array we do:

arr[1]="apples"
arr[2]="bears"
arr[3]="peaches"
arr[4]="oranges"

arr[((${#arr[@]}+1))]="EXIT"

Notice that "((${#arr[@]}+1))" will always evaluate to the number of array elements plus 1. So now, it evaluates to "5", but if i remove the line with the oranges it would evaluate to "4" and if i would add a line it would evaluate to "6". We do something similar to the PS3 prompt:

PS3="Enter 1-$((${#arr[@]}-1)) to select a fruit or ${#arr[@]} to exit => "

Notice that since the EXIT-selection will have a varying index but always the same value (the string "EXIT") we do not need to change the loop itself. So here is our script, were you can remove or add items to the initial array without changing anything else:

#! /bin/bash

arr[1]="apples"
arr[2]="bears"
arr[3]="peaches"
arr[4]="oranges"

arr[((${#arr[@]}+1))]="EXIT"

selection=""

PS3="Enter 1-$((${#arr[@]}-1)) to select a fruit or ${#arr[@]} to exit => "

select selection in ${arr[@]} ; do
     case "$selection" in
          "")
               echo "You selected a non-existing item"
               ;;

          "EXIT")
               echo "You are leaving the loop"
               break
               ;;

          *)
               echo "You selected $selection"
               ;;

     esac
done

echo "here is the end of the program"
exit 0

I will stop here now, because you are sure anxious to try your newfound toy and play around with it. I also suggest that when you write scripts you do the indenting like i do and not like you did. Experience shows that it is easier to discern the program flow and the structure that way.

Have fun and if you still have questions don't hesitate to ask.

I hope this helps.

bakunin

3 Likes

Wow. Thanks so much bakunin. You are certainly right that it is better to "teach a man to fish". I struggled with this for hours before i posted as it was hard finding something on google that was relevant enough to my problem to learn from. So now not only am i one step closer to resolving my problem i have learned a lot more.

I'll have a play and let you know how i get on.

------ Post updated at 05:24 PM ------

Having much more fun now i know how the array works, but I'm struggling to get the row to display as a whole selection. The array is splitting all the individual items into options.

I want to display like:

  1. apples : 1111 : aaaa
  2. pears : 2222 : bbbb
  3. peaches : 3333 : cccc

but its displaying

  1. apples
  2. 1111
  3. aaaa
  4. pears
  5. 2222

etc

Here is my new code:

options=( $(awless -l list routetables --columns ID,NAME,VPC --filter VPC=vpc-0025xxxxxxxe52 | grep rtb | sed 's/|//g') )
selection=""
options[((${#options[@]}+1))]="EXIT"

selection=""

PS3="Enter 1-$((${#options[@]}-1)) to select an item or ${#options[@]} to exit => "

#PS3="Select an item or 5 to exit => "
select selection in ${options[@]} ; do
     case "$selection" in
          "")
               echo "You selected a non-existing item"
               ;;

          "EXIT")
               echo "You are leaving the loop."
               break
               ;;

          *)
               echo "You selected $selection"
               ;;

     esac
done

echo "here is the end of the program"
exit 0

Great! You know, UNIX utilities you use in shell scripting are behaving like a fine-tuned orchestra: it is possible to make them sound awful but let the right conductor enter the stage and they will create true magic. It is fun to write well written programs like it elating to witness or perform any other piece of art.

Now, to your problem:

The problem is in this line and it is something that will certainly cross your path more often in the future. It is an ability (as well as a fundamental behavior) of the shell, which is called field splitting.

Let us start with a simple question. Consider the following command:

# ls -l /some/directory/file.txt

We will certainly agree that ls is a command, -l is an option and /some/directory/file.txt is the filename (and the argument to the command). But why is this so? Wouldn't it be equally possible that -l /some/directory/file.txt is the filename and there are no options? Or that ls -l is the commands name?

In fact it is not: the reason is that the three distinct parts are separated by a blank and therefore constitute three separate "words". The shell takes the first word as the commands name, which is why ls gets invoked. Then it passses the other two words as separate entities to ls and this program then knows which to take as option and which as argument. But the basis for this was that the shell has split the input to three words first.

You may have come across the "IFS". This stands for "internal field separator" and is a variable which contains the character(s) the shell uses to separate fields. Per default this is set to "blank" and ""tab" (the characters, not the strings). It is possible, though, to set these to other values.

You can use this whole mechanism to your advantage, i.e. in a read-statement: read can not only read input and assign one variable but many. Here is a little example program that illustrates that:

#! /bin/bash

one=""
two=""
# first we set up an input file to use:
echo "first word" > /tmp/myinput
echo "second line" >> /tmp/myinput
echo "third and last line" >> /tmp/myinput

# Now, this is not very impressive, right?
while read one two ; do
     echo "$one" "$two"
done < /tmp/myinput

# but this is exchanging the words, showing what the two variables really hold and adding visible delimiters:
while read one two ; do
     echo "\"$two\"" "\"$one\""
done < /tmp/myinput

exit 0

You may notice that in the last line of the file the last variable in the read-statement gets all the "leftover" words if there are fewer variables than words. This is intentional. You can also change the IFS so that what constitutes a "word" changes:

#! /bin/bash

one=""
two=""
# first we set up an input file to use:
echo "first word" > /tmp/myinput
echo "second line" >> /tmp/myinput
echo "third and last line" >> /tmp/myinput

# we change the IFS so that words are not split along blank-boundaries any more:
while IFS="," read one two ; do
     echo "\"$two\"" "\"$one\""
done < /tmp/myinput

exit 0

Because no "," was found in the whole input every line consisted of only one word which went to "$one" and "$two" remained empty.

Now, coming back to your problem: You have input that looks like this:

apples : 1111 : aaaa
pears : 2222 : bbbb
peaches : 3333 : cccc

Because the IFS is set to blanks per default it is split at blanks into words by the shell and this is why you get everything as a separate menu entry. It would be logical to simply set the IFS but i have tried it and it doesn't work - don't ask me why, i don't really understand bash. But there is another way: let us say you have a file called: file with a blank in its name . What would you do if you have to ls it:

# ls -l *name
total 196
-rw-rw-r-- 1 bakunin bakunin    35 Sep 12 08:26 file with a blank in its name

# ls -l file with a blank in its name
ls: cannot access 'file': No such file or directory
ls: cannot access 'with': No such file or directory
ls: cannot access 'a': No such file or directory
ls: cannot access 'blank': No such file or directory
ls: cannot access 'in': No such file or directory
ls: cannot access 'its': No such file or directory
ls: cannot access 'name': No such file or directory

We need a way to switch off the shells field splitting. Fortunately there is one: quoting. Quoting means to surround a string by quotes (single or double quotes). This does some other things too, but it also turns off field splitting:

# ls -l "file with a blank in its name"
total 196
-rw-rw-r-- 1 bakunin bakunin    35 Sep 12 08:26 file with a blank in its name

This is why most shell programmers are adamant about quoting, like i quoted the strings in the example programs above: it makes sure the script works even if there is a string with a blank or tab in it. Alas this won't work in your case either because you do not have fixed strings but generate the arrays contents in a subprocess dynamically. You can of course change the last sed -statement to surround the output with quotes (i took the liberty to remove the superfluous grep because sed can do that too):

options=( $(awless -l list routetables --columns ID,NAME,VPC --filter VPC=vpc-0025xxxxxxxe52 | sed -n '/rtb/ {;s/|//g;s/^/"/;s/$/"/;p;}') )

And there is another way you probably have figured out already and wonder why i haven't mentioned it all the time (well, actually because i wanted to show you all the alternatives): you can use a while-loop to read in the array and if you only use one variable all the content of the line ends there. Notice the quotation at the assignment of the array elements, though, otherwise you get a syntax error:

while read LINE ; do
     array[((${#array[@]}+1))]="$LINE"
done <<EOF
$(awless -l list routetables --columns ID,NAME,VPC --filter VPC=vpc-0025xxxxxxxe52 | sed -n '/rtb/ s/|//gp')
EOF

You sure ask yourself why this instead of:

awless -l list routetables --columns ID,NAME,VPC --filter VPC=vpc-0025xxxxxxxe52 | sed -n '/rtb/ s/|//gp' |\
while read LINE ; do
     array[((${#array[@]}+1))]="$LINE"
done

This is one of the great shortcomings of bash and one of the reasons why i prefer Korn shell: the second process of a pipeline (which is in this case the while-loop) is executed in a subshell, therefore the variables defined there are local to that subshell. If you want to use the cotent of array outside of this you need to use a here-document, a subshell and some redirection.

OK, finish for today. I hope it helps and you enjoyed it.

bakunin