Variable substitution with arrays

Shell Programming and Scripting
1

Hi all,

I have a script with the following gist:

declare -a index=(0 1 2 3 4);

declare -a animals=(dog cat horse penguin cow);

declare -a fruits=(orange apple grapes peach mango);

declare -a drinks=(juice milk coffee tea coke);

declare -a cities=(toronto paris london glasgow sydney);

declare -a countries=(canada france england scotland australia);

declare -a all=(animals fruits drinks cities countries);

for i in "${index[@]}" ; do
	echo ${i};
	animals="${animals["${i}"]}";
	echo $animals;
	fruits="${fruits["${i}"]}";
	echo $fruits;
	drinks="${drinks["${i}"]}";
	echo $drinks;
	cities="${cities["${i}"]}";
	echo $cities;
	countries="${countries["${i}"]}";
	echo $countries;
done

And this is the output:

0
dog
orange
juice
toronto
canada
1
cat
apple
milk
paris
france
2
horse
grapes
coffee
london
england
3
penguin
peach
tea
glasgow
scotland
4
cow
mango
coke
sydney
australia

Because my actual script has more arrays to loop through and I'll be adding more as time goes on, I came up with an 'all' array that has the array names in it. Then I attempted to loop through each array as such:

for i in "${index[@]}" ; do
	echo ${i};
	for j in "${all[@]}" ; do
		echo ${!j[${i}]};
	done
done

But this time I'm only getting items at index 0, as per the output below:

0
dog
orange
juice
toronto
canada
1





2





3





4





abc@xyz$

I've been struggling with this one for a while now, and I'm not entirely sure what I'm trying to accomplish is possible with Bash. Any help is appreciated!

Thanks!

2

I don't know if you can do what you're trying to do with bash , but you can do it with a 1993 or later version of ksh by using a name reference variable. You can't do it quite the way you were trying to do it because you can't currently create arrays of nameref variables, but the following seems to produce results similar to what I think you were trying to do:

#!/bin/ksh
animals=(dog cat horse penguin cow)
fruits=(orange apple grapes peach mango)
drinks=(juice milk coffee tea coke)
cities=(toronto paris london glasgow sydney)
countries=(canada france england scotland australia)
sparse[1]=first
sparse[10]=tenth
sparse[33]='thirty-third'
sparse[100]=hundredth

typeset -A associative
associative["a b"]='A B'
associative["x y z"]='X Y Z'
associative["SomeOtherString"]='Anything you might want'

arrays=(animals fruits drinks cities countries sparse associative)
typeset -n arrayname

for arrayname in "${arrays[@]}"
do	for subscript in "${!arrayname[@]}"
	do	printf '%s[%s]=%s\n' "${!arrayname}" "$subscript" "${arrayname[$subscript]}"
	done
done

which produces the output:

animals[0]=dog
animals[1]=cat
animals[2]=horse
animals[3]=penguin
animals[4]=cow
fruits[0]=orange
fruits[1]=apple
fruits[2]=grapes
fruits[3]=peach
fruits[4]=mango
drinks[0]=juice
drinks[1]=milk
drinks[2]=coffee
drinks[3]=tea
drinks[4]=coke
cities[0]=toronto
cities[1]=paris
cities[2]=london
cities[3]=glasgow
cities[4]=sydney
countries[0]=canada
countries[1]=france
countries[2]=england
countries[3]=scotland
countries[4]=australia
sparse[1]=first
sparse[10]=tenth
sparse[33]=thirty-third
sparse[100]=hundredth
associative[SomeOtherString]=Anything you might want
associative[a b]=A B
associative[x y z]=X Y Z
2 Likes
3

The underlying problem is: arrays in bash (and ksh88 as well) are ONE-dimensional. Therefore, you can create a variable holding a one-dimensional array, but you can't put other array variables as elements into this array.

You could use the following workaround, but i strongly suggest you don't. Stretching the limits of what can be done is fun and helps learning the trade, but you shouldn't put circus tricks into production code. So, with this (rather big) grain of salt, here it goes:

Variables are evaluated always in the same step and all at the same time, which is why you cannot do things like this:

xfoo="abc"
yfoo="def"
selector="x"

echo ${${selector}foo}

This would rely on "${selector}" to be evaluated first and only then the resulting "${xfoo}" to be evaluated again. But, as i said, this is not the case and therefore this will fail.

There is one remedy for that, though: the keyword eval . eval starts the evaluation process again and this way you get (among other things) a second evaluation phase for your variables:

xfoo="abc"
yfoo="def"
selector="x"

eval echo \${${selector}foo}

The same way you can create sort-of two-dimensional arrays by using this mechanism:

arr1[1]="arr1.1"
arr1[2]="arr1.2"
arr1[3]="arr1.3"
arr1[4]="arr1.4"

arr2[1]="arr2.1"
arr2[2]="arr2.2"
arr2[3]="arr2.3"
arr2[4]="arr2.4"

arr3[1]="arr3.1"
arr3[2]="arr3.2"
arr3[3]="arr3.3"
arr3[4]="arr3.4"


for i in 1 2 3 ; do
     for j in 1 2 3 4 ; do
         eval echo \${arr${i}[$j]}
     done
done

But again: avoid eval like the plague and if you have to use it this is usually indicative that you better search for an alternative. As a show-off of skill, though, it is pretty cool. No?

I hope this helps.

bakunin

1 Like
4

Hi Don and Bakunin,

Thank you both for your replies and explanations.

Don: It looks like I could accomplish what I want with ksh, except the output would need to be grouped by index as opposed to array names:

animals[0]=dog
fruits[0]=orange
drinks[0]=juice
cities[0]=toronto
countries[0]=canada

animals[1]=cat
fruits[1]=apple
drinks[1]=milk
cities[1]=paris
countries[1]=france

animals[2]=horse
fruits[2]=grapes
drinks[2]=coffee
cities[2]=london
countries[2]=england

animals[3]=penguin
fruits[3]=peach
drinks[3]=tea
cities[3]=glasgow
countries[3]=scotland

animals[4]=cow
fruits[4]=mango
drinks[4]=coke
cities[4]=sydney
countries[4]=australia

I had a feeling I'd get that kind of answer but was still hoping for a syntax fix :frowning:

Yes that's exactly what I wanted, evaluate "${selector}" first then "${xfoo}" :S

Because my script is pretty much done and I've never written ksh before, I guess I'll just stick to adding array names manually as I go. At least this implementation works in bash.

Thanks!

---------- Post updated at 12:11 PM ---------- Previous update was at 12:09 PM ----------

It's always good to know :smiley:

5

Don't despair - typeset -n is understood by bash as well:

typeset -n j
for i in "${index[@]}" ; do    echo ${i};      for j in "${all[@]}" ; do echo ${!j}, ${j[$i]};         done; done
0
animals, dog
fruits, orange
drinks, juice
cities, toronto
countries, canada
1
animals, cat
fruits, apple
drinks, milk
cities, paris
countries, france
2
animals, horse
fruits, grapes
drinks, coffee
cities, london
countries, england
3
animals, penguin
fruits, peach
drinks, tea
cities, glasgow
countries, scotland
4
animals, cow
fruits, mango
drinks, coke
cities, sydney
countries, australia

And, with eval , it would look like

for i in "${index[@]}" ; do    echo ${i};      for j in "${all[@]}" ; do eval echo \$j, \${$j[\$i]};   done; done
0
animals, dog
fruits, orange
drinks, juice
cities, toronto
countries, canada
1
animals, cat
fruits, apple
drinks, milk
cities, paris
countries, france
2
animals, horse
fruits, grapes
drinks, coffee
cities, london
countries, england
3
animals, penguin
fruits, peach
drinks, tea
cities, glasgow
countries, scotland
4
animals, cow
fruits, mango
drinks, coke
cities, sydney
countries, australia

Does any of these come close to what you want?

6

Hello RudiC,

Yes that's what I want, with the correct grouping and all. :slight_smile:

I am getting the expected output with 'eval' but not so with typeset:

abc@xyz$ ./loop.sh
./loop.sh: line 17: typeset: -n: invalid option
typeset: usage: typeset [-afFirtx] [-p] name[=value] ...
0
dog, animals
orange, fruits
juice, drinks
toronto, cities
canada, countries
1
dog,
orange,
juice,
toronto,
canada,
2
dog,
orange,
juice,
toronto,
canada,
3
dog,
orange,
juice,
toronto,
canada,
4
dog,
orange,
juice,
toronto,
canada,
abc@xyz$ bash --version
GNU bash, version 4.4.12(1)-release (x86_64-apple-darwin16.3.0)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

I'm using side-loaded bash 4 on OS X courtesy of Homebrew, if it matters.

Also, I'm confused as to why the order is flipped.

0
dog, animals

instead of

0
animals, dog
7

Hi RudiC,
Thanks for letting me know bash now recognizes typeset -n . (It doesn't in the bash I have on macOS Sierra 10.12.6 which includes GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16) . And, apparently it doesn't with the bash on whatever OS Kingzy is using.)

Hi Kingzy,
It is always a good idea to tell us what operating system you're using (in addition to the shell). If we know what OS and shells you have available, we're less likely to offer suggestions that won't work in your environment. Assuming that you have access to a 1993 or later version of the Korn shell, the following should work...

I had hoped that with the example code I had provided, you would have been able to reconfigure the loops in my example to get whatever output was desired. For the output format requested in post #4, the following should suffice:

#!/bin/ksh
index=(0 1 2 3 4)
all=(animals fruits drinks cities countries)

animals=(dog cat horse penguin cow)
fruits=(orange apple grapes peach mango)
drinks=(juice milk coffee tea coke)
cities=(toronto paris london glasgow sydney)
countries=(canada france england scotland australia)

typeset -n arrayname

separator=
for subscript in "${!index[@]}"
do	printf "$separator"
	separator='\n'
	for arrayname in "${all[@]}"
	do	printf '%s[%s]=%s\n' "${!arrayname}" "$subscript" "${arrayname[$subscript]}"
	done
done

which just switches the inner and outer loops in the code I suggested before, changes the names of the arrays used as looping values, and adds code to add an empty line between groups. The above produces the output requested in post #4.

8

My GNU bash, version 4.3.46(1)-release (x86_64-pc-linux-gnu) has typeset -n , my GNU bash, version 4.1.11(1)-release (amd64-portbld-freebsd9.0) doesn't.

@Kingzy: Unfortunately you don't tell us which code snippet led to the unsatisfying output. You may need to redefine ALL the arrays and unset any variables used before you rerun any of the scripts given in here to be sure that the start conditions are identical. One warning: it's a bad idea to use like

    animals="${animals["${i}"]}";

because the array will be modified sort of randomly depending on the i - value.

9

Don: My apologies for the initial lack of info. I am also using MacOS Sierra.

I also tried running that script on my work shell.

[work@shell ~]$ ./loop.sh
./loop.sh: line 17: typeset: -n: invalid option
typeset: usage: typeset [-aAfFgilrtux] [-p] name[=value] ...
0
dog, animals
orange, fruits
juice, drinks
toronto, cities
canada, countries
1
dog,
orange,
juice,
toronto,
canada,
2
dog,
orange,
juice,
toronto,
canada,
3
dog,
orange,
juice,
toronto,
canada,
4
dog,
orange,
juice,
toronto,
canada,
[work@shell ~]$ bash --version
GNU bash, version 4.2.46(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Thanks for the edit. I unfortunately do not have the option to rewrite the script my contrived example is based on from scratch. But I'll surely give ksh a try on another project! It seems to be more flexible than bash.

---------- Post updated at 05:45 PM ---------- Previous update was at 05:35 PM ----------

RudiC: Here is the code with typeset -n that I wasn't lucky with:

#!/bin/sh

declare -a index=(0 1 2 3 4);

declare -a animals=(dog cat horse penguin cow);

declare -a fruits=(orange apple grapes peach mango);

declare -a drinks=(juice milk coffee tea coke);

declare -a cities=(toronto paris london glasgow sydney);

declare -a countries=(canada france england scotland australia);

declare -a all=(animals fruits drinks cities countries);

typeset -n j

for i in "${index[@]}" ; do
  echo ${i};
  for j in "${all[@]}" ; do
    echo ${!j}, ${j[$i]};
  done;
done

It looks like neither shell at home or work has typeset -n :frowning:

10

Try again with the shebang in the first line set to #!/bin/bash or whatever the correct path be.

1 Like
11

RudiC: Thanks a lot! Changing the bash path did the trick, at least on my home machine.

I tried with the following 2 paths:

#/usr/local/Cellar/bash/4.3.33/bin/bash

#/usr/local/bin

Alas, it looks like the shell at my workplace doesn't have typeset. I tried running the script with the following path:

#/usr/local/bin
12

Try whereis bash (or whereis ksh ) or which bash or locate bash ...

13

RudiC: Yup I had tried echo $PATH before. Here is the output with the other commands:

[abc@xyz ~]$ which bash
/usr/bin/bash
[abc@xyz ~]$ whereis bash
bash: /usr/bin/bash /usr/share/man/man1/bash.1.gz
[abc@xyz ~]$ echo $PATH
/usr/local/bin:/usr/bin
[abc@xyz ~]$ locate bash
/usr/bin/bash
/usr/bin/bashbug
/usr/bin/bashbug-64
/usr/share/doc/bash-4.2.46
[abc@xyz ~]$

NB: locate bash dumped other paths which I removed from my copy-paste.

14

Only after writing my last post here i realised that you could construct a sort-of two-dimensional array out of delimited strings. Consider the following:

typeset array[1]="1-1:1-2:1-3"
typeset array[2]="2-1:2-2:2-3"
typeset array[3]="3-1:3-2:3-3"

typeset -i i=1
typeset -i j=1

# display row-wise
for i in 1 2 3 ; do
     for j in 1 2 3 ; do
          echo "${array[$i]}" | cut -d':' -f $j
     done
done

# display column-wise
for i in 1 2 3 ; do
     for j in 1 2 3 ; do
          echo "${array[$j]}" | cut -d':' -f $i
     done
done

You separate the columns by some delimiter (here ":") and use this as a means to create several entries into a single string.

I hope this helps.

bakunin

1 Like
15

Bakunin: My apologies for the late reply!

Your solution looks very promising :b:

I played with it a little bit to get the hang of it.

I hit a roadblock as soon as I replaced "1 2 3 4 5" in the nested for loop with "animals fruits drinks cities countries". I tried it in 2 ways but got the same result in both attempts. I also tried putting

typeset array[animals]="dog cat horse penguin cow"
typeset array[fruits]="orange apple grapes peach mango"
typeset array[drinks]="juice milk coffee tea coke"
typeset array[cities]="toronto paris london glasgow sydney"
typeset array[countries]="canada france england scotland australia"
all=(animals fruits drinks cities countries)

typeset -i i=1
typeset -i j=1

printf 'Output 1\n\n';
# display column-wise
for i in 1 2 3 4 5; do
     for j in animals fruits drinks cities countries; do
          echo "${array[${j}]}" | cut -d' ' -f $i
     done
done

printf '\n\nOutput 2\n\n';

# display column-wise
for i in 1 2 3 4 5; do
     for j in "${all[@]}" ; do
          echo "${array[${j}]}" | cut -d' ' -f $i
     done
done

abc@xyz$ ./typeset.sh
Output 1

canada
canada
canada
canada
canada
france
france
france
france
france
england
england
england
england
england
scotland
scotland
scotland
scotland
scotland
australia
australia
australia
australia
australia


Output 2

canada
canada
canada
canada
canada
france
france
france
france
france
england
england
england
england
england
scotland
scotland
scotland
scotland
scotland
australia
australia
australia
australia
australia

I had also tried the following and still got the same results:

typeset array['animals']="dog cat horse penguin cow"
typeset array['fruits']="orange apple grapes peach mango"
typeset array['drinks']="juice milk coffee tea coke"
typeset array['cities']="toronto paris london glasgow sydney"
typeset array['countries']="canada france england scotland australia"
all=('animals' 'fruits' 'drinks' 'cities' 'countries')

At least this alternative works just fine:

typeset array[1]="dog cat horse penguin cow"
typeset array[2]="orange apple grapes peach mango"
typeset array[3]="juice milk coffee tea coke"
typeset array[4]="toronto paris london glasgow sydney"
typeset array[5]="canada france england scotland australia"
all=(1 2 3 4 5)

# display column-wise
for i in 1 2 3 4 5; do 
     for j in "${all[@]}"; do 
          echo "${array[$j]}" | cut -d' ' -f $i
     done
done

abc@xyz$ ./typeset.sh
dog
orange
juice
toronto
canada
cat
apple
milk
paris
france
horse
grapes
coffee
london
england
penguin
peach
tea
glasgow
scotland
cow
mango
coke
sydney
australia

Is there a way to make it work with all=(animals fruits drinks cities countries) ?

16

I hate to say it, but: this was to be expected. What you tried was a so-called "associative array". This is an array where the index is not numbers but (arbitrary) strings. There some programming languages which offer this kind of arrays ( awk , for instance), but not bash. You can use ksh93 (Korn shell in its '93 version), which does offer such a functionality, but not bash or ksh88.

Lets see. I warn you beforehand, this is more a workaround, not really a solution. Consider the following sample script:

#! /bin/bash

typeset -i animals=1
typeset -i fruits=2
typeset -i drinks=3
typeset -i cities=4
typeset -i countries=5
typeset    all=($animals $fruits $drinks $cities $countries) 

typeset arr[$animals]="dog cat horse penguin cow"
typeset arr[$fruits]="orange apple grapes peach mango"
typeset arr[$drinks]="juice milk coffee tea coke"
typeset arr[$cities]="toronto paris london glasgow sydney"
typeset arr[$countries]="canada france england scotland australia"

for i in ${all[@]} ; do
     for j in 1 2 3 4 5 ; do
          echo ${i}:${j} $(echo ${arr[$i]} | cut -d' ' -f$j)
     done
done

exit 0

I hope this helps.

bakunin

1 Like
17

As much as I hate to say it: ever since version 4 (which the OP seems to run), bash HAS associative arrays:

man bash :

You need to declare / typeset them correctly, though ...

typeset -A arr=([animals]="dog cat horse penguin cow" [fruits]="orange apple grapes peach mango" [drinks]="juice milk coffee tea coke" [cities]="toronto paris london glasgow sydney" [countries]="canada france england scotland australia")
for i in ${!arr[@]}
  do    echo $i
        TMP=(${arr[$i]})
        for j in ${!TMP[@]}
          do    echo "  " ${TMP[$j]}
          done
  done
animals
     dog
     cat
     horse
     penguin
     cow
drinks
     juice
     milk
     coffee
     tea
     coke
fruits
     orange
     apple
     grapes
     peach
     mango
countries
     canada
     france
     england
     scotland
     australia
cities
     toronto
     paris
     london
     glasgow
     sydney
2 Likes
18

Oh! I stand corrected - thanks for pointing that out.

bakunin

19

Thanks a lot RudiC!

I've been spending some time trying to have a column-wise output, as opposed to row-wise as in your example, but most importantly target specific items in each "array" as well. All while typesetting everything the way you did. Here is what I came up with:

typeset -A arr=(

[animals]="dog cat horse penguin cow" 
[fruits]="orange apple grapes peach mango" 
[drinks]="juice milk coffee tea coke" 
[cities]="toronto paris london glasgow sydney" 
[countries]="canada france england scotland australia"

)

declare all=(animals fruits drinks cities countries)

j=0
k=2
for i in ${!arr[@]}
  do
        TMP=(${arr[$i]})
        echo ${all[j]} "  ->  " ${TMP[k]}
        (( j++ ))
  done

Here I hardcoded k but I do have a way in my actual script to increment it. As per the output below, I am getting the 2nd item of each array as expected but their order is somehow incorrect :frowning:

abc@xyz$ ./loop.sh
animals   ->   coffee
fruits   ->   grapes
drinks   ->   london
cities   ->   horse
countries   ->   england

k=3 outputs the same [incorrect] pattern:

abc@xyz./loop.sh
animals   ->   tea
fruits   ->   peach
drinks   ->   glasgow
cities   ->   penguin
countries   ->   scotland

Bakunin: Thank you as well! But why is it "not really a solution"? :confused:

As with RudiC's solution, I'm outputting column-wise and hardcoded $y to target a specific index, but this time the order is correct:

#! /bin/bash

typeset -i animals=1
typeset -i fruits=2
typeset -i drinks=3
typeset -i cities=4
typeset -i countries=5
typeset    all=($animals $fruits $drinks $cities $countries) 

typeset arr[$animals]="dog cat horse penguin cow"
typeset arr[$fruits]="orange apple grapes peach mango"
typeset arr[$drinks]="juice milk coffee tea coke"
typeset arr[$cities]="toronto paris london glasgow sydney"
typeset arr[$countries]="canada france england scotland australia"

for j in {1..5} ; do
  for i in ${all[@]} ; do
          echo ${i}:${j} $(echo ${arr[$i]} | cut -d' ' -f$j)
     done
done

printf "\n";
printf "Customized\n";
printf "\n";

y=2
z=`expr $y + 1`
for i in ${all[@]} ; do
    echo ${i}:$y $(echo ${arr[$i]} | cut -d' ' -f$z)
done

exit 0

abc@xyz./loop2.sh
1:1 dog
2:1 orange
3:1 juice
4:1 toronto
5:1 canada
1:2 cat
2:2 apple
3:2 milk
4:2 paris
5:2 france
1:3 horse
2:3 grapes
3:3 coffee
4:3 london
5:3 england
1:4 penguin
2:4 peach
3:4 tea
4:4 glasgow
5:4 scotland
1:5 cow
2:5 mango
3:5 coke
4:5 sydney
5:5 australia

Customized

1:2 horse
2:2 grapes
3:2 coffee
4:2 london
5:2 england


As much as I hate to say it: ever since version 4 (which the OP seems to run), bash HAS associative arrays:

Yes I'm using version 4 both at home and work, though I don't have the luxury of using typeset -n at work.

---------- Post updated at 05:55 PM ---------- Previous update was at 02:15 PM ----------

I fixed the order! I just had to change the for loop :rolleyes:

typeset -A arr=(
  [animals]="dog cat horse penguin cow" 
  [fruits]="orange apple grapes peach mango" 
  [drinks]="juice milk coffee tea coke" 
  [cities]="toronto paris london glasgow sydney" 
  [countries]="canada france england scotland australia"
)
declare all=(animals fruits drinks cities countries)

j=0
k=4
for i in ${all[@]}
  do
        TMP=(${arr[$i]})
        echo ${all[j]} "  ->  " ${TMP[k]}
        (( j++ ))
  done

abc@xyz$ ./loop.sh
animals   ->   cow
fruits   ->   mango
drinks   ->   coke
cities   ->   sydney
countries   ->   australia