Unable to print python array in shell script loop.

mohtashims · August 8, 2019, 1:31pm

I am unable to loop print a python string array in my unix shell script:

~/readarr.sh '{{ myarr }}'

more readarr.sh

echo "Parameter 1:"$1
MYARRAY= $1
IFS=
MYARRAY=`python <<< "print ' '.join($MYARRAY)"`

for a in "$MYARRAY"; do
  echo "Printing Array: $a"
done

Can you suggest what is the issue with my code ?

I wish to run the shell script on IBM AIX machine non-bash shell.

rdrtx1 · August 8, 2019, 3:28pm

start with:

echo "Parameter 1:"$1
MYARRAY=($1)
IFS=
MYARRAY=`python <<< "print ''.join( \"${MYARRAY}\" )"`

for a in "${MYARRAY[@]}"; do
   echo "Printing Array: $a"
done

apmcd47 · August 8, 2019, 7:14pm

But you are demonstrating bash constructs, particularly the string redirection, <<< .

You also need to quote each element of the python array:

$ python <<< "print ' '.join(['u/tmp/file.js', 'u/var/test.txt', 'u/tmp/llo.rft'])"
u/tmp/file.js u/var/test.txt u/tmp/llo.rft

Do you really need to pass this python-style array into your script? Can you pass just a comma-delimited string?
Alternatively:

printf "Parameter 1: %s\n" "$1"
MYARRAY=`echo "$1" | sed 's/^\[//;s\]$//'`
oldIFS="$IFS"
IFS=,
for a in $MYARRAY
do printf "Printing Array: %s\n" "$a"
done
IFS="$oldIFS"

Assuming AIX has the printf utility.

Andrew

mohtashims · August 8, 2019, 10:24pm

apmcd47:

But you are demonstrating bash constructs, particularly the string redirection, <<< .

You also need to quote each element of the python array:
$ python <<< "print ' '.join(['u/tmp/file.js', 'u/var/test.txt', 'u/tmp/llo.rft'])"
u/tmp/file.js u/var/test.txt u/tmp/llo.rft
 
Do you really need to pass this python-style array into your script? Can you pass just a comma-delimited string?

@Andrew unfortunately the python string array is what is constructed by Ansible's loop / with _items by defaults and then it has to be passed to the shell script for processing.

I do not know if there is an easy way to convert the python string array to something else that suits our requirement inside of the ansible YML.

mohtashims · August 9, 2019, 1:07am

This suggestion does not work.

Below is the incorrect output recieved with this code:

Neo · August 9, 2019, 1:24am

Why not run the loop in python instead of putting the script in a "shell script" loop?

My guest is that the python script is already written and you want a "wrapper" to run the python script in a loop.

Maybe I am wrong?

Why not copy the python script and modify the original script do perform as you wish?

mohtashims · August 9, 2019, 1:34am

apmcd47:

But you are demonstrating bash constructs, particularly the string redirection, <<< .

You also need to quote each element of the python array:
$ python <<< "print ' '.join(['u/tmp/file.js', 'u/var/test.txt', 'u/tmp/llo.rft'])"
u/tmp/file.js u/var/test.txt u/tmp/llo.rft
 
Do you really need to pass this python-style array into your script? Can you pass just a comma-delimited string?
Alternatively:
printf "Parameter 1: %s\n" "$1"
MYARRAY=`echo "$1" | sed 's/^\[//;s\]$//'`
oldIFS="$IFS"
IFS=,
for a in $MYARRAY
do printf "Printing Array: %s\n" "$a"
done
IFS="$oldIFS"
 
Assuming AIX has the printf utility.

Andrew

The suggestion does not work and does not print anything.

As far as putting single quotes around parameters fine however, the appended 'u' is put by ansible's python array and we do not have control over it. Hence, with single quotes our array will not look as you shared but look something like below:

Please suggest.

apmcd47 · August 9, 2019, 4:52am

My original sed solution had a typo. This version works for both the samples you gave, with and without the quotes:

#!/bin/sh

printf "Parameter 1: %s\n" "$1"
MYARRAY=`echo "$1" | sed 's/^\[//; s/,/ /g; s/\]$//'`

for a in $MYARRAY
do
   a=`expr "$a" : 'u\(.*\)$'`
   printf "Printing Array: %s\n" "${a}"
done

While this is written in dash on Ubuntu, it should work for a 1980s-era Bourne Shell (I've no idea what AIX uses). One caveat is that if any of the strings don't start with a u the value will be lost. If your shell supports the prefix removal parameter expansion you could replace

   a=`expr "$a" : 'u\(.*\)$'`
   printf "Printing Array: %s\n" "${a}"

with

printf "Printing Array: %s\n" "${a#u}"

Your latter example should work with this script, using python:

#!/bin/sh

printf "Parameter 1: %s\n" "$1"
MYARRAY=`echo "print ' '.join($1)" | python`

for a in $MYARRAY
do
   printf "Printing Array: %s\n" "${a}"
done

Both of these have been tested.

Andrew

mohtashims · August 9, 2019, 6:19am

apmcd47:

My original sed solution had a typo. This version works for both the samples you gave, with and without the quotes:
#!/bin/sh

printf "Parameter 1: %s\n" "$1"
MYARRAY=`echo "$1" | sed 's/^\[//; s/,/ /g; s/\]$//'`

for a in $MYARRAY
do
   a=`expr "$a" : 'u$.*$$'`
   printf "Printing Array: %s\n" "${a}"
done
While this is written in dash on Ubuntu, it should work for a 1980s-era Bourne Shell (I've no idea what AIX uses). One caveat is that if any of the strings don't start with a u the value will be lost. If your shell supports the prefix removal parameter expansion you could replace
   a=`expr "$a" : 'u$.*$$'`
   printf "Printing Array: %s\n" "${a}"
with
printf "Printing Array: %s\n" "${a#u}"
Your latter example should work with this script, using python:
#!/bin/sh

printf "Parameter 1: %s\n" "$1"
MYARRAY=`echo "print ' '.join($1)" | python`

for a in $MYARRAY
do
   printf "Printing Array: %s\n" "${a}"
done
Both of these have been tested.

Andrew

Thank you Andrew however, both the solutions still do not work !! See the output below:

$ ./test.sh [u/tmp/file.js, u/var/test.txt, u/tmp/llo.rft]
Parameter 1: [u/tmp/file.js,
Printing Array: /tmp/file.js
[ ~]$ more test.sh
#!/bin/sh

FPATH=($1)

printf "Parameter 1: %s\n" "$FPATH"
FPATH=`echo "$FPATH" | sed 's/^\[//; s/,/ /g; s/\]$//'`

for a in $FPATH
do
   a=`expr "$a" : 'u\(.*\)$'`
   printf "Printing Array: %s\n" "${a}"
done

PFA snapshot:

Can you please check ? I'm using python 2.7.5 however, your non-pyton solution too does not work and yeild the same incorrect output as shared above.

apmcd47 · August 9, 2019, 7:34am

mohtashims:

Thank you Andrew however, both the solutions still do not work !! See the output below:
$ ./test.sh [u/tmp/file.js, u/var/test.txt, u/tmp/llo.rft]
Parameter 1: [u/tmp/file.js,
Printing Array: /tmp/file.js
[ ~]$ more test.sh
#!/bin/sh

FPATH=($1)

printf "Parameter 1: %s\n" "$FPATH"
FPATH=`echo "$FPATH" | sed 's/^\[//; s/,/ /g; s/\]$//'`

for a in $FPATH
do
   a=`expr "$a" : 'u$.*$$'`
   printf "Printing Array: %s\n" "${a}"
done
PFA snapshot:

Can you please check ? I'm using python 2.7.5 however, your non-pyton solution too does not work and yeild the same incorrect output as shared above.

First, you need to quote your input:

 ./test.sh "[u/tmp/file.js, u/var/test.txt, u/tmp/llo.rft]"

Second, don't use the shell array construct - do this instead:

#!/bin/sh

FPATH="$1"

printf "Parameter 1: %s\n" "$FPATH"
FPATH=`echo "$FPATH" | sed 's/^\[//; s/,/ /g; s/\]$//'`

for a in $FPATH
do
   a=`expr "$a" : 'u\(.*\)$'`
   printf "Printing Array: %s\n" "${a}"
done

Which shell are you using on AIX?

mohtashims · August 9, 2019, 1:02pm

apmcd47:

First, you need to quote your input:
 ./test.sh "[u/tmp/file.js, u/var/test.txt, u/tmp/llo.rft]"
Second, don't use the shell array construct - do this instead:
#!/bin/sh

FPATH="$1"

printf "Parameter 1: %s\n" "$FPATH"
FPATH=`echo "$FPATH" | sed 's/^\[//; s/,/ /g; s/\]$//'`

for a in $FPATH
do
   a=`expr "$a" : 'u$.*$$'`
   printf "Printing Array: %s\n" "${a}"
done
Which shell are you using on AIX?

I suspect I may be doing something wrong !!

I must say this works fine even when invoked from Ansible, however I have tested this on a My personal Linux System.

I would only be able to test this on AIX after 4 days as we have long weekend.

Special thanks to user @apm and others