Hi,
I'm building an installation system that uses separate data
files that contain the individual characteristics of each package.
Within the data file, I would like to incorporate a section that
contains a bash script that is loaded into an array which is
then redirected to bash to be executed.
Before I go into detail, I'd like to ascertain if it is possible ?
A.
Yes it is, usually with using the eval statement.
Fantastic!!!
Thanks for reply.
The data file would contain a section with the following tag
that is filtered from the other tags and loaded into an array.
The section is terminated either by another tag or end of file.
declare -a ARRAY
[script-nameofscript]
do bash stuff
[script-otherscripts]
more bash stuff
The tags are then removed leaving the raw bash code to be
redirected to bash to execute.
Would I need to include the bash header shebang (I think it's
called) at the beginning or does bash assume it's a bash script?
Having the option to use perl, python or ruby would be convenient.
Lastly, I'm not that familiar with the 'eval' command regarding
redirecting arrays that contain scripts to be executed in bash.
Therefore, an example would be greatly appreciated.
A.
---------- Post updated 15-01-10 at 08:59 AM ---------- Previous update was 14-01-10 at 10:48 PM ----------
Quick update.
I've found a number of examples that use strings that
contain commands to be executed with eval,
STRING="cmd args; cmd args"
eval "$STRING"
The way the install system works, will be to contain
a script in the vertical style in an array as opposed to the
horizontal scripting style associated with strings.
script example to be loaded into array
-------------------------------------------
#!/bin/bash
declare str
if $str
then do something
else otherwise
fi
Although it is possible to convert the vertical script into a
horizontal script, I would like to preserve the first line
that identifies the script type keeping the system flexible
and hence my preference for the vertical scripting style.
This may seem obvious, but from experience I get the
feeling this will not work,
eval "${CMD[@]}"
I've also considered the possibility, but not tried due to
unfamiliarity, of using IFS=$'\n' and possibly echo or
printf.
Any help would be appreciated.
A.
I'm unsure about what you mean with vertical and horizontal scripts (are the latter oneliners?) but in any case you can write:
STRING="#!/bin/bash
cmd args
cmd args"
eval "$STRING"
Note that the shebang directive will be ignored as eval is using the current shell anyway.
Sry, should have explained.
Horizontal scripting style does refer to the appending
of commands one after the other separated with semi-
colons otherwise referred to as 'one liners' ;-). As
opposed to using new lines for each command.
With reference to your example, when the code is loaded
into the array it will look something like this:
array[0]="#!/bin/bash"
array[1]="cmd args"
array[2]="cmd args"
However, the array is most likely to contain complex
bash scripts with for loops, if's and case statements as
well as variables of all types.
Bash will have to be able to evaluate all the normal things
it does but also take into account that it is a script that is
held in an array.
What I don't know is if Bash will recognise the whole script
in the array when it is evaluated so that I don't get syntax
errors when it tries to recognise the different elements of
each command e.g. the 'do' and 'done' part of a 'for' loop.
A.
This just works:
#!/bin/bash
array[0]="#!/bin/bash" # This line is a no op
array[1]="echo one"
array[2]="date"
array[3]="pwd"
IFS="
"
script="$(echo "${array[*]}")"
eval "$script"
Outstanding work my European friend.
I just tried a more complex script with a for loop
and that also seems to work,
#!/bin/bash
declare IFS=$'\n'
ARRAY[0]="#!/bin/bash"
ARRAY[1]="declare -i INT=10"
ARRAY[2]="for (( i=0 ; i<INT ; i++ )); do"
ARRAY[3]="echo "hello""
ARRAY[4]="done"
eval "$(echo "${ARRAY[*]}")"
Getting back to the shebang, if I wanted to run a
perl script, would this work from within bash,
#!/usr/bin/perl
Thanks for all your help.
A.
---------- Post updated 16-01-10 at 09:30 AM ---------- Previous update was 15-01-10 at 02:23 PM ----------
Additional.
Found some perl and python scripts,
#!/usr/bin/perl -w
use strict;
print "Hello, Perl!\n";
#!/usr/bin/python
print "Hello, Python!"
but the method that worked for the bash scripts
doesn't seem to work for the other scripts.
Using the previous method with the perl script,
#!/bin/bash
ARRAY[0]="#!/usr/bin/perl -w"
ARRAY[1]="use strict;"
ARRAY[2]="print "Hello, Perl!\n";"
IFS="
"
script="$(echo "${array[*]}")"
eval "$script"
I get the following error.
Perl!n;: command not found
The same with the python script,
#!/bin/bash
ARRAY[0]="#!/usr/bin/python"
ARRAY[1]="print "Hello, Python!""
IFS="
"
script="$(echo "${array[*]}")"
eval "$script"
produces this error.
Python!: command not found
I tested the perl and python scripts in terminal and the
output was as expected.
Although I have *no* idea what I'm talking about, as an
alternative, I'm experimenting with redirection to bash
so that the script is executed from stdin if it is possible.
Any further help would be appreciated.
A.
First of all, you should use single quotes when assigning to the array variables or escape the inner double quotes. Also, you are intermingling ARRAY and array.
Try:
#!/bin/bash
ARRAY[0]='#!/usr/bin/perl -w'
ARRAY[1]='use strict;'
ARRAY[2]='print "Hello, Perl!\n";'
To use other interpreters than bash you have to somehow invoke those interpreters. One way would be to use the contents of the shebang:
eval "${ARRAY[0]#\#!}" <(printf "$script")
-or-
printf "$script" | eval "${ARRAY[0]#\#!}"
Fantastic.
Should have seen the intermingling but I was working from
several examples and ended up copy/pasting from several
scripts.
Thanks to both jlliagre and Scrutinizer for all your help.
For future reference, these scripts work:
#!/bin/bash
ARRAY[0]='#!/usr/bin/python'
ARRAY[1]='print "Hello, Python!"'
IFS="
"
script="$(echo "${ARRAY[*]}")"
eval "${ARRAY[0]#\#!}" <(printf "$script")
and
#!/bin/bash
ARRAY[0]='#!/usr/bin/perl -w'
ARRAY[1]='use strict;'
ARRAY[2]='print "Hello, Perl!\n";'
IFS="
"
script="$(echo "${ARRAY[*]}")"
eval "${ARRAY[0]#\#!}" <(printf "$script")
---------- Post updated at 07:55 AM ---------- Previous update was at 06:21 AM ----------
Just a small problem.
The function that loads and assigns the array that contains the
script seems to imply double quotes. Is their a quick method
that could be built into a function to convert each element of
an array from double quotes to single quotes. Or if possible, to
include it within the eval statement.
I tried putting single quote escape characters around ARRAY
but that didn't seem to work.
A.
Maybe we are making this too complicated. If I have an input file like so:
# cat infile
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
[script-nameofscript]
#!/usr/bin/perl -w
use strict;
print "Hello, Perl!\n";
[/script-nameofscript]
This is line 10
This is line 11
This is line 12
This is line 13
This is line 14
And if I use a script like this:
#!/bin/bash
script=""
shebang=""
reading_script=0
while IFS= read -r line
do
if [[ $reading_script -eq 1 ]]; then
if [[ $line == "[/script-"* ]]; then
reading_script=0
eval "${shebang#\#!}" <(printf "$script")
script=""
shebang=""
else
script="$script$line"
fi
else
if [[ $line == "[script-"* ]]; then
reading_script=1
IFS= read -r shebang
else
printf "$line\n"
fi
fi
done < infile
Then the output is:
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
Hello, Perl!
This is line 10
This is line 11
This is line 12
This is line 13
This is line 14
Or is that not what you had in mind?
---------- Post updated at 14:39 ---------- Previous update was at 14:16 ----------
Slightly optimized script:
#!/bin/bash
reading_script=false
while IFS= read -r line
do
case $line in
"[/script-"*)
reading_script=false
eval "${shebang#\#!}" <<< $script
script=""; shebang="" ;;
"[script-"*)
reading_script=true
IFS= read -r shebang ;;
*)
if $reading_script; then
script="$script$line"
else
printf "$line\n"
fi ;;
esac
done < infile
---------- Post updated at 14:42 ---------- Previous update was at 14:39 ----------
Posix compliant version:
#!/bin/sh
reading_script=false
while IFS= read -r line
do
case $line in
"[/script-"*)
reading_script=false
printf "$script" | eval "${shebang#\#!}"
script=""; shebang="" ;;
"[script-"*)
reading_script=true
IFS= read -r shebang ;;
*)
if $reading_script; then
script="$script$line"
else
printf "$line\n"
fi ;;
esac
done < infile
This is an extract from the project,
#!/bin/bash
################################################################################
function array_append { eval "$1=( \"\${$1[@]}\" \"\${$2[@]}\" )"; }
function array_assign { eval "$1=( \"\${$2[@]}\" )"; }
function array_rm_front { eval "$1=( \"\${$1[@]##$2}\" )"; }
function array_loads { local IFS=$'\n'; eval "$1=( \$( < \"$2\" ) )"; }
function array_script { local IFS=$'\n'; eval "\${$1[0]#\#!} <(printf \${$1[*]:1} )"; }
function string_assigns { eval "$1=( \"\$2\" )"; }
################################################################################
function tag_filters
{
local -a _FLTR_SRC_; array_assign _FLTR_SRC_ $2
local _FLTR_TAG_; string_assigns _FLTR_TAG_ $3
local _FLTR_APP_; string_assigns _FLTR_APP_ "false"
local -a _FLTR_RTN_
for _LINE_ in "${_FLTR_SRC_[@]}"; do
case $_LINE_ in
\[$_FLTR_TAG_\] ) _FLTR_APP_=true;;
\[*\] ) _FLTR_APP_=false;;
* ) ;;
esac
case $_FLTR_APP_ in
true ) array_append _FLTR_RTN_ _LINE_;;
* ) continue;;
esac
done
array_assign $1 _FLTR_RTN_
}
################################################################################
function array_defrag
{
local -a _DFRG_SRC_; array_assign _DFRG_SRC_ $1
local -a _DFRG_RTN_
for _ITEM_ in "${_DFRG_SRC_[@]}"; do
if [ -n "$_ITEM_" ]
then array_append _DFRG_RTN_ _ITEM_
else continue
fi
done
array_assign $1 _DFRG_RTN_
}
################################################################################
# MAIN
################################################################################
declare -a _file_cfg_ _file_tmp_
array_loads _file_cfg_ "/where/script/file/is"
tag_filters _file_tmp_ _file_cfg_ "script-python"
array_rm_front _file_tmp_ '##'* # remove remarks
array_rm_front _file_tmp_ '['* # remove tags
array_defrag _file_tmp_ # defrag array
array_script _file_tmp_ # execute script
And this is an example of a data file that would contain tags including
scripts as well as others. The location of which has to be specified in
the variable above.
[different-tags]
################################################################################
## LOCAL SCRIPTS
################################################################################
[script-bash]
#!/bin/bash
declare -i INT=10
for (( i=0 ; i<INT ; i++ )); do
echo "Hello, Bash!"
done
################################################################################
[script-perl]
#!/usr/bin/perl -w
use strict;
print "Hello, Perl!\n";
################################################################################
[script-python]
#!/usr/bin/python
print "Hello, Python!"
################################################################################
[more-different-tags]
The sections don't have a corresponding closing tag. They are terminated
with either another tag or end of file, handled by the 'for' loop in the
'tag_filters' function. It was designed to take wildcards so that similar
tags would be filtered and then the contents of the array could be
refined and/or appended to each other with another function.
Conveniently, it also takes absolutes.
Getting back to the problem... I'm still experiencing problems with
incomplete commands, due to the lack of single quotes around the
elements of the array. The whole process is programatically applied
and I have no opportunity to manually insert single quotes around the
elements of the array.
To solve the problem, I was wondering if it was possible to somehow
incorporate the single quote solution into the 'array_script' command
so that whenever a script is invoked, it is processed there.
Alternatively, an acceptable solution would be a function that converts
an array so that each element is surround by single quotes just before
the 'array_script' function is called.
I hope the above is more informative.
A.
Hi ASGR, you could give this a try:
function array_script { local IFS=$'\n'; eval "\${$1[0]#\#!} <(printf \"\${$1[*]}\")"; }
I removed the :1 and added the escape inner double quotes.
---
I think the script you provided is perhaps a tad difficult to read. I am not convinced arrays are the way to go since IMO plain variables will serve that same purpose just fine. Also I am not sure if the abundant use of functions together with call by name paramter passing through the eval mechanism adds much to the clarity of the script.
---
FWIW, based on your latest input file specifications I created the following example script for you to use as a comparison:
#!/bin/sh
reading_script=false
while IFS= read -r line
do
case $line in
"[script-"*)
if $reading_script; then
printf "$script" | eval "${shebang#\#!}"
script=""; shebang=""
fi
reading_script=true
IFS= read -r shebang
;;
"["*)
reading_script=false
printf "$script" | eval "${shebang#\#!}"
script=""; shebang=""
;;
*) if $reading_script; then
script="$script$line\n"
else
printf "$line\n"
fi
;;
esac
done < infile
if $reading_script; then
printf "$script" | eval "${shebang#\#!}"
fi
Output:
################################################################################
## LOCAL SCRIPTS
################################################################################
Hello, Bash!
Hello, Bash!
Hello, Bash!
Hello, Bash!
Hello, Bash!
Hello, Bash!
Hello, Bash!
Hello, Bash!
Hello, Bash!
Hello, Bash!
Hello, Perl!
Hello, Python!
And here is the same script using a plain function with global variables and no parameters passing:
#!/bin/sh
exec_script()
{
if $READING_SCRIPT; then
printf "$SCRIPT" | eval "${SHEBANG#\#!}"
SCRIPT=""; SHEBANG=""
fi
}
READING_SCRIPT=false
while IFS= read -r line
do
case $line in
"[script-"*)
exec_script
READING_SCRIPT=true
IFS= read -r SHEBANG
;;
"["*)
exec_script
READING_SCRIPT=false
;;
*) if $READING_SCRIPT; then
SCRIPT="$SCRIPT$line\n"
else
printf "$line\n"
fi
;;
esac
done < infile
exec_script
S.
Thanks for reply.
I'll review the code when I have a chance.
Just to explain the method behind the madness, I'm trying
to create a reusable library of functions that is majority
array related.
Whilst consulting with a fellow forum user, it was determined
that using 'eval' was the best, if not only, solution to assign
and return arrays from functions. From there-on everything
became eval based for the sake of consistency.
A.
The consequence of using bash and arrays with functions is the use of name passing through eval. I think that you can probably also build those functions on the basis of plain variables and the read shell builtin. I think it is just a matter of preference. The use of variables is merely an alternative approach that may lead to simpler code..