Help with Backup Shell Script 'ksh + awk over ssh'

Hi

newbeeeee alarm

i want to send a little script over ssh
this script mus download a report.tar then rename and move. the report name format is report_<host.with.dot>-10-09-20-11:55:25.tar

function remote_cmd_mv
{
  _host=$1

  ARCHROOTDIR='/tmp'
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPM='{lo -altr report_*.tar | awk -F" " '{print $9}'}'
  MV_NAME='{ echo ${LS_REPM} | awk -F"." '{print $1"." $2"." $3"." $4"." $5"." "tar_moved"}' }'


  cmd_string="cd \/tmp \
      for list in ${LS_REPM}; do ${list} ${ARCHMOVED}\/${MV_NAME}; done \
      cd - ; \
  "

  remote_cmd "${_host}" "${cmd_string}" ; ret_val=$? ;
}

error:


bash-3.00$ ksh script.ksh
script.ksh[69]: syntax error at line 108 : `"

  remote_cmd "${_host}" "$cmd_string" ; ret_val=$?
}

function remote_cmd_cp_str
{
  _host=$1

  ARCHROOTDIR="/tmp"
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPC='ls -1tr /tmp/report_* | tail -1'
  echo $LS_REPC

  cmd_string="${LS_REPC}"

  remote_cmd_cp "${_host}" "$cmd_string" ; ret_val=$? ;
}

function remote_cmd_mv
{
  _host=$1

  ARCHROOTDIR='/tmp'
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPM='{lo -altr report_*.tar | awk -F"' unexpected

bash-3.00$

You probably have some backtick-construct open, which isn't closed properly. Look from your line 108 backwards for something like

variable=`somecommand ... 

instead of

variable=`somecommand ... `

Notice the rightmost "`" in the second variant.

Furthermore, you shouldn't use backticks any more, as their usage is discouraged and the whole language construct works only for purposes of backward-compatibility. Use "$(...)" instead:

old:     variable=`somecommand | othercommand`
correct: variable=$(somecommand | othercommand)

This is easier to read, less error-prone and even usable recursively:

var=$(sub-shell-level-1 | command "$(sub-shell-level-2)")

I hope this helps.

bakunin

1 Like

thanks vor a quick help but something is wrong.

  102  function remote_cmd_mv
   103  {
   104    _host=$1
   105
   106    ARCHROOTDIR='/tmp'
   107    ARCHMOVED="$ARCHROOTDIR/report_moved"
   108    LS_REPM=$(ls -altr report_*.tar | awk -F" " '{print $9}')
   109    MV_NAME=$( echo ${LS_REPM} | awk -F"." '{print $1"." $2"." $3"." $4"." $5"." "tar_moved"}' )
   110
   111
   112    cmd_string="cd \/tmp \
   113        for list in ${LS_REPM}; do ${list} ${ARCHMOVED}\/${MV_NAME}; done \
   114        cd - ; \
   115    "
   116
   117    remote_cmd "${_host}" "${cmd_string}" ; ret_val=$? ;
   118  }
bash-3.00$ ksh script.ksh localhost
script.ksh[69]: syntax error at line 112 : `"

  remote_cmd "${_host}" "$cmd_string" ; ret_val=$?
}

function remote_cmd_cp_str
{
  _host=$1

  ARCHROOTDIR="/tmp"
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPC='ls -1tr /tmp/report_* | tail -1'
  echo $LS_REPC

  cmd_string="${LS_REPC}"

  remote_cmd_cp "${_host}" "$cmd_string" ; ret_val=$? ;
}

function remote_cmd_mv
{
  _host=$1

  ARCHROOTDIR='/tmp'
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPM=$(ls -altr report_*.tar | awk -F" " "{print $9}")
  MV_NAME=$( echo ${LS_REPM} | awk -F"." "{print $1"." $2"." $3"." $4"." $5"." "tar_moved"}" )


  cmd_string="cd' unexpected
bash-3.00$

---------- Post updated at 06:00 AM ---------- Previous update was at 03:12 AM ----------

doesn't work :frowning:

hmm...

The "script.ksh[69]" looks like the real error is in line 69. This means NOT the line number of the file the script resides in, but the 69th line of the main() routine of script.ksh. Once several functions reside in one file interpreting the line numbers becomes quite tricky.

As you haven't posted your complete script so far it is quite impossible for me to find out where the problem is - the code you posted seems to be correct as fas as i could see.

To further investigate issue at the start of your script a "set -xv" (don't forget to unset via "set +xv" at the end) and call your script this way:

script.ksh 2>&1 | more

I hope this helps.

bakunin

1 Like
bash-3.00$ cat script.ksh
#!/usr/bin/ksh
#------------------------------------------------------
# ./${0##*/} ${remote_host} #Useage
#------------------------------------------------------


#------------------------------------------------------
# Definicion
#------------------------------------------------------

this_script=${0##*/}
remote_host=${1}

export TIMEFORMAT='%Y%m%d%H%M%S'              # time format for logfile names
export TIMESTAMP=$(date +"${TIMEFORMAT}")

#export MOVED=report.${TIMESTAMP}.$$

[ -z "${AUTOMATION_DIR}" ] && { echo "Please set export AUTOMATION_DIR=<automation_dir_path> in $HOME/.bashrc" ; return 1 ; }
[ -z "${COSMOS_DIR}" ] && { echo "Please set export COSMOS_DIR=<cosmos_dir_path> in $HOME/.bashrc" ; return 1 ; }
[ -z "${COSMOS_GCM_DIR}" ] && { echo "Please set export COSMOS_GCM_DIR=<cosmos_gcm_dir_path> in $HOME/.bashrc" ; return 1 ; }
[ -z "${CDS_REPOSITORY_DIR}" ] && { echo "Please set export CDS_REPOSITORY_DIR=<CDS_REPOSITORY_DIR> in $HOME/.bashrc" ; return 1 ; }

[ -z "${remote_host}" ] && { echo "Please use ${this_script} host . If the report tar file is already in tmp then type ${this_script} localhost" ; return 1 ; }

#------------------------------------------------------
# Funktions
#------------------------------------------------------

function remote_cmd
{
  #sub_count=$((sub_count + 1 )) ; _remote_cmd=$FUNC ; FUNC=remote_cmd

  loginhost="$1"
  cmd_string="$2"
  cmd_login='eval ' ;
  localhost="$(hostname)" ;

  [ "$loginhost" = 'localhost' ] || [ "$loginhost" != "$localhost" ]&& cmd_login="ssh $loginhost " ;

  #msgout "L: ${FUNC}: $cmd_login $cmd_string" ;

  $cmd_login "$cmd_string" ; ret_val=$?

  #sub_count=$(($sub_count-1)) ; FUNC=$_remote_cmd
  return $ret_val
}

function remote_cmd_cp
{
  #sub_count=$((sub_count + 1 )) ; _remote_cmd_cp=$FUNC ; FUNC=remote_cmd_cp

  loginhost="$1"
  cmd_string="$2"
  cmd_login='eval ' ;
  localhost="$(hostname)" ;
  filename=$($cmd_string | awk -F"/" '{print $3}')

  [ "$loginhost" = 'localhost' ] || [ "$loginhost" != "$localhost" ]&& cmd_login="ssh $loginhost " ;

  #msgout "L: ${FUNC}: $cmd_login $cmd_string" ;

  $cmd_login "$cmd_string" > $filename ; ret_val=$?

  #sub_count=$(($sub_count-1)) ; FUNC=$_remote_cmd
  return $ret_val
}

function remote_cmd_test
{
  _host=$1

  ARCHROOTDIR="/tmp"
  ARCHMOVED="$ARCHROOTDIR/report_moved"


  cmd_string="\
      cd ; \
      [ !-d \${ARCHROOTDIR} ] && { echo "${ARCHROOTDIR} doesn't exist!" ; return 1 ; } \
      LS_REPT=$(ls -altr report_*.tar | awk -F" " '{print $9}') ; \
      [ -z "${LS_REPT}" ] && { echo "Reports in ${ARCHROOTDIR} doesn't exist" ; return 1 ; } \
      [ !-d \${ARCHMOVED} ] && { mkdir -p -m 775 \"${ARCHMOVED}" ; } \
  "

  remote_cmd "${_host}" "$cmd_string" ; ret_val=$?
}

function remote_cmd_cp_str
{
  _host=$1

  ARCHROOTDIR="/tmp"
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPC='ls -1tr /tmp/report_* | tail -1'
  echo $LS_REPC

  cmd_string="\${LS_REPC}"

  remote_cmd_cp "${_host}" "$cmd_string" ; ret_val=$? ;
}

function remote_cmd_mv
{
  _host=$1

  ARCHROOTDIR='/tmp'
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPM=$(ls -altr report_*.tar | awk -F" " '{print $9}')
  MV_NAME=$( echo ${LS_REPM} | awk -F"." '{print $1"." $2"." $3"." $4"." $5"." "tar_moved"}' )


  cmd_string="cd /tmp \
      for list in ${LS_REPM}; do ${list} ${ARCHMOVED}/${MV_NAME}; done \
      cd - ; \
  "

  remote_cmd "${_host}" "${cmd_string}" ; ret_val=$? ;
}


function tar_test
{

  ARCHROOTDIR="/tmp"
  [ ! -d ${ARCHROOTDIR} ] && { echo "${ARCHROOTDIR} doesn't exist!" ; return 1 ; }
  cd ${ARCHROOTDIR}
  ARCHUP="$ARCHROOTDIR/report_unpacked"

  LS_REP=$(ls -altr report_*.tar | awk -F" " '{print $9}') ;
  [ -z "${LS_REP}" ] && { echo "Reports in ${ARCHROOTDIR} doesn't exist" ; return 1 ; }
  [ ! -d "${ARCHUP}" ] && { mkdir -p -m 775 ${ARCHUP} ; }
  cd - ;

  tar_cmd; ret_val=$? ;
}


function tar_cmd
{

  ARCHROOTDIR="/tmp"
  cd ${ARCHROOTDIR}
  LS_REP=$(ls -altr report_*.tar | awk -F" " '{print $9}')
  REP_HOST=$(ls -altr report_*.tar | awk -F" " '{print $9}' | cut -c 8- | cut -f1 -d'-')
  UP_NAME=$(echo ${LS_REP} | awk -F"." '{print $1"." $2"." $3"." $4"." $5"." "tar_unpacked"})
  ARCHUP="$ARCHROOTDIR/report_unpacked"

  for list in ${LS_REP} ; do
        [ ! -d ${CDS_REPOSITORY_DIR}/${REP_HOST} ] && { tar -xvf "${list}" "${CDS_REPOSITORY_DIR}" ; } && \
        { svn add "${CDS_REPOSITORY_DIR}/${REP_HOST}" ; } && \
        { svn ci "${CDS_REPOSITORY_DIR}/${REP_HOST}" ; } && \
        { ret_val=$? ; }
    done

  for list2 in ${LS_REP} ; do
        [ -d ${CDS_REPOSITORY_DIR}/${REP_HOST} ] && { tar -xvf "${list2} ${CDS_REPOSITORY_DIR}" ; } && \
        { svn co "${CDS_REPOSITORY_DIR}/${REP_HOST}" ; } && \
        { svn add --force "${CDS_REPOSITORY_DIR}/${REP_HOST}" ; } && \
        { svn ci "${CDS_REPOSITORY_DIR}/${REP_HOST}" ; } && \
        { ret_val=$? ; } && \
    done

  for list in ${LS_REP}; do { mv ${LS_REP} ${ARCHUP}/${UP_NAME} ; } done
  cd -
  return $ret_val

}

function main
{
_host=$1

[ ${_host} = 'localhost' ] && { tar_test ; }
[ ${_host} != 'localhost' ] && { remote_cmd_test "$_host"; remote_cmd_cp_str "$_host"; remote_cmd_mv "$_host"; tar_test ; }
}

#------------------------------------------------------
# Main
#------------------------------------------------------


main "$remote_host";

# EOF
bash-3.00$ ksh -xv script.ksh localhost 2>&1 | more
#!/usr/bin/ksh
#------------------------------------------------------
# ./${0##*/} ${remote_host} #Useage
#------------------------------------------------------


#------------------------------------------------------
# Definicion
#------------------------------------------------------

this_script=${0##*/}
+ this_script=script.ksh
remote_host=${1}
+ remote_host=localhost

export TIMEFORMAT='%Y%m%d%H%M%S'              # time format for logfile names
+ export TIMEFORMAT=%Y%m%d%H%M%S
export TIMESTAMP=$(date +"${TIMEFORMAT}")
+ date +%Y%m%d%H%M%S
+ export TIMESTAMP=20101005132606

#export MOVED=report.${TIMESTAMP}.$$

[ -z "${AUTOMATION_DIR}" ] && { echo "Please set export AUTOMATION_DIR=<automation_dir_path> in $HOME/.bashrc" ; return 1 ; }
+ [ -z /home/pjt130/automation ]
[ -z "${COSMOS_DIR}" ] && { echo "Please set export COSMOS_DIR=<cosmos_dir_path> in $HOME/.bashrc" ; return 1 ; }
+ [ -z /home/pjt130/cosmos ]
[ -z "${COSMOS_GCM_DIR}" ] && { echo "Please set export COSMOS_GCM_DIR=<cosmos_gcm_dir_path> in $HOME/.bashrc" ; return 1 ; }
+ [ -z /home/pjt130/cosmos_gcm ]
[ -z "${CDS_REPOSITORY_DIR}" ] && { echo "Please set export CDS_REPOSITORY_DIR=<CDS_REPOSITORY_DIR> in $HOME/.bashrc" ; return 1 ; }
+ [ -z /home/pjt130/cds_repository ]

[ -z "${remote_host}" ] && { echo "Please use ${this_script} host . If the report tar file is already in tmp then type ${this_script} localhost" ; retur
n 1 ; }
+ [ -z localhost ]

#------------------------------------------------------
# Funktions
#------------------------------------------------------

function remote_cmd
{
  #sub_count=$((sub_count + 1 )) ; _remote_cmd=$FUNC ; FUNC=remote_cmd

  loginhost="$1"
  cmd_string="$2"
  cmd_login='eval ' ;
  localhost="$(hostname)" ;

  [ "$loginhost" = 'localhost' ] || [ "$loginhost" != "$localhost" ]&& cmd_login="ssh $loginhost " ;

  #msgout "L: ${FUNC}: $cmd_login $cmd_string" ;

  $cmd_login "$cmd_string" ; ret_val=$?

  #sub_count=$(($sub_count-1)) ; FUNC=$_remote_cmd
  return $ret_val
}

function remote_cmd_cp
{
  #sub_count=$((sub_count + 1 )) ; _remote_cmd_cp=$FUNC ; FUNC=remote_cmd_cp

  loginhost="$1"
  cmd_string="$2"
  cmd_login='eval ' ;
  localhost="$(hostname)" ;
  filename=$($cmd_string | awk -F"/" '{print $3}')

  [ "$loginhost" = 'localhost' ] || [ "$loginhost" != "$localhost" ]&& cmd_login="ssh $loginhost " ;

  #msgout "L: ${FUNC}: $cmd_login $cmd_string" ;

  $cmd_login "$cmd_string" > $filename ; ret_val=$?

  #sub_count=$(($sub_count-1)) ; FUNC=$_remote_cmd
  return $ret_val
}

function remote_cmd_test
{
  _host=$1

  ARCHROOTDIR="/tmp"
  ARCHMOVED="$ARCHROOTDIR/report_moved"


  cmd_string="\
      cd ; \
      [ !-d \${ARCHROOTDIR} ] && { echo "${ARCHROOTDIR} doesn't exist!" ; return 1 ; } \
      LS_REPT=$(ls -altr report_*.tar | awk -F" " '{print $9}') ; \
      [ -z "${LS_REPT}" ] && { echo "Reports in ${ARCHROOTDIR} doesn't exist" ; return 1 ; } \
      [ !-d \${ARCHMOVED} ] && { mkdir -p -m 775 \"${ARCHMOVED}" ; } \
  "

  remote_cmd "${_host}" "$cmd_string" ; ret_val=$?
}

function remote_cmd_cp_str
{
  _host=$1

  ARCHROOTDIR="/tmp"
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPC='ls -1tr /tmp/report_* | tail -1'
  echo $LS_REPC

  cmd_string="\${LS_REPC}"

  remote_cmd_cp "${_host}" "$cmd_string" ; ret_val=$? ;
}

function remote_cmd_mv
{
  _host=$1

  ARCHROOTDIR='/tmp'
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPM=$(ls -altr report_*.tar | awk -F" " '{print $9}')
  MV_NAME=$( echo ${LS_REPM} | awk -F"." '{print $1"." $2"." $3"." $4"." $5"." "tar_moved"}' )


  cmd_string="cd script.ksh[69]: syntax error at line 112 : `"

  remote_cmd "${_host}" "$cmd_string" ; ret_val=$?
}

function remote_cmd_cp_str
{
  _host=$1

  ARCHROOTDIR="/tmp"
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPC='ls -1tr /tmp/report_* | tail -1'
  echo $LS_REPC

  cmd_string="${LS_REPC}"

  remote_cmd_cp "${_host}" "$cmd_string" ; ret_val=$? ;
}

function remote_cmd_mv
{
  _host=$1

  ARCHROOTDIR='/tmp'
  ARCHMOVED="$ARCHROOTDIR/report_moved"
  LS_REPM=$(ls -altr report_*.tar | awk -F" " "{print $9}")
  MV_NAME=$( echo ${LS_REPM} | awk -F"." "{print $1"." $2"." $3"." $4"." $5"." "tar_moved"}" )


  cmd_string="cd' unexpected
bash-3.00$

---------- Post updated 06-10-10 at 12:23 AM ---------- Previous update was 05-10-10 at 06:26 AM ----------

New day, new hope... :smiley:

Found a few bugs
(1) Should use exit not return when testing env vars
(2) Double quotes need escaping with string
(3) Missing close single quote on awk command
(4) Floating &&

diff is a follows:

19,22c19,22
< [ -z "${AUTOMATION_DIR}" ] && { echo "Please set export AUTOMATION_DIR=<automation_dir_path> in $HOME/.bashrc" ; exit 1 ; }
< [ -z "${COSMOS_DIR}" ] && { echo "Please set export COSMOS_DIR=<cosmos_dir_path> in $HOME/.bashrc" ; exit 1 ; }
< [ -z "${COSMOS_GCM_DIR}" ] && { echo "Please set export COSMOS_GCM_DIR=<cosmos_gcm_dir_path> in $HOME/.bashrc" ; exit 1 ; }
< [ -z "${CDS_REPOSITORY_DIR}" ] && { echo "Please set export CDS_REPOSITORY_DIR=<CDS_REPOSITORY_DIR> in $HOME/.bashrc" ; exit 1 ; }
---
> [ -z "${AUTOMATION_DIR}" ] && { echo "Please set export AUTOMATION_DIR=<automation_dir_path> in $HOME/.bashrc" ; return 1 ; }
> [ -z "${COSMOS_DIR}" ] && { echo "Please set export COSMOS_DIR=<cosmos_dir_path> in $HOME/.bashrc" ; return 1 ; }
> [ -z "${COSMOS_GCM_DIR}" ] && { echo "Please set export COSMOS_GCM_DIR=<cosmos_gcm_dir_path> in $HOME/.bashrc" ; return 1 ; }
> [ -z "${CDS_REPOSITORY_DIR}" ] && { echo "Please set export CDS_REPOSITORY_DIR=<CDS_REPOSITORY_DIR> in $HOME/.bashrc" ; return 1 ; }
24c24
< [ -z "${remote_host}" ] && { echo "Please use ${this_script} host . If the report tar file is already in tmp then type ${this_script} localhost" ; exit 1 ; }
---
> [ -z "${remote_host}" ] && { echo "Please use ${this_script} host . If the report tar file is already in tmp then type ${this_script} localhost" ; return 1 ; }
79,82c79,82
<       [ !-d \${ARCHROOTDIR} ] && { echo \"${ARCHROOTDIR} doesn't exist!\" ; return 1 ; } \
<       LS_REPT=$(ls -altr report_*.tar | awk -F\" \" '{print $9}') ; \
<       [ -z "${LS_REPT}" ] && { echo \"Reports in ${ARCHROOTDIR} doesn't exist\" ; return 1 ; } \
<       [ !-d \${ARCHMOVED} ] && { mkdir -p -m 775 \"${ARCHMOVED}\" ; } \
---
>       [ !-d \${ARCHROOTDIR} ] && { echo "${ARCHROOTDIR} doesn't exist!" ; return 1 ; } \
>       LS_REPT=$(ls -altr report_*.tar | awk -F" " '{print $9}') ; \
>       [ -z "${LS_REPT}" ] && { echo "Reports in ${ARCHROOTDIR} doesn't exist" ; return 1 ; } \
>       [ !-d \${ARCHMOVED} ] && { mkdir -p -m 775 \"${ARCHMOVED}" ; } \
145c145
<   UP_NAME=$(echo ${LS_REP} | awk -F"." '{print $1"." $2"." $3"." $4"." $5"." "tar_unpacked"}')
---
>   UP_NAME=$(echo ${LS_REP} | awk -F"." '{print $1"." $2"." $3"." $4"." $5"." "tar_unpacked"})
160c160
<         { ret_val=$? ; }
---
>         { ret_val=$? ; } && \
1 Like

thanks for a note, it looks like much better

I am not sure where exactly the problem is, but i see at a glance that the script is poorly written:

1) instead of using "&&" and "||" to perform if-statements write them out. It is much easier to read (as well as to debug) that way.

Instead of

 [some condition] && { command1 ; command2 ; .... ; } || { command3 ; ... ; }

or something similar you should write it this way:

if [ some_condition ] ; then
     command1
     command2
     ....
else
     command3
     command4
     ....
fi

If you want to do yourself a favour and make it extra readable, document the more complicated branches by verbosely comment what the branches do. For instance:

if [ some_very_complicated_condition_about $FILE ] ; then     # if the file is a x-y type file
     command1
     command2
fi

2) When you use single quotes in strings make sure they are not misinterpreted as single-quoted strings:

"some str'ing"

might probably be misinterpreted as a single-quoted string 'ing' with just the closing single quote missing. You have used the single-quotes in "doesn't" several times. Write "does not" instead or - this may work, i haven't tested it - use escaping to mask the single-quotes. for instance:

x="string with a \' in it"

I hope this helps.

bakunin

1 Like

this script is maded for some stupid manager, and i don't want that it can be readed easily. So no comment and no easier way. :))