problem with echo inserting single quotes

Shell Programming and Scripting
1

Consider the following script:

#!/bin/bash

exclude='Archive PST,SystemState'

IFS=$","
rsyncExclusions=$(for exclude in ${exclude}; do echo -n -e --exclude=\"${exclude}\"\  ; done)
unset IFS

echo rsync $rsyncExclusions test
rsync -avh --delete --delete-excluded "$rsyncExclusions" /tmp/test1 /tmp/test2

When the script runs it fails to exclude the exclusions from $exclude. the output from bash shows that when it echoes the exclusions into the rsync line they appear to have single quotes around them which ruins rsync's interpretation of the excludes.

The echo of the variable appears fine.

$ bash -x /tmp/exclude 
+ exclude='Archive PST,SystemState'
+ IFS=,
++ for exclude in '${exclude}'
++ echo -n -e '--exclude="Archive PST" '
++ for exclude in '${exclude}'
++ echo -n -e '--exclude="SystemState" '
+ rsyncExclusions='--exclude="Archive PST" --exclude="SystemState" '
+ unset IFS
+ echo rsync '--exclude="Archive' 'PST"' '--exclude="SystemState"' test
rsync --exclude="Archive PST" --exclude="SystemState" test
+ rsync -avh --delete --delete-excluded '--exclude="Archive PST" --exclude="SystemState" ' /tmp/test1 /tmp/test2
sending incremental file list
test1/
test1/Archive PST/
test1/SystemState/

sent 98 bytes  received 24 bytes  244.00 bytes/sec
total size is 0  speedup is 0.00

How can I get the output of the exclusions to appear exactly how echoing the variable looks?

2

Though I can't reproduce your fault there are a couple of oddities.

IFS=$","
unset IFS

Would probably be better as:

OLSIFS="${IFS}"
IFS=","
IFS="${OLDIFS}"

Same variable name?
for exclude in ${exclude}

We can avoid messing with IFS and also use a different variable name:

exclude='Archive PST,SystemState'

echo "${exclude}"|tr ',' '\n'|while read exclusion
do
        rsyncExclusions="${rsyncExclusions} --exclude=\"${exclusion}\""
done
3

I can't seem to get this to work.

I am using the following...

#!/bin/bash

exclude='Archive PST,SystemState'

echo "${exclude}" | tr ',' '\n' | while read exclusion
do
        rsyncExclusions="${rsyncExclusions} --exclude=\"${exclusion}\""
done

echo "$rsyncExclusions"
rsync -avh --delete --delete-excluded "$rsyncExclusions" /tmp/test1 /tmp/test2

This is the output:

$ bash -x /tmp/exclude 
+ exclude='Archive PST,SystemState'
+ echo 'Archive PST,SystemState'
+ tr , '\n'
+ read exclusion
+ rsyncExclusions=' --exclude="Archive PST"'
+ read exclusion
+ rsyncExclusions=' --exclude="Archive PST" --exclude="SystemState"'
+ read exclusion
+ echo ''

+ rsync -avh --delete --delete-excluded '' /tmp/test1 /tmp/test2
sending incremental file list
./
.viminfo

sent 11.85K bytes  received 40 bytes  23.77K bytes/sec
total size is 165.35K  speedup is 13.91
4

The read solution will not work in bash, since the part after the pipe runs in a subshell and thus the variables lose their values.
Your original idea would work, with a different variable name and a correct way of setting/unsetting IFS like methyl suggested.

You can do something like this:

rsyncExclusions=$(IFS=,;for excl in ${exclude}; do printf "%s" "--exclude=\"$excl\" "; done)

then you temporarily change IFS and you do not have to set IFS to its original value.

1 Like
5

Actually, I cannot get this to work either... I am using:

#!/bin/bash

exclude='Archive PST,SystemState'

#rsyncExclusions=$(IFS=,;for excl in ${exclude}; do printf "%s" "--exclude=\"$excl\" "; done)

echo "${rsyncExclusions}"
rsync -avh --delete --delete-excluded "${rsyncExclusions}" /tmp/test1/ /tmp/test2

I also tried to add "tr" at the end to remove the single quotes which are causing the problems but this did not work either.

rsyncExclusions=$(IFS=,;for excl in ${exclude}; do printf "%s" "--exclude=\"$excl\" " | tr -d "'" ; done)

output is:

$ bash -x /tmp/exclude 
+ exclude='Archive PST,SystemState'
++ IFS=,
++ for excl in '${exclude}'
++ printf %s '--exclude="Archive PST" '
++ for excl in '${exclude}'
++ printf %s '--exclude="SystemState" '
+ rsyncExclusions='--exclude="Archive PST" --exclude="SystemState" '
+ echo '--exclude="Archive PST" --exclude="SystemState" '
--exclude="Archive PST" --exclude="SystemState" 
+ rsync -avh --delete --delete-excluded '--exclude="Archive PST" --exclude="SystemState" ' /tmp/test1/ /tmp/test2
sending incremental file list

sent 78 bytes  received 14 bytes  184.00 bytes/sec
total size is 0  speedup is 0.00
$ ll /tmp/test2
total 16K
drwxr-xr-x 4 root root 4.0K 2010-09-07 23:41 ./
drwxrwxrwt 6 root root 4.0K 2010-09-08 01:25 ../
drwxr-xr-x 2 root root 4.0K 2010-09-07 23:32 Archive PST/
drwxr-xr-x 2 root root 4.0K 2010-09-07 23:41 SystemState/
6

Hi jelloir, try this:

eval rsync -avh --delete --delete-excluded "$rsyncExclusions" /tmp/test1 /tmp/test2
7

Great! that got it, Thanks Scrutinizer.

I'll do some reading up on eval to understand how eval helped.

$ bash -x /tmp/exclude 
+ exclude='Archive PST,SystemState'
++ IFS=,
++ for excl in '${exclude}'
++ printf %s '--exclude="Archive PST" '
++ for excl in '${exclude}'
++ printf %s '--exclude="SystemState" '
+ rsyncExclusions='--exclude="Archive PST" --exclude="SystemState" '
+ echo '--exclude="Archive PST" --exclude="SystemState" '
--exclude="Archive PST" --exclude="SystemState" 
+ eval rsync -avh --delete --delete-excluded '--exclude="Archive PST" --exclude="SystemState" ' /tmp/test1/ /tmp/test2
++ rsync -avh --delete --delete-excluded '--exclude=Archive PST' --exclude=SystemState /tmp/test1/ /tmp/test2
sending incremental file list
deleting SystemState/
deleting Archive PST/

sent 26 bytes  received 12 bytes  76.00 bytes/sec
total size is 0  speedup is 0.00
8

It's also cleaner to use arrays



#!/bin/bash

EXCLUDE=("Archive PST" "SystemState")

RSYNCEXCLUSIONS=()

for A in "${EXCLUDE[@]}"; do
	RSYNCEXCLUSIONS[${#RSYNCEXCLUSIONS[@]}]=$A
done

echo "rsync ${RSYNCEXCLUSIONS[*]} test"

rsync -avh --delete --delete-excluded "${RSYNCEXCLUSIONS[@]}" /tmp/test1 /tmp/test2

or

....

EXCLUDELIST="Archive PST,SystemState"
IFS=,
read -a EXCLUDE <<< "$EXCLUDELIST"

....