Help cannot concatenate Ksh variables ?

pcpinkerton · February 20, 2012, 11:57am

Cannot combine these two strings into one line, either as a 3rd variable or echo or printing ? Frustrating.

for i in `cat /scripts/pathList.dat`
do
    OldRepo= grep Oldhostname ${i}/.svn/entries | tail -1
    NewRepo= grep Oldhostname ${i}/.svn/entries | tail -1 | sed '/Oldhostname/ s/Oldhostname/Newhostname/g' | sed 's/\/oldpath1//g' | sed 's/\/oldpath2//g'
    echo "cd $i"
    echo "svn relocate $OldRepo $NewRepo" 
done

this always returns:

cd /path   ( this is ok )
svn relocate $OldRepo  
$NewRepo

I want "svn relocate $OldRepo $NewRepo "    all on the same line

I have tried RepLoc="$OldRepo $NewRepo"
echo $RepLoc with the same result each on a separate line.

and numerous others using echo -e with \c but nothing I do will concatenate $OldRepo $NewRepo

Please advise
:wall:

methyl · February 20, 2012, 12:15pm

Possible that your data files contain MS-DOS control characters.

Can you post a sample of a representative file using this "sed" statement which is designed to make control characters visible.

sed -n l .svn/entries

Also, I assume that the missing backticks on lines 3 and 4 of the script in post #1 are because this post came via a Microsoft editor?

pcpinkerton · February 20, 2012, 12:31pm

The script was written on a RedHat linux server using vi
The script was ran on the same Linux server using directories copied from a Windows server to a NAS mounted to the Linux server.

This post was created from an iMac (lion 10.7.x) using TexEditor application ( cut and paste from linux vi saved files.

in each path below there is a .svn/entries file the hostname is extracted from this file

snip from PathList:

/webdata/tmpDNAbld/PDSApp/PDSScrPd/Source
/webdata/tmpDNAbld/PDSApp/PDSScrPd/Source/Distrib
/webdata/tmpDNAbld/PDSApp/PDSScrPd/Source/Distrib/Release
/webdata/tmpDNAbld/PDSApp/PDSScrPd/Source/BC
/webdata/tmpDNAbld/PDSApp/PDSExclusive/Source/PDSExclusive
/webdata/tmpDNAbld/PDSApp/PDSExclusive/Source/PDSExclusive/Distrib
/webdata/tmpDNAbld/PDSApp/PDSExclusive/Source/PDSExclusive/Distrib/Release
/webdata/tmpDNAbld/PDSApp/PDSExclusive/Source/PDSExclusive/BC
/webdata/tmpDNAbld/PDSApp/PDSCompare/Source
/webdata/tmpDNAbld/PDSApp/PDSCompare/Source/distrib
/webdata/tmpDNAbld/PDSApp/PDSCompare/Source/distrib/release
/webdata/tmpDNAbld/PDSApp/PDSCompare/Source/BC
/webdata/tmpDNAbld/PDSApp/PDSOrganization/Source
/webdata/tmpDNAbld/PDSApp/PDSOrganization/Source/Distrib
/webdata/tmpDNAbld/PDSApp/PDSOrganization/Source/Distrib/Release
/webdata/tmpDNAbld/PDSApp/PDSOrganization/Source/BC
/webdata/tmpDNAbld/PDSApp/PDSMassXfer/Source
/webdata/tmpDNAbld/PDSApp/PDSMassXfer/Source/Distrib
/webdata/tmpDNAbld/PDSApp/PDSMassXfer/Source/Distrib/Release
/webdata/tmpDNAbld/PDSApp/PDSMassXfer/Source/BC
/webdata/tmpDNAbld/PDSApp/PDSBatch/Source

bsrepellant · February 20, 2012, 12:42pm

Try updating this line.

OldRepo= grep Oldhostname ${i}/.svn/entries | tail -1

to

OldRepo= grep Oldhostname ${i}/.svn/entries | tail -1 | tr -d '\n'

pcpinkerton · February 20, 2012, 1:13pm

this was an improvement. Both strings are now echo'ed together. Thanks

Now I just need "svn relocate" in the front of them and a space between OldRep and NewRep.

methyl · February 20, 2012, 1:18pm

We'd still like to see a sample of the ".svn/entries" file displayed with the "sed" I mentioned earlier. This would tell us what the line terminators are in this file and whether it is a recognisable text file format.

Further to bsrepellant, I suspect that we'll need to remove carriage-return characters (\r) not linefeed characters (\n).

With a bit of lateral thought we can use the same sed display to show what the script is actually outputting (rather than what it looks like on the screen):

./scriptname   |  sed -n l

pcpinkerton · February 20, 2012, 1:20pm

almost got it just changed to this (space not working yet):

echo -e "svn relocate\c" $OldRepo" " $NewRepo

bsrepellant · February 20, 2012, 1:36pm

Don't know why it wouldn't. What shell are you using?

echo "svn relocate " $OldRepo " " $NewRepo
echo "svn relocate ${OldRepo} ${NewRepo} " should work also.

pcpinkerton · February 20, 2012, 1:43pm

to find path and create file pathList.dat file

find /webdata/tmpDNAbld -type d -exec sh -c '[ -d "$1"/.svn ] || exit 1' sh {} \; -print | cat > pathList.dat

what the "entries file" looks like (where hostname is located) we are only interested in the second URL line

8

dir
5254
https://Oldhostname/oldpath1/svn/oldpath2/NBH-HMS-HealthPromotion/tags/hp_rewrite/WebApp/Views/Account
https://Oldhostname/oldpath1/svn/oldpath2/NBH-HMS-HealthPromotion

... snip

---------- Post updated at 01:43 PM ---------- Previous update was at 01:41 PM ----------

ksh

methyl · February 20, 2012, 2:02pm

@pcpinkerton
That doesn't look like the output of a sample:

sed -n l ./svn/entries

The format of these files is probably the problem. We need to see the control codes before a screen driver interprets them.
The "sed" is just designed to make control codes visible. When displayed by this method a normal unix line terminator shows as a dollar sign. Because these files came from a MAC they may contain carriage-return characters.

bsrepellant · February 20, 2012, 2:40pm

There was no confirmation from methyl's post about the missing backticks that assign the variables. Make sure this is not a typo.

pcpinkerton · February 20, 2012, 2:55pm

methyl:

We'd still like to see a sample of the ".svn/entries" file displayed with the "sed" I mentioned earlier. This would tell us what the line terminators are in this file and whether it is a recognisable text file format.

Further to bsrepellant, I suspect that we'll need to remove carriage-return characters (\r) not linefeed characters (\n).

With a bit of lateral thought we can use the same sed display to show what the script is actually outputting (rather than what it looks like on the screen):
./scriptname   |  sed -n l

results:

cd /path/to/destin\
ation$   
svn relocate OldUrl/path/destin\
ationNewUrl/path/destination$

seems a next line is inserted and no space between first destination and URL

2nd line terminated with $

---------- Post updated at 02:55 PM ---------- Previous update was at 02:41 PM ----------

What is a backtick ?

bsrepellant · February 20, 2012, 3:05pm

The character left of the 1 key.
Example:
OldRepo=`grep Oldhostname ${i}/.svn/entries | tail -1`

methyl · February 20, 2012, 3:05pm

Script line 1 contains two left quote characters. These are known in unix shell scripting as "backticks". In your ksh the code between the backticks is executed and the results placed on the command line.
Script lines 3 and 4 contain space characters where I would expect to see backticks.

variable=`command string`

bsrepellant · February 20, 2012, 3:11pm

Sorry for stepping on you methyl. That was your discovery. I'll back off...

methyl · February 20, 2012, 3:19pm

@bsrepellant
No problem at all with all of us working on this. If we ever manage to see a clean sample of the input data format we can run a test.
If the sample output posted is real (i.e. unexpanded variables), then there is something mega wrong with this shell.

pcpinkerton · February 20, 2012, 3:26pm

Previously when using the backtick it did not work at all.

Now with backticks only on the OldRepo line it is working

I changed this also

svnCommand="svn relocate"
set svnCommand

echo -e "${svnCommand}  ${OldRepo} ${NewRepo}"\c"

methyl · February 20, 2012, 7:27pm

Hmm the original script was a non-runner. Without the backticks the commands never executed. Please post the current version of the script (preferably intact), the sample input requested several times, the actual output of the script, and what you expect the actual output to look like.

The two sed statements at the end of the NewRepo= line look unlikely because they just delete strings which are not present in your data sample.
The delimiter for "sed" does not have to be a solidus. When editing data containing a solidus it is better to choose a different delimiter.

pcpinkerton · February 22, 2012, 12:25pm

ksh script output not as expected a continuation of "Help cannot concatenate variables" .

The issue seems to be the Input source file contains PATH with spaces in some of the directory names. Added snip from input file, console output and output file.

Output format expected

cd /PATH.....
svn relocate Oldrepo NewRepo

repeated for each path from Input file

#!/bin/ksh
#
# Read the list of Subversion working copies to Relocate to new Subversion Server
OutputLoc="/scripts/dataout"
bldDirLoc="/webdata/tmpDNAbld/"
wcfiles=${OutputLoc}/dnaRelwc1.dat  
svnRelcmd="svn relocate"
set svnRelcmd

cd ${bldDirLoc}

for i in `cat ${wcfiles}`
do
    OldRepo=`grep svn01g.gdc ${i}/.svn/entries | tail -1 | tr -d '\n'`
    NewRepo=`grep svn01g.gdc ${i}/.svn/entries | tail -1 | sed '/svn01g.gdc/ s/svn01g.gdc/svnprod/g' | sed 's/\/svn01p//g' | sed 's/\/Repositories//g'`
    echo "cd" $i
    echo -e "${svnRelcmd}  ${OldRepo}  ${NewRepo}\c"
done

Input Source snip ... from dnaRelwc1.dat

AgentWebPages/AgentWebPages/Source
AgentWebPages/AgentWebPages/Source/BinaryCompatibleComponent
AgentWebPages/AgentWebPages/Source/Distrib
AgentWebPages/AgentWebPages/Source/Distrib/Release
AgncyInfoInProc/SalesApplication/Source
AgncyInfoInProc/SalesApplication/Source/bc
AgncyInfoInProc/SalesApplication/Source/Distrib
AgncyInfoInProc/SalesApplication/Source/Distrib/Release
ALPS_ATTACHMENT/Utility EXE Attachment
ALPS_ATTACHMENT/Utility EXE Attachment/DotNET
ALPS_ATTACHMENT/Utility EXE Attachment/DotNET/bin
ALPS_ATTACHMENT/Utility EXE Attachment/DotNET/My Project
ALPS_ATTACHMENT/Utility EXE Attachment/DotNET/obj
ALPS_ATTACHMENT/Utility EXE Attachment/DotNET/obj/Debug
ALPS_ATTACHMENT/Utility EXE Attachment/DotNET/obj/Debug/TempPE
ALPS_ATTACHMENT/Utility EXE Attachment/DotNET/obj/Release
ALPS_ATTACHMENT/Utility EXE Attachment/DotNET/obj/Release/TempPE
ALPS_ATTACHMENT/Utility EXE Attachment/Release
ALPS_ATTACHMENT/Utility EXE Attachment/VB6
ALPS_ATTACHMENT/Utility EXE Attachment/VB6/Release
ALPS_ATTACHMENT/VB6
ALPS_ATTACHMENT/VB6/Release
ALPS_CACHE/Athena COM Trackers
ALPS_CACHE/COM Cache
ALPS_CACHE/COM Cache/Release
ALPS_CACHE/MSI
ALPS_COM/Athena COM Trackers
ALPS_COM/COM Attachment
ALPS_COM/COM Business Layer
ALPS_COM/COM Data Layer
ALPS_COM/COM Data Layer/Release
ALPS_COM/COM Utilities
ALPS_COM/MSI
ALPS_COMMUNICATIONS/Utility EXE Communication
ALPS_COMMUNICATIONS/Utility EXE Communication/DotNET
ALPS_COMMUNICATIONS/Utility EXE Communication/DotNET/bin
ALPS_COMMUNICATIONS/Utility EXE Communication/DotNET/My Project
ALPS_COMMUNICATIONS/Utility EXE Communication/DotNET/obj
...snip

Console output snip... via script utility

$ ./createRelocwcMstr.sh > dataout/relocMstr.dat
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/.svn/entries: No such file or directory
grep: Attachment/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/DotNET/.svn/entries: No such file or directory
grep: Attachment/DotNET/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/DotNET/bin/.svn/entries: No such file or directory
grep: Attachment/DotNET/bin/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/DotNET/My/.svn/entries: No such file or directory
grep: Attachment/DotNET/My/.svn/entries: No such file or directory
grep: Project/.svn/entries: No such file or directory
grep: Project/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/Debug/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/Debug/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/Debug/TempPE/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/Debug/TempPE/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/Release/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/Release/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/Release/TempPE/.svn/entries: No such file or directory
grep: Attachment/DotNET/obj/Release/TempPE/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/Release/.svn/entries: No such file or directory
grep: Attachment/Release/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/VB6/.svn/entries: No such file or directory
grep: Attachment/VB6/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: ALPS_ATTACHMENT/Utility/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: EXE/.svn/entries: No such file or directory
grep: Attachment/VB6/Release/.svn/entries: No such file or directory
grep: Attachment/VB6/Release/.svn/entries: No such file or directory
grep: ALPS_CACHE/Athena/.svn/entries: No such file or directory
grep: ALPS_CACHE/Athena/.svn/entries: No such file or directory
grep: COM/.svn/entries: No such file or directory
grep: COM/.svn/entries: No such file or directory
grep: Trackers/.svn/entries: No such file or directory
grep: Trackers/.svn/entries: No such file or directory
grep: ALPS_CACHE/COM/.svn/entries: No such file or directory
grep: ALPS_CACHE/COM/.svn/entries: No such file or directory
grep: Cache/.svn/entries: No such file or directory
grep: Cache/.svn/entries: No such file or directory
grep: ALPS_CACHE/COM/.svn/entries: No such file or directory
grep: ALPS_CACHE/COM/.svn/entries: No such file or directory
grep: Cache/Release/.svn/entries: No such file or directory
grep: Cache/Release/.svn/entries: No such file or directory
grep: ALPS_COM/Athena/.svn/entries: No such file or directory
grep: ALPS_COM/Athena/.svn/entries: No such file or directory
grep: COM/.svn/entries: No such file or directory
grep: COM/.svn/entries: No such file or directory
grep: Trackers/.svn/entries: No such file or directory
grep: Trackers/.svn/entries: No such file or directory
grep: ALPS_COM/COM/.svn/entries: No such file or directory
grep: ALPS_COM/COM/.svn/entries: No such file or directory
grep: Attachment/.svn/entries: No such file or directory
grep: Attachment/.svn/entries: No such file or directory
grep: ALPS_COM/COM/.svn/entries: No such file or directory
grep: ALPS_COM/COM/.svn/entries: No such file or directory
grep: Business/.svn/entries: No such file or directory
grep: Business/.svn/entries: No such file or directory
grep: Layer/.svn/entries: No such file or directory
grep: Layer/.svn/entries: No such file or directory
grep: ALPS_COM/COM/.svn/entries: No such file or directory
grep: ALPS_COM/COM/.svn/entries: No such file or directory
grep: Data/.svn/entries: No such file or directory

Output File #1 snip ...

cd AgentWebPages/AgentWebPages/Source/Distrib
svn relocate https://svn01g.gdc.nwie.net/svn01p/svn/Repositories/ePDS2 https://svnprod.nwie.net/svn/ePDS2
cd AgentWebPages/AgentWebPages/Source/Distrib/Release
svn relocate https://svn01g.gdc.nwie.net/svn01p/svn/Repositories/ePDS2 https://svnprod.nwie.net/svn/SWAT
cd AgncyInfoInProc/SalesApplication/Source
svn relocate https://svn01g.gdc.nwie.net/svn01p/svn/Repositories/SWAT https://svnprod.nwie.net/svn/SWAT
cd AgncyInfoInProc/SalesApplication/Source/bc
svn relocate https://svn01g.gdc.nwie.net/svn01p/svn/Repositories/SWAT https://svnprod.nwie.net/svn/SWAT
cd AgncyInfoInProc/SalesApplication/Source/Distrib
svn relocate https://svn01g.gdc.nwie.net/svn01p/svn/Repositories/SWAT https://svnprod.nwie.net/svn/SWAT
cd AgncyInfoInProc/SalesApplication/Source/Distrib/Release
svn relocate https://svn01g.gdc.nwie.net/svn01p/svn/Repositories/SWAT cd ALPS_ATTACHMENT/Utility
svn relocate  cd EXE
svn relocate  cd Attachment
svn relocate  cd ALPS_ATTACHMENT/Utility
svn relocate  cd EXE
svn relocate  cd Attachment/DotNET
svn relocate  cd ALPS_ATTACHMENT/Utility
svn relocate  cd EXE
svn relocate  cd Attachment/DotNET/bin
svn relocate  cd ALPS_ATTACHMENT/Utility
svn relocate  cd EXE
svn relocate  cd Attachment/DotNET/My
svn relocate  cd Project
svn relocate  cd ALPS_ATTACHMENT/Utility
svn relocate  cd EXE

bsrepellant · February 22, 2012, 12:55pm

Try this:

    OldRepo=`grep svn01g.gdc ${i}/.svn/entries 2>/dev/null | tail -1 | tr -d '\n'`
    NewRepo=`grep svn01g.gdc ${i}/.svn/entries 2>/dev/null| tail -1 | sed '/svn01g.gdc/ s/svn01g.gdc/svnprod/g' | sed 's/\/svn01p//g' | sed 's/\/Repositories//g'`