Shell Script to replace tokens in multiple files

I have multiple script files that I have created, that allow me to simply replace a few tokens at the top of the file, and then not have to go through the actual script and change anything. I have about 10 of them, but I was hoping to find a way to write a small script that would allow me to input what value I want those tokens to hold, and then run that script, and it would go through all the scripts I have and replace the tokens with the values I input in to the first script.

Not sure if I explained it very well, but here's an example.

replaceScript.sh -

serverScript1.sh
serverScript2.sh
serverScript3.sh

when I run replaceScript.sh, I'd like to be able to input, say, 5 variables(var1-var2-...var5), then the replaceScript.sh will go through and replace those same variables that have been set in serverScript1-3, with the values I input in to the replaceScript.sh

Yes, we can help you do that, but we need some examples
ie. input and expected output.
control file name control.ctl:

value=1 value=32
value7=a value7=b
for file in *.txt
do
      while read old new
      do
          sed "s/$old/$new" "$file" > tmp
          mv tmp "$file"
      done < control.ctl
done 

This will not always work, especially when you try to replace what are regex metacharacters.

Okay, I have about, 8 variables(now that I think about it, variable isn't the best name for them. More just tags that I have created in the following format to make it easy to replace them manually, which I am now trying to do with a script). All named in this format

@tag1@
@tag2@
@tag3@
etc
.
.
.
Now the text inside the @..@ tag doesn't follow the naming convention of tag1.2.3.4.... etc, but it was just easier to use that on here.

The file names that hold these tags are as follows:

startTestABC
stopTestABC
deleteABC
rotateABC

A little more information would be that those ABC scripts all set some variable equal to the tag, ie

TEST_NAME="@tag3@"
ANOT_VAR="@tag4@"

That way, when I run the script I can replace all the @...@ tags with what I need, and then the bottom of the script calls everything by the constant variable names.

I was thinking either the script has a header that I could vi it and change it b/c this script will be used over and over with different values for the tags each time. So, either a header section, or even an interactive script where it asks for the values, stores them, then uses those to run the replace command in each script.

Output is obviously the updated ABC script. It doesn't even need to create the new files, all it has to do is modify the existing script, b/c I have my directory which holds these scripts, and then when I use them I always have to move them somewhere else, and then run the replace script from the new directory.

Okay, got a better way to explain it

##################################
#Header of script, place values in variable names#
##################################
VAR_MAN_NAME="@MAN_NAME@"
VAR_PORT_NUM="@PORT_NUM@"
VAR_TEST_NUM="@TEST_NUM@"
###################################
#---------------Script Logic---------------------#
###################################

.
.
.
.
This section uses all the variable names, and doesn't use any flags
.
.
.
.

I have multiple different scripts that use this same format and have the same variables and tags in each of them. Let's say I have hundreds of different things I need to run this script for. Right now it works fine, I just replace the @...@ tags manually each time I need a new set of scripts. That's the part I want to automate, I need a script where I can input the new tag values only once, and then that script will go through and replace the tags of my 10 scripts that use the same tags.

Looks like sed can do it, but I still haven't figured out the bst way to do it when I have 10 different strings I need to replace(that will be different everytime I run the replace.sh script) in 10 different files.

The replacement part itself ( replaceScript.sh script ) is not very hard. The point is that the *ABC scripts will be running multiple times, and after their first change through the replacement script, there won't be any more tags @..@ left in them ( or am I missing something here ? ), and you need to change those values back again in the future.

    before 			after
VAR_MAN_NAME="@MAN_NAME@"  VAR_MAN_NAME=other         NO more tags @..@!
VAR_PORT_NUM="@PORT_NUM@"  VAR_PORT_NUM=1001 
VAR_TEST_NUM="@TEST_NUM@"  VAR_TEST_NUM=20

So how are you planning to rechange the new values ( other,1001,20,...) to the old tag values @..@, for the next run ?

Are the variable names VAR_MAN_NAME, VAR_PORT_NUM,..., the same in all *ABC scripts ? If so, you can replace their values based on their unique names in all the files.

Another way, you might use their same record number ( NR ) in all the files, if they are in the same position.

I think you need to provide more details about the unique properties of these variables.

I didn't explain this part very well.

The ABC scripts I have in a separate directory. What I'm going to do is copy all those ABC scripts to a different directory along with the replace.sh script. Then I'll run the replace script. So I'll still have the originals untouched, and then the new ones are what I'll use for that. Then I can do the same thing with the original ABC scripts, whenever I need to replace the @...@ tokens with something else.

Edit:

to add to that, the name of the scripts are always going to be static, and won't change, that's why I just want to copy them to a temp directory, run the script, and then I can do what I need to with them. No need to rename the files or anything like that.

Assuming the same format and your initial conditions, try and adapt the following:

#!/bin/bash
#set -x

cd /full/path/to/your/ABC_scripts

read -p "enter 1st value: "  v1
read -p "enter 2rd value: "  v2
read -p "enter 3rd value: "  v3
# ... Enter more variables here ( v4, v5,...), in the same way as above.

read -p "enter full path to temp dir: "  path

#If you want the path hardcoded use,
# path="/full/path/to/temp-dir"



for file in *ABC

 do

  awk -F= '/@MAN_NAME@/{$0=$1"="v1}
           /@PORT_NUM@/{$0=$1"="v2}
           /@TEST_NUM@/{$0=$1"="v3}

           #/@other_tags_here@/{ "preserve same format as above" }

                                  1' v1="$v1" v2="$v2" v3="$v3" "$file" > "$path"/"$file" 

 done

deleted

Okay, forget that last part, it was b/c I was trying to read the files in the same directory I was trying to then write them to.

One other issue though, the scripts that have the tags in them that I'm trying to replace have these lines:

S_N="@A_S_N@"
B_D="@B_I_D@"
D_B="${B_D}/user_projects/@B_D_N@"
D_D="${D_B}/@D_N@"
S_U="t3://@U_S@:@A_S_P@"
A_U="t3://@A_U_S@:@A_S_P@"
K_D="@U_H@/keys/@B_D_N@/@K_S_D@"

So I would be expecting the tags to get replaced with the values I entered, but instead I'm like in the A_U line, instead of getting:

A_U="t3://U_S_Tag_Value:A_S_P_Tag_Value"

I get:

A_U=A_S_P_Tag_Value

No quotations, the colon was taken out, and the first tag value was completely taken out.

Same with the rest of them, all the quotations are gone.

This is my code, tag names are changed and stuff like that, but this is the exact structure of it:

#!/bin/bash
#set -x

cd /path/to/ABC/scripts/

read -p "TAG_1: "  v1
read -p "TAG_2: "  v2
read -p "TAG_3: "  v3
read -p "TAG_4: "  v4
read -p "TAG_5: "  v5
read -p "TAG_6: "  v6
read -p "TAG_7: "  v7
read -p "TAG_8: "  v8
read -p "TAG_9: "  v9
read -p "TAG_10: "  v10
read -p "TAG_11: "  v11
read -p "TAG_12: "  v12
read -p "TAG_13: "  v13

# ... Enter more variables here ( v4, v5,...), in the same way as above.

#read -p "/path/to/tmp/: "  path

#If you want the path hardcoded use,
path="/path/to/tmp/"



for file in *ABC

 do

  awk -F= '/@TAG_1@/{$0=$1"="v1}
           /@TAG_2@/{$0=$1"="v2}
           /@TAG_3@/{$0=$1"="v3}
           /@TAG_4@/{$0=$1"="v4}
           /@TAG_5@/{$0=$1"="v5}
           /@TAG_6@/{$0=$1"="v6}
           /@TAG_7@/{$0=$1"="v7}
           /@TAG_8@/{$0=$1"="v8}
           /@TAG_9@/{$0=$1"="v9}
           /@TAG_10@/{$0=$1"="v10}
           /@TAG_11@/{$0=$1"="v11}
           /@TAG_12@/{$0=$1"="v12}
           /@TAG_13@/{$0=$1"="v13}



#/@other_tags_here@/{ "preserve same format as above" }

                                  1' v1="$v1" v2="$v2" v3="$v3" v4="$v4" v5="$v5" v6="$v6" v7="$v7" v8="$v8" v9="$v9" v10="$v10" v11="$v11" v12="$v12" v13="$v13" "$file" > "$path"/"$file"

 done

Well, the situation has changed, your tokens have special characters in them, and you need to escape all of them. See the sample below and make the proper changes to your code.

awk -F= '/\$\{B_D\}\/user_projects\/@B_D_N@/{$0=$1"="v1}
          /t3:\/\/@U_S@:@A_S_P@/            {$0=$1"="v2}
          /@U_H@\/keys\/@B_D_N@\/@K_S_D@/   {$0=$1"="v3}

          ## repeat the same thing for all of your tokens, using their actual names, and escape special chars:     - $ / + * ? . etc 

                                  1' v1="$v1" v2="$v2" v3="$v3" "$file" > "$path"/"$file"

That's the point, the structure looks fine, but it's the names of the tokens that are important, the script depends totally on their unique names and formation.

The code worked fine for me, look at a test using the above adjustments, with three random tokens given above:

# ./replace.sh

enter 1st value:
"t3://U_S_Tag_Value:A_S_P_Tag_Value"
enter 2nd value:
"other variable goes here"
enter 3rd value:
"000 000 000"
enter full path to temp dir:
/home/user/test

------

$ ll test
-rw-r--r--   1 user       sys            557 Oct 28 19:37 file_1ABC
-rw-r--r--   1 user       sys            557 Oct 28 19:37 file_2ABC
-rw-r--r--   1 user       sys            557 Oct 28 19:37 file_3ABC
-rw-r--r--   1 user       sys            557 Oct 28 19:37 file_4ABC


$ cat test/file_2ABC

##################################
#Header of script, place values in variable names#
##################################
S_N="@A_S_N@"
B_D="@B_I_D@"
D_B="t3://u_s_Tag_Value:a_s_p_Tag_Value"
D_D="${D_B}/@D_N@"
S_U="other variable goes here"
A_U="t3://@A_U_S@:@A_S_P@"
K_D="000 000 000"
VAR_MAN_NAME="@MAN_NAME@"
VAR_PORT_NUM="@PORT_NUM@"
VAR_TEST_NUM="@TEST_NUM@"
###################################
#---------------Script Logic---------------------#
###################################
.
.
.
This section uses all the variable names, and doesn't use any flags

Here is one of the header of one of the actual scripts for an example.

SERVER_NAME="@ADMIN_SERVER_NAME@"
BASE_DIR="@BEA_INSTALL_DIR@"
DOMAIN_BASE="${BASE_DIR}/user_projects/@BASE_DOMAIN_NAME@"
DOMAIN_DIR="${DOMAIN_BASE}/@DOMAIN_NAME@"
SERVER_URL="t3://@UNIX_SERVER@:@ADMIN_SERVER_PORT@"
ADMIN_URL="t3://@ADMIN_UNIX_SERVER@:@ADMIN_SERVER_PORT@"
KEYS_DIR="@USER_HOME@/keys/@BASE_DOMAIN_NAME@/@KEYS_SUB_DIR@"

Some of those aren't inside the tags and I'm only looking to replace the tags, not entire variable names. And as you can see some variables are made up of multiple tags b/c those shorter tags may be used in other parts of the script as well. But the variable is not needed, just part of it(the tag).

There are only two tags that will have something besides a number or letter, and that's the @BEA_INSTALL_DIR@ and the @USER_HOME@ directory.

Can you post :

  • All of your variables ( with all the tokens ) in your ABC files ,
  • An exact sample of how the variables will look like after the replacement ?

Does the script work for you for variables with a single token in them ( only one set of tags @..@ ) ?

It works, but for some reason it gets rid of the quotations that are around the tokens.

#!/bin/bash
#set -x

cd /path/to/files/

read -p "MAN_SERVER_NAME: "  v1
read -p "BEA_INSTALL_DIR: "  v2
read -p "BASE_DOMAIN_NAME: "  v3
read -p "DOMAIN_NAME: "  v4
read -p "ADMIN_SERVER_NAME: "  v5
read -p "UNIX_SERVER: "  v6
read -p "ADMIN_SERVER_PORT: "  v7
read -p "ADMIN_UNIX_SERVER: "  v8
read -p "USER_HOME: "  v9
read -p "KEYS_SUB_DIR: "  v10
read -p "MAN_SERVER_PORT: "  v11
read -p "WLS_ENV: "  v12
read -p "WEBLOGIC_VERSION: "  v13

# ... Enter more variables here ( v4, v5,...), in the same way as above.

#read -p "/: "  path

#If you want the path hardcoded use,
path="/path/to/tmp/dir/"



for file in *WLS

 do

  awk -F= '/@MAN_SERVER_NAME@/{$0=$1"="v1}
           /@BEA_INSTALL_DIR@/{$0=$1"="v2}
           /@BASE_DOMAIN_NAME@/{$0=$1"="v3}
           /@DOMAIN_NAME@/{$0=$1"="v4}
           /@ADMIN_SERVER_NAME@/{$0=$1"="v5}
           /@UNIX_SERVER@/{$0=$1"="v6}
           /@ADMIN_SERVER_PORT@/{$0=$1"="v7}
           /@ADMIN_UNIX_SERVER@/{$0=$1"="v8}
           /@USER_HOME@/{$0=$1"="v9}
           /@KEYS_SUB_DIR@/{$0=$1"="v10}
           /@MAN_SERVER_PORT@/{$0=$1"="v11}
           /@WLS_ENV@/{$0=$1"="v12}
           /@WEBLOGIC_VERSION@/{$0=$1"="v13}



#/@other_tags_here@/{ "preserve same format as above" }

                                  1' v1="$v1" v2="$v2" v3="$v3" v4="$v4" v5="$v5" v6="$v6" v7="$v7" v8="$v8" v9="$v9" v10="$v10" v11="$v11" v12="$v12" v13="$v13" "$file" > "$path"/"$file"

 done

Header Sections

SERVER_NAME="@MAN_SERVER_NAME@"
BASE_DIR="@BEA_INSTALL_DIR@"
DOMAIN_BASE="${BASE_DIR}/user_projects/@BASE_DOMAIN_NAME@"
DOMAIN_DIR="${DOMAIN_BASE}/@DOMAIN_NAME@"
MANSERVER_DIR="${DOMAIN_BASE}/managedservers"
SERVER_NAME="@MAN_SERVER_NAME@"
BASE_DIR="@BEA_INSTALL_DIR@"
DOMAIN_BASE="${BASE_DIR}/user_projects/@BASE_DOMAIN_NAME@"
DOMAIN_DIR="${DOMAIN_BASE}/@DOMAIN_NAME@"
MANSERVER_DIR="${DOMAIN_BASE}/managedservers"
SERVER_NAME="@ADMIN_SERVER_NAME@"
BASE_DIR="@BEA_INSTALL_DIR@"
DOMAIN_BASE="${BASE_DIR}/user_projects/@BASE_DOMAIN_NAME@"
DOMAIN_DIR="${DOMAIN_BASE}/@DOMAIN_NAME@"
SERVER_NAME="@MAN_SERVER_NAME@"
BASE_DIR="@BEA_INSTALL_DIR@"
DOMAIN_BASE="${BASE_DIR}/user_projects/@BASE_DOMAIN_NAME@"
MANSERVER_DIR="${DOMAIN_BASE}/managedservers"
SERVER_NAME="@ADMIN_SERVER_NAME@"
BASE_DIR="@BEA_INSTALL_DIR@"
DOMAIN_BASE="${BASE_DIR}/user_projects/@BASE_DOMAIN_NAME@"
DOMAIN_DIR="${DOMAIN_BASE}/@DOMAIN_NAME@"
WLS_VERSION="@WEBLOGIC_VERSION@"
SERVER_NAME="@ADMIN_SERVER_NAME@"
BASE_DIR="@BEA_INSTALL_DIR@"
DOMAIN_BASE="${BASE_DIR}/user_projects/@BASE_DOMAIN_NAME@"
DOMAIN_DIR="${DOMAIN_BASE}/@DOMAIN_NAME@"
SERVER_URL="t3://@UNIX_SERVER@:@ADMIN_SERVER_PORT@"
ADMIN_URL="t3://@ADMIN_UNIX_SERVER@:@ADMIN_SERVER_PORT@"
KEYS_DIR="@USER_HOME@/keys/@BASE_DOMAIN_NAME@/@KEYS_SUB_DIR@"

Inside @USER_HOME@ will be a directory, and inside @BEA_INSTALL_DIR@ will be a directory. The rest will just be a combo of letters and numbers

Still one question left unanswered,

example,

SERVER_NAME="@ADMIN_SERVER_NAME@"   --->                      SERVER_NAME="server A"                               
SERVER_URL="t3://@UNIX_SERVER@:@ADMIN_SERVER_PORT@"     --->    SERVER_URL="t3://Server UNIX:1001"

I'm assuming the headers that you posted, are exactly as they look in of all of the files, and there are no more of them (?).

And when you enter the variable values at the prompt, do you use quotes around them ( say "server A" ), or simply:

enter SERVER_NAME:

server A

I have not been using quotes when typing in the values.

The one will look like

SERVER_URL="t3://servername:9999"

There are many more scripts, but those are the only real distinct headers, the rest are just simple one variable, one tag.

The code is tested with the data you posted here and it works fine. When you provide the missing info ( see below ) I'll update the post.

#!/bin/bash
#set -x

cd /path/to/files/

read -p "MAN_SERVER_NAME: "   v1
read -p "BEA_INSTALL_DIR: "   v2
read -p "BASE_DOMAIN_NAME: "  v3
read -p "DOMAIN_NAME: "       v4
read -p "ADMIN_SERVER_NAME: " v5
read -p "UNIX_SERVER: "       v6
read -p "ADMIN_SERVER_PORT: " v7
read -p "ADMIN_UNIX_SERVER: " v8
read -p "USER_HOME: "         v9
read -p "KEYS_SUB_DIR: "      v10
read -p "MAN_SERVER_PORT: "   v11
read -p "WLS_ENV: "           v12
read -p "WEBLOGIC_VERSION: "  v13

#read -p "/: "  path

#If you want the path hardcoded use,
path="/path/to/tmp/dir/"


for file in *WLS

 do

  awk -F@  '
	    /@MAN_SERVER_NAME@/   			     {$0=$1 v1 $3}
            /@BEA_INSTALL_DIR@/   			     {$0=$1 v2 $3}
            /@BASE_DOMAIN_NAME@/  			     {$0=$1 v3 $3}
            /@DOMAIN_NAME@/       			     {$0=$1 v4 $3}
            /@ADMIN_SERVER_NAME@/ 			     {$0=$1 v5 $3}
            /@UNIX_SERVER@/ && /@ADMIN_SERVER_PORT@/         {$0=$1 v6 $3 v7 $5}
            /@ADMIN_UNIX_SERVER@/			     {$0=$1 v8 $3 v7 $5}
            /@USER_HOME@/  && /@KEYS_SUB_DIR@/               {$0=$1 v9 $3 v3 $5 v10 $7}     	        
             # missing in headers -> /@MAN_SERVER_PORT@/     {$0=...v11...}
             # missing in headers -> /@WLS_ENV@/             {$0=...v12...}
            /@WEBLOGIC_VERSION@/  			     {$0=$1 v13 $3}

 1' v1="$v1" v2="$v2" v3="$v3" v4="$v4" v5="$v5" v6="$v6" v7="$v7" v8="$v8" v9="$v9" v10="$v10" v11="$v11" v12="$v12" v13="$v13" "$file" > "$path"/"$file"



 done
# [MAKE CHANGES TO THIS SECTION ONLY] ==================================]
# [VARIABLES -----------------------------------------------------------]

SERVER_NAME="@MAN_SERVER_NAME@"
LISTEN_PORT="@MAN_SERVER_PORT@"
UNIX_SERVER_NAME="@UNIX_SERVER@"
BASE_DIR="@BEA_INSTALL_DIR@"
DOMAIN_BASE="${BASE_DIR}/user_projects/@BASE_DOMAIN_NAME@"
DOMAIN_DIR="${DOMAIN_BASE}/@DOMAIN_NAME@"
MANSERVER_DIR="${DOMAIN_BASE}/managedservers"
KEYS_DIR="@USER_HOME@/keys/@BASE_DOMAIN_NAME@/@KEYS_SUB_DIR@"

# [DO NOT MAKE ANY CHANGES BELOW THIS LINE] ============================]
# [CONSTANTS -----------------------------------------------------------]

DOMAIN_NAME="@DOMAIN_NAME@"
PID_FILE="${SERVER_DIR}/server_${SERVER_NAME}.pid"
SERVER_DIR="${MANSERVER_DIR}/${SERVER_NAME}"
SERVER_URL="t3://${UNIX_SERVER_NAME}:${LISTEN_PORT}"
ADMIN_URL="t3://@ADMIN_UNIX_SERVER@:@ADMIN_SERVER_PORT@"
AUDIT_FILE="${DOMAIN_DIR}/logs/audit.txt"
PY_NAME=".forceshutdown.py"
WL_KEY_FILE="${KEYS_DIR}/system-WLKey.secure"
WL_CONFIG_FILE="${DOMAIN_DIR}/system-WLConfig.secure"
WLS_ENV="@WLS_ENV@"
# [MAKE CHANGES TO THIS SECTION ONLY] ==================================]
# [VARIABLES -----------------------------------------------------------]

SERVER_NAME="@MAN_SERVER_NAME@"
LISTEN_PORT="@MAN_SERVER_PORT@"
UNIX_SERVER_NAME="@UNIX_SERVER@"
BASE_DIR="@BEA_INSTALL_DIR@"
DOMAIN_BASE="${BASE_DIR}/user_projects/@BASE_DOMAIN_NAME@"
DOMAIN_DIR="${DOMAIN_BASE}/@DOMAIN_NAME@"
MANSERVER_DIR="${DOMAIN_BASE}/managedservers"
KEYS_DIR="@USER_HOME@/keys/@BASE_DOMAIN_NAME@/@KEYS_SUB_DIR@"

# [DO NOT MAKE ANY CHANGES BELOW THIS LINE] ============================]
# [CONSTANTS -----------------------------------------------------------]

DOMAIN_NAME="@DOMAIN_NAME@"
PID_FILE="${SERVER_DIR}/server_${SERVER_NAME}.pid"
SERVER_DIR="${MANSERVER_DIR}/${SERVER_NAME}"
SERVER_URL="t3://${UNIX_SERVER_NAME}:${LISTEN_PORT}"
ADMIN_URL="t3://@ADMIN_UNIX_SERVER@:@ADMIN_SERVER_PORT@"
AUDIT_FILE=${DOMAIN_DIR}/logs/audit.txt
PY_NAME=".shutdown.py"
WL_KEY_FILE="${KEYS_DIR}/system-WLKey.secure"
WL_CONFIG_FILE="${DOMAIN_DIR}/system-WLConfig.secure"
WLS_ENV="@WLS_ENV@"

Those are the last two headers from the last scripts.

Again discrepancies in your requirements.

#!/bin/bash

cd /path/to/files/

read -p "MAN_SERVER_NAME: "   v1
read -p "BEA_INSTALL_DIR: "   v2
read -p "BASE_DOMAIN_NAME: "  v3
read -p "DOMAIN_NAME: "       v4
read -p "UNIX_SERVER: "       v5
read -p "USER HOME: "         v6
read -p "KEYS_SUB_DIR: "      v7
read -p "MAN_SERVER_PORT: "   v8


path="/path/to/tmp/dir/"

for file in *WLS

 do

   awk -F'@|}|\\${'  '
           		   /SERVER_NAME="@MAN_SERVER_NAME@"/   { $0=$1 v1 $3 }
          		   /LISTEN_PORT="@MAN_SERVER_PORT@"/   { $0=$1 v8 $3 }
           	           /BASE_DIR="@BEA_INSTALL_DIR@"/      { $0=$1 v2 $3 }
            	           /UNIX_SERVER_NAME="@UNIX_SERVER@"/  { $0=$1 v5 $3 }
	                   /DOMAIN_BASE=/ && /\/user_projects\// { p=v2 $3 v3; $0=$1 v2 $3 v3 $5 }
           		   /DOMAIN_DIR="/ && /\/@DOMAIN_NAME@"/  { $0=$1 p $3 v4 $5 }
            	           /MANSERVER_DIR=/ && /\/managedservers"/ { $0=$1 p $3 }
	             	   /KEYS_DIR="@USER_HOME@\/keys\//       { $0=$1 v6 $3 v3 $5 v7 $7 }

                       1'  v1="$v1" v2="$v2" v3="$v3" v4="$v4" v5="$v5" v6="$v6" v7="$v7" v8="$v8" "$file" > "$path"/"$file"
 done