Need to add a numeric & special char to end of the first line

Shell Programming and Scripting
1

Need to add a numeric & special char to end of the first line

Existing file:

12-11-16|11 2016 Jan 12:34:55|03:55|
13-10-16|10 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|
14-10-16|19 2016 Jan 12:34:55|03:55|13-11-16|11 2016 Jan 12:34:55|04:55|
15-10-16|18 2016 Jan 12:34:55|03:55|14-11-16|11 2016 Jan 12:34:55|05:55|
16-10-16|17 2016 Jan 12:34:55|03:55|15-11-16|11 2016 Jan 12:34:55|06:55|
17-10-16|16 2016 Jan 12:34:55|03:55|16-11-16|11 2016 Jan 12:34:55|07:55|

I have used the below command to perform that, but it is working only for string

VAR="12-11-16|11 2016 Jan 12:34:55|03:55|"

sed -i '1s/$/"$VAR"/' file

when am using this for above $VAR which has special char in it, it is failing, is there a way to do this in awk or in sed without redirecting to another file?

2

Hello Joselouis,

Welcome to forums, hope you will enjoy learning and sharing knowledge and learning here. Could you please try following as a starting point and let me know if this helps you.

sed -i '1s/^/'"$s1"'/;1s/$/'"$s1"'/'  Input_file

Where variable named s1 have the value of " and it will provide the output as follows.

sed -i '1s/^/'"$s1"'/;1s/$/'"$s1"'/'  Input_file

Output will be as follows.

"12-11-16|11 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|"
13-10-16|10 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|
14-10-16|19 2016 Jan 12:34:55|03:55|13-11-16|11 2016 Jan 12:34:55|04:55|
15-10-16|18 2016 Jan 12:34:55|03:55|14-11-16|11 2016 Jan 12:34:55|05:55|
16-10-16|17 2016 Jan 12:34:55|03:55|15-11-16|11 2016 Jan 12:34:55|06:55|
17-10-16|16 2016 Jan 12:34:55|03:55|16-11-16|11 2016 Jan 12:34:55|07:55|

NOTE: Though we could use -i option of sed but it is not good practice to use it with large size files, it is better to use redirect option of standard output to a temp file and then rename it to the orginal Input_file.

Thanks,
R. Singh

3

Hi Singh,

Thanks for your reply, sorry it didn't work out, it again failed because of the special characters in it.

4

Hello Joselouis,

Could you please try command as follows to check if there are garbage characters inside your Input_file.

cat -v Input_file

If you see Control M characters in your Input_file then you could anyone of the following options to remove them.

i- use dos2unix utility to remove control M(garbage characters) inside Input_file(Also your box should have utility installed if not as a root uer you could do it.)
ii- use tr command, just an example as follows too.

tr -d '\r' < Input_file

iii- Use awk 's subsitutation functionality i.e as follows(haven't tested though).

awk '{gsub('\r',X,$0);print}' Input_file

Once you are done with removing the garbage characters in your Input_file and then you could my suggestion provided into POST#2. kindly do so and do let me know if this helps you.

Thanks,
R. Singh

5

Hi,

I don't think it is problem with quotes in sed instead of variable declaration.

Here is the example:

A="Hi"; echo $A
Hi
A=\"Hi\"; echo $A
"Hi"
A="\"12-11-16|11 2016 Jan 12:34:55|03:55|\""
echo "13-10-16|10 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|" | sed -e "s/$/$A/"

Gives output:

If this is not your expected output, please post your input , expected output and variable value. Always write what was the error / output deviation from desired output etc .

@ Ravinder : you can use sed -i.bak to create a backupfile (.bak or any extension) first before making substitution in original file automatically ( and can avoid interim file , rename etc ) .

1 Like
6

In the below input file, end of the first line I want to add the below variable value

VAR="12-11-16|11 2016 Jan 12:34:55|03:55|"

Input_file:
12-11-16|11 2016 Jan 12:34:55|03:55|
13-10-16|10 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|
14-10-16|19 2016 Jan 12:34:55|03:55|13-11-16|11 2016 Jan 12:34:55|04:55|
15-10-16|18 2016 Jan 12:34:55|03:55|14-11-16|11 2016 Jan 12:34:55|05:55|
16-10-16|17 2016 Jan 12:34:55|03:55|15-11-16|11 2016 Jan 12:34:55|06:55|
17-10-16|16 2016 Jan 12:34:55|03:55|16-11-16|11 2016 Jan 12:34:55|07:55|

Desire output:

12-11-16|11 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|
13-10-16|10 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|
14-10-16|19 2016 Jan 12:34:55|03:55|13-11-16|11 2016 Jan 12:34:55|04:55|
15-10-16|18 2016 Jan 12:34:55|03:55|14-11-16|11 2016 Jan 12:34:55|05:55|
16-10-16|17 2016 Jan 12:34:55|03:55|15-11-16|11 2016 Jan 12:34:55|06:55|
17-10-16|16 2016 Jan 12:34:55|03:55|16-11-16|11 2016 Jan 12:34:55|07:55|

Command and error:

sed -i "1s/$/"$VAR"/" Input_file 
Error: sed: -e expression #1, char 20: unknown option to `s'

Hi Singh, I tried your three fixes and didn't work

Hi Greet sed, yes that is not my expected out, I have given my expected output above (tried with escap "\" and didnt work out) and want to edit in a file, I want to add the

$VAR

value in end of the first line

7

You have too many quotes (leaving the expansion of $VAR unquoted. Change:

sed -i "1s/$/"$VAR"/" Input_file

to:

sed -i "1s/$/$VAR/" Input_file
8

Or more general (not needed here but useful in complex sed scripts)

sed -i '1s/$/'"$VAR"'/' file

Again the $var is in " " letting the shell expand it but not see+treat any special characters.
Of course the / delimiter in sed may not occur in the $var string, and no & character that is special in the replacement part of a sed substitution.

9

Hi Don/MadeinGermany, I have already tried these, but they are giving me the below error.

sed -i "1s/$/$VAR/" Input_file
sed -i "1s/$/"$VAR"/" Input_file
sed -i '1s/$/"$VAR"/' Input_file
sed -i "1s/$/'"$VAR"'/" Input_file

Error:

sed: -e expression #1, char 20: unknown option to `s'

Is there a way in awk to get the below desired output, so that I will redirect the output to another file

12-11-16|11 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|
13-10-16|10 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|
14-10-16|19 2016 Jan 12:34:55|03:55|13-11-16|11 2016 Jan 12:34:55|04:55|
15-10-16|18 2016 Jan 12:34:55|03:55|14-11-16|11 2016 Jan 12:34:55|05:55|
16-10-16|17 2016 Jan 12:34:55|03:55|15-11-16|11 2016 Jan 12:34:55|06:55|
17-10-16|16 2016 Jan 12:34:55|03:55|16-11-16|11 2016 Jan 12:34:55|07:55|
10

If all four of the commands:

sed -i "1s/$/$VAR/" Input_file
sed -i "1s/$/"$VAR"/" Input_file
sed -i '1s/$/"$VAR"/' Input_file
sed -i "1s/$/'"$VAR"'/" Input_file

are giving you the same output, you are not running these four commands!

What operating system are you using?

What shell are you using?

Please show us the exact output you get from running the following four commands:

set -xv
VAR="12-11-16|11 2016 Jan 12:34:55|03:55|"
sed -i "1s/$/$VAR/" Input_file
set +xv
11

Hello Joselouis,

Could you please try following and let me know if this helps you.

awk -vvar="$VAR" 'NR==1{print $0 var;next} 1'  Input_file

Output will be as follows.

12-11-16|11 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|
13-10-16|10 2016 Jan 12:34:55|03:55|12-11-16|11 2016 Jan 12:34:55|03:55|
14-10-16|19 2016 Jan 12:34:55|03:55|13-11-16|11 2016 Jan 12:34:55|04:55|
15-10-16|18 2016 Jan 12:34:55|03:55|14-11-16|11 2016 Jan 12:34:55|05:55|
16-10-16|17 2016 Jan 12:34:55|03:55|15-11-16|11 2016 Jan 12:34:55|06:55|
17-10-16|16 2016 Jan 12:34:55|03:55|16-11-16|11 2016 Jan 12:34:55|07:55|

Also you could see in above code of awk have used -vvar="$VAR" which means in awk we have to define the shell variable values into awk 's variables. You could re-direct this command's output to a temp_file and then could change it to Input_fie again. If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk .

Thanks,
R. Singh

2 Likes
12

Thank you R.Singh.. code has worked :slight_smile: