sed replace pattern

I have a file with multiple lines, all in the same format. For each line, I need to replace the sequence of digits after the last : with a new value, but keep the single quote at the end of the line.

Example:
Input: ( two lines of file)

Name: 'text1:200/text2:1.2.3.4'
Name2: 'text3:200/text3:1.2.3.17'

New Value to use instead of 1.2.3.4:

1.2.3.x or 4.5.6.7 ( the first 3 digits don't always match)

Output:

Name: 'text1:200/text2:1.2.3.x'
Name2: 'text3:200/text3:1.2.3.x'

Can I use sed to do this?
Even if I can just replace the last digit(4) after the last dot with a new value (x) that would work for now....

Thanks!

Let the source pattern is "WINDOWS".
Target pattern is "UNIX"
File name is Pattern.txt
Then the command for replacing WINDOWS with UNIX is

1:

sed 's/WINDOWS/UNIX/' Pattern.txt

It will replace the first occurrence of WINDOWS with UNIX.

2:

sed 's/WINDOWS/UNIX/g' Pattern.txt

It will substitute the pattern globally.

What do you mean with

? How and where do the 4.5.6.7 come into play?
And, why does

replace 1.2.3.17 ?

If you got a single value to replace portions of the input line, sed might do. If there's more to be done, advanced tools like awk or perl might be the better choice.

Hi,

can you try the following and see if it helps ?

sed -re 's/(.*)\.[0-9]{1,2}/\1.x/' file

other way:

sed -re 's/\.[0-9]{1,2}'"'"'/.X/' file

you might need to adapt depends on your requirement if your input is different.

1 Like

thanks greet_sed!
in the second option, the value I want to replace is a variable, not X
so $val=45

I tried this but doesnt resolve $val :

sed -re 's/\.[0-9]{1,2}'"'"'/.$val/' file

so I used double quotes for sed:

sed -re "s/\.[0-9]{1,2}'"'"'/.$val/" file

but there seems to a mismatch, it doesnt work wither.

I also want to write back to the same file, so

sed -i -r

should work right?

---------- Post updated at 11:41 AM ---------- Previous update was at 11:40 AM ----------

thanks greet_sed!:b::slight_smile:
in the second option, the value I want to replace is a variable, not X
so $val=45

I tried this but doesnt resolve $val :

sed -re 's/\.[0-9]{1,2}'"'"'/.$val/' file

so I used double quotes for sed:

sed -re "s/\.[0-9]{1,2}'"'"'/.$val/" file

but there seems to a mismatch, it doesnt work wither.

I also want to write back to the same file, so

sed -i -r 

should work right?

Hi,

Can you try the below one ?

A=45;
sed -re "s/(.*)\.[0-9]{1,2}/\1.$A/" file

use -i to do infile edit.
use -i.bak to create backup before editing.

Hello Beginner101,

Could you please try following too and let me know if this helps you.

A=45;
sed -re 's/(.*)\.[0-9]{1,2}/\1.'"${A}"'/'  Input_file

Thanks,
R. Singh

If the \. is within the ( ) then the \1 puts it back in.

sed -r 's/(.*\.)[0-9]{1,2}/\1'"$val"'/' file

The -r is ERE in GNU sed. A Unix sed needs the standard RE or BRE:

sed 's/\(.*\.\)[0-9]\{1,2\}/\1'"$val"'/' file

Thanks!
Slight modification to the original problem:

So previously I wanted to replace the last digit before the single quote at end of line with a new value, but now I want to only do it based on the previous digits and a matching pattern.

If any(part of) line in file matches a certain pattern, replace the value after the last dot (and keep the single quote at end of line-same as before) with the value in pattern.
Can I do this using sed?

example:
Pattern to match=name1: 1.2.300.Y (or could be name1: 1.300.Y , so Y can be any number, I don't care what it is at this point).

All lines will be in the same format and ideally there will only be one line containing 'name1'.

Pattern to match= name1:1.2.300.Y

Original file:

person1_desc: 'info:200/text/name1:1.2.300.X'
person1_desc: 'info:200/text/name1:1.2.400.A'
person1_desc: 'info:200/text/name1:5.6.700.B'
person2_desc: 'info:200/text/name2:1.2.300.C'

Output file:

person1_desc: 'info:200/text/name1:1.2.300.Y'
person1_desc: 'info:200/text/name1:1.2.400.A'
person1_desc: 'info:200/text/name1:5.6.700.B'
person2_desc: 'info:200/text/name2:1.2.300.C'

So only the first line was a match and updated, 'X' was replaced with 'Y'. Line 4 wasn't changed( name2)

Pattern I'm looking for:

name1:1.2.300.Y

In other words:
{string}:{ a combination of one or more numbers and dots, not always 2 dots, but grouped together}.{one or more digits}

Or if I could also extract each field above from the original file into variables:

Var1=name1
Var2=1.2.300
Var3=Y

If it helps, my pattern-to-look-for could also be:

info:200/text/name1:1.2.300.Y

Instead of just:

name1:1.2.300.Y

I hope I explained it well, sorry if I might have over done it!
Thanks!!

---------- Post updated at 10:49 PM ---------- Previous update was at 10:48 PM ----------

Yes that works, thank you!
Slight modification to original problem:

previously I wanted to replace the last digit before the single quote at end of line with a new value, but now I want to only do it based on the previous digits and a matching pattern.

If any(part of) line in file matches a certain pattern, replace the value after the last dot (and keep the single quote at end of line-same as before) with the value in pattern.
Can I do this using sed?

example:
Pattern to match= name1:1.2.300.Y (or could be name1:1.300.Y, so Y can be any number, I don't care what it is at this point).

All lines will be in the same format and ideally there will only be one line containing 'name1'.

Pattern to match= name1:1.2.300.Y

Original file:

person1_desc: 'info:200/text/name1:1.2.300.X'
person1_desc: 'info:200/text/name1:1.2.400.A'
person1_desc: 'info:200/text/name1:5.6.700.B'
person2_desc: 'info:200/text/name2:1.2.300.C'

Output file:

person1_desc: 'info:200/text/name1:1.2.300.Y'
person1_desc: 'info:200/text/name1:1.2.400.A'
person1_desc: 'info:200/text/name1:5.6.700.B'
person2_desc: 'info:200/text/name2:1.2.300.C'

So only the first line was a match and updated, 'X' was replaced with 'Y'. Line 4 wasn't changed( name2)

Pattern I'm looking for:

name1:1.2.300.Y

In other words:
{string}:{ a combination of one or more numbers and dots, not always 2 dots, but grouped together}.{one or more digits}

Or if I could also extract each field above from the original file into variables:

Var1=name1
Var2=1.2.300
Var3=Y

If it helps, my pattern-to-look-for could also be:

info:200/text/name1:1.2.300.Y

Instead of just:

name1:1.2.300.Y

I hope I explained it well, sorry if I might have over done it!
Thanks!!

PLEASE exercise some care when editing a post NOT to duplicate its contents!

You best have two strings, one is the search pattern and one is the replacement value.

#!/bin/sh
pattern=$1
newval=$2
if [ $# -ne 2 ]; then
  echo "usage: $0 'RE pattern' newval"
  exit 1
fi
sed 's/\(.*'"$pattern"'\.\)[0-9]\{1,\}/\1'"$newval"'/' file

Here the given pattern must reach to the last . (before the last number). For example you can run your script with
./scriptname 'name1:1\.2\.300' 199 or ./scriptname ':1\.2\.300' 199 but not ./scriptname 'name1:1\.2\.' 199 .
The following modification searches the pattern in the whole line:

sed '/'"$pattern"'/ s/\(.*\.\)[0-9]\{1,\}/\1'"$newval"'/' file

This requires you to end the given pattern with a \. (otherwise there is a risk you match the first part of the number).
As an alternative you can put the trailing \. into the sed code

sed '/'"$pattern"\.'/ s/\(.*\.\)[0-9]\{1,\}/\1'"$newval"'/' file

and you must *not* give the trailing \. .

Hi,

Can you please try and adapt if your input/output changes ?

cat file
person1_desc: 'info:200/text/name1:1.2.300.X'
person1_desc: 'info:200/text/name1:1.2.400.A'
person1_desc: 'info:200/text/name1:5.6.700.B'
person2_desc: 'info:200/text/name2:1.2.300.C'
person2_desc: 'info:200/text/name1:1.298.X'
sed -re 's/([a-z]+1:[0-9].[0-9]?.?[0-9]{3}.)(X)/\1Y/' file

Thanks

,
Could you please explain how the sed is working? so I can understand how to make slight modificiations :slight_smile:

Also, my input will actually be the whole line:

info:200/text/name1:1.2.300.Y

So based on that I need to do:

1.get pattern

pattern=info:200/text/name1:1.2.300

2.escape the / in the pattern (right?)

pattern=info:200\/text\/name1:1.2.300

3.put pattern in single quptes

pattern='info:200\/text\/name1:1.2.300'

4.get newval

newval=Y

and then call the sed
how can I extract all those values?

Thanks again!

Hi,
Did you try the code posted in my previous post ?

"how can I extract all those values"
What do you mean by that ?
Please provide input & expected output.

I am confused also because my script proposal was for an input file where a number is to be substituted, as you specified in your post#1.
Now it has become a letter.?

Hi guys
sorry for the confusion. Ill explain.

@MadeInGermany:
the solution you provided with pattern/newval works perfectly fine.
I can modify "pattern" to be any portion of the string in each line, and the last digit will be replaced.

THe only change right now is I want to get the "pattern" and "newval" parameters required to do the sed by parsing a third variable, instead of defining them explicitly. The main problem seems to be with evaluating the vars.

Right now if I specify everything on the command line it works:

pattern='info:200\/text\/name1:1.2.300'  

-->(im escaping the / and im definning it in single quotes )

newval=8
echo " $pattern "

-> info:200\/text\/name1:1.2.300

echo " $newval "

--> 8

and call sed:

sed '/'"$pattern"\.'/ s/\(.*\.\)[0-9]\{1,\}/\1'"$newval"'/' file

input (from file):

person1_desc: 'info:200/text/name1:1.2.300.X'

output:

person1_desc: 'info:200/text/name1:1.2.300.8'

This is great. So now I will try to get the values from a new variable called $input_line

input_line=info:200/text/name1:1.2.300.8

parse it:

pattern_2=$( echo $input_line | sed 's/\.[^.]*$//' ) 

--> info:200/text/name1:1.2.300

newval_2=$( echo  $input_line | sed 's/.*\.//' )  

--> 8

echo " $pattern_2 " 

--> info:200/text/name1:1.2.300

echo "$newval_2 "

--> 8

so now if I call the exact same sed command, it doesnt work:

sed '/'"$pattern"\.'/ s/\(.*\.\)[0-9]\{1,\}/\1'"$newval"'/' file

I get:

sed: -e expression #1, char 33: extra characters after command

so thats why I want to

  1. escape the slash in $pattern_2
  2. not sure if I need to put $pattern_2 in single quotes?

Hi,

I am sorry still it is not clear that what you are trying to achieve .

See if it helps :

pattern='info:200/text/name1:1.2.300'
pattern=$(echo $pattern | sed -e 's,/,\\/,g')
echo $pattern
info:200\/text\/name1:1.2.300

I am still a bit confused: when you say .X you seem to mean that X is a number...
The / has no special meaning in an RE (=regular expression). But my sed scripts used the / delimiter. The following is with a different delimiter, #

sed '\#'"$pattern"\.'# s#\(.*\.\)[0-9]\{1,\}#\1'"$newval"'#' file

Perhaps the following is a little more precise, because it requires the search pattern to be placed left from the replacement area.

sed 's#\('"$pattern"'.*\.\)[0-9]\{1,\}#\1'"$newval"'#' file

Now you do not need to escape the / in the search pattern (instead need to escape a # ).
But you *must* escape the . character! In a RE it matches any character. For example pattern="2.0/text" would find 200/text . But pattern="2\.0/text" only finds 2.0/text .
--
You want to split the string at the last dot. You can use shell-builtins for that

input_line="info:200/text/name1:1.2.300.8"
pattern_2=${input_line%.*}
echo "$pattern_2"
newval_2=${input_line##*.}
echo "$newval_2"
sed 's/./3/6;s/./3/16;s/./17/24' sedd1.txt
# cat mytst
Name: 'text1:200/text2:1.2.3.4'
Name2: 'text3:200/text3:1.2.3.17'
# sed 's/[^:]*.$/whatever'"'"'/g' mytst
Name: 'text1:200/text2:whatever'
Name2: 'text3:200/text3:whatever'
# sed 's/[^.]*.$/whatever'"'"'/g' mytst
Name: 'text1:200/text2:1.2.3.whatever'
Name2: 'text3:200/text3:1.2.3.whatever'

@Beginner101
Note :
a) sometimes it's easier for people to understand what you need when you provide a well-chosen example of input and output file, rather than long explainations... :o
b) the dot before the dollar signe is useless here, but i leave it just to keep in mind that this last char matched (which is here a simple quote), is a little special and need therefore a special threatment.

---------- Post updated at 16:29 ---------- Previous update was at 13:02 ----------

Yep this is due to the fact that your pattern contains some / which get parsed.

In order to avoir tedious "escaping escape" gym ... you may want to give a try using ! as reference character in the s!pattern1!pattern2! expression rather than / so that the slash will be taken as litteral.

1 Like