Find and replace using sed

Shell Programming and Scripting
1

File

1,2,33,C,B 
3,5,66,K,R
1,2,33,H,M 
3,5,66,M,C
6,9,66,J,F

I will use the below command to find and replace in sed, where I'm using variable to find pattern.

while read line 
do 
sed 's/$line/77/' file 
done<inputfile

But here I need to find value in column 3 and want to replace value in column 5.

Output like:

1,2,33,C,B 
3,5,66,K,77 
1,2,33,H,M 
3,5,66,M,77
6,9,66,J,77

how to do this in sed?

2

Will you please explain on what basis you replaced R C and F in column 5 with 77 ? Whether it's column5 = column 3 value + 11 ? If yes why not for B and M ?

3

For field seperated columns, use awk .

 awk -F, '{if($3=="66") $5=77;print $0}'  OFS="," file
4

Hello,

1 more awk approach without OFS

awk -F"," '$3==66 {$5=77} {print $0}' file_name

Output will be as follows.

1,2,33,C,B
3 5 66 K 77
1,2,33,H,M
3 5 66 M 77
6 9 66 J 77

Thanks,
R. Singh

5

Greet sed : Command not replacing the value, just displaying the input.

Akshay:
I'm not doing any column5 = column 3 value + 11

Just finding pattern in column3 and if pattern founds, replace value on the same line in column5.

6

Thanks Roozo

You may try if replacement of value in column5 is based on column3 value

$ awk -F, '$5 = $3 == 66 ? 77 : $5 ' OFS=, file

OR

$ awk -F, '$5 = $3 == find ? replace : $5 ' OFS=, find=66 replace=77  file
7

@ Roozo:
All the solutions written here so far has to be redirected to new file otherwise it will be displayed in terminal as you said.

Like this:

awk '{code}' file > output
8

In general awk is much better suited for these kind of substitutions, but if for some reasen sed is still your prefered choice you could try something like this:

sed 's/\([0-9]*,[0-9]*,66,[A-Z]*,\).*/\177/' file
9

Thanks all,

Akshay, find and replace is working well, when I pass multiple inputs using while loop having concerns. The pattern lines are getting eliminated.

file
1,2,33,C,B 
3,5,66,K,R
1,2,33,H,M 
3,5,66,M,C
6,9,66,J,F
4,8,55,H,P
5,7,44,N,U
 
input.txt
66
55
 
CODE:
awk '{print $1}' input.txt | while read line 
do 
awk -F, '$5 = $3 == find ? replace : $5 ' OFS=, find=$line replace=77  file >>Output.txt
done<inputfile
 
Output.txt
1,2,33,H,M   --->loop 1 
4,8,55,H,P    --->loop 1 
5,7,44,N,U   --->loop 1 
1,2,33,C,B    --->loop 2
3,5,66,K,R   --->loop 2
1,2,33,H,M   --->loop 2 
3,5,66,M,C  --->loop 2 
6,9,66,J,F   --->loop 2 
5,7,44,N,U  --->loop 2

And as we know it will append into the Output.txt file.

Is there a way to get the below output, when passing values in while loop?

1,2,33,C,B 
3,5,66,K,77
1,2,33,H,M 
3,5,66,M,77
6,9,66,J,77
4,8,55,H,77
5,7,44,N,U
10

You are appending in every iteration below one does your job

Try :

$ awk -F,  'FNR==NR{A[$1]++;next}A[$3]{$5=replace}1' replace=77 OFS=\, input.txt file >output.txt

$ cat output.txt
1,2,33,C,B 
3,5,66,K,77
1,2,33,H,M 
3,5,66,M,77
6,9,66,J,77
4,8,55,H,77
5,7,44,N,U

awk is the best tool for this type of work, if you still wants to do using looping you can do something like this, its bit lengthy even

#!/bin/bash

replace_file=input.txt
input_file=file
out_file=output.txt
replace=77

n=0
while read main ; do

       while read sub; do

        main=$(echo $main | sed 's/\(.*,.*,'$sub',.*,\).*/\1'$replace'/')
   # OR main=$(echo $main | awk -F, -v val=$sub '$3==val{$5=replace}1' replace=$replace OFS=\, )

       done < "$replace_file"

       [ $n -eq 0 ] && echo $main >$out_file || echo $main >>$out_file

       n=$((n+1))

done < "$input_file"
1 Like
11

Akshay, thanks it working as expected :slight_smile:

12

It is more efficient to use $3 in A instead of A[$3] :

awk 'FNR==NR{A[$1]; next} $3 in A{$5=replace}1' replace=77 FS=, OFS=, input.txt file

Then there is no need for A[$1]++ which saves an addition operation for every line. Also, using A[$3] creates an extra empty array element for every $3 that does not exist in array A while reading the second file..

1 Like
13 
file
1,2,33,C,B 
3,5,66,K,R
1,2,33,H,M 
3,5,66,M,C
6,9,66,J,F
4,8,155,H,P
5,7,44,N,U
 
input.txt
066
55

When I passing my input with 0 in beginning, it searching for the exact "066" and not considering the below lines in file to replace

3,5,66,K,R
3,5,66,M,C
6,9,66,J,F

And when I passing only 66 in file input.txt, when the file contain 066 it should consider for replacing

input.txt
66
55
 
file
1,2,33,C,B 
3,5,066,K,R
1,2,33,H,M 
3,5,066,M,C
6,9,066,J,F
4,8,155,H,P
5,7,44,N,U

And also my input file has 55 which do not want to match with 155 "4,8,155,H,P" in file
I used double quotes for $3 in the below but isn't gave me o/p, how to achieve this with

awk -F,  'FNR==NR{A[$1]++;next}A[$3]{$5=replace}1' replace=77 OFS=\, input.txt file >output.txt

or

awk -F,  'FNR==NR{A[$1]++;next}A[$3]{$5=replace}1' replace=77 OFS=\, input.txt file >output.txt
14

Hi, try this fix:

awk 'FNR==NR{A[$1+0]; next} $3+0 in A{$5=replace}1' replace=77 FS=, OFS=, input.txt file
1 Like
15

Thanks Working:), can you explain the command flow $+0 and $3+0 ?

16

By adding 0 awk treats array index $1 as numeric value, if you do not add 0 then string

Example :

$ awk 'FNR==NR{A[$1]; next}END{for(i in A)print i }' input
55
066

$ awk 'FNR==NR{A[$1+0]; next}END{for(i in A)print i}' input
55
66

# string
$ cat <<eof | awk 'FNR==NR{A[$1]; next}END{for(i in A)print i }'
text
12
eof

text
12

# Numeric
$ cat <<eof | awk 'FNR==NR{A[$1+0]; next}END{for(i in A)print i }'
text
12
eof

12
0     -----> text is treated as 0 here

In fact adding numeric value to string or anything, awk does numeric conversion, and treats it as numeric as far as I know.

1 Like
17 
#!/usr/bin/env perl
use strict;
use warnings;

while(<>){
	my @sp = split(/,/ , $_);
	$sp[4] = "77\n" if $sp[2] == "66" ;	
	print join(",", @sp); 
}
18

Thanks,

awk 'FNR==NR{A[$1+0]; next} $3+0 in A{$5=replace}1' replace=77 FS=, OFS=, input.txt file

How to move the replaced lines to the beginning of the file, like the below

3,5,066,K,77
3,5,066,M,77
6,9,066,J,77
1,2,33,C,B 
1,2,33,H,M
4,8,155,H,P
5,7,44,N,U
19

Try:

awk 'FNR==NR{A[$1+0]; next} !($3+0 in A){p=p $0 ORS; next} {$5=replace}1 END{printf "%s",p}' replace=77 FS=, OFS=, input.txt file

or, perhaps more legible

awk '
  FNR==NR{
    A[$1+0]
    next
  }
  !($3+0 in A){
    p=p $0 ORS
    next
  }
  {
    $5=replace
  }
  1 
  END{
    printf "%s",p
  }
' replace=77 FS=, OFS=, input.txt file
20

OR

Try

if second file size is too big then you may have to face memory problem

$ awk 'FNR==NR{A[$1+0]; next}{if($3+0 in A){$5=replace;print}else{last[++i]=$0}}END{for(j=1;j<=i;j++)print last[j]}' replace=77 FS=, OFS=, input file

or just pipe and sort

1 Like