Replace Block

I'd like to replace content within the markers "(div id=m1)" and "(/div)". Between the markers there are several lines of text but no nested brackets (so no parser is required).

In the beginning I thought this to be an easy task like:

"sed s/(div id=m1).*(/div)/(div id=m1)$replacetextfromfile(/div)/"

But I failed (eg. sed's greedyness, mulitline problem; i tried preprocessors like Gema. Is it really required to write an awk script?).

Question: is there a straightforward way to replace text between some xml-brackets by text read from file?

Thanks!

If your replacement text contains the same character you are using as a delimiter, you need to escape it. In the example you provided, you included a closing DIV element, i.e. </div>. Try using <\/div> instead.

I have used multi-line replacements with sed, but I tend to use single quotes and break out when I need to insert variables. For example,

var=Search
sed 's/searchText/new'${var}'Text/'

Thanks for your comments. But the main problem: how do you define "searchText"? It has to be a non-greedy regular expression spanning several lines because the text between both delimiters does not matter.

Advanced SED Commands (?)

Unfortunately no solution there.

Perhaps there are more tipps how to

• replace a block of multiline text, surrounded by xml-brackets?

Hi

For example let my file is like below

[root@rhnserver ~]# cat testfile
......................
............................
................................
"(div idm1)"
The first line
The second line
This third test
This fourth test
The last line
"(/div)"
......................
............................
................................

I take line number between "(div idm1)" and "(/div)"
My goal is learn the how many line to delete..

linecount=$( (sed -n '/"(div idm1)"/,/"(\/div)"/{;/"(div idm1)"/!p;}' testfile | sed -n '$!p' | wc -l) )

Now I remove between lines..

while [ $(( linecount -= 1 )) -gt -1 ]
do
sed -i '/^"(div idm1)"/ {n;d;q}' testfile
done

And let the other file is look like

[root@rhnserver ~]# cat file_to_add
test
test12
deneme
test123

Finally add to lines between "(div idm1)" and "(/div)" from "file_to_add" file.

#Add Other File to My Org File
while read line
   do
     sed -i "/^\"(\/div)\"/ i$line" testfile
   done < file_to_add
 
[root@rhnserver ~]# cat testfile
......................
............................
................................
"(div idm1)"
test
test12
deneme
test123
"(/div)"
......................
............................
................................

I escape I shall sleep

Regards
ygemici

Hello, Dieter5, and welcome to the forums.

Writing a sed script to replace a multi-line span of text which is terminated by a sequence of characters which does not occur within the text to be replaced is easy. The replacement text may be the more complicated part. The replacement text cannot contain any characters that sed treats specially in replacement text, such as an ampersand, a backslash, the s command's delimiter, and perhaps others that i'm overlooking at this moment. Is it possible that the replacement text could contain any such characters? If so, then it will need to be escaped properly.

If you would like me to spend some of my time putting together a possible solution, please answer those questions and provide some sample input (including replacement text) and the desired output, so that I can test without having to contrive input data which may not match your requirements.

Regards,
Alister

It contains a lot of punctuation and other special characters..

"(div idm1)" and "(/div)" are a line

 
[root@sistem1lnx ~]# cat testfile
This is a test
This is test
"(div idm1)"
The first line\'/'\.\'.[];][[+_)_()*&^&^&%^#%@#$!~[][[][=-00-99889763412`>?>`,';\';][p][{}]12`//'-0
This is a test
This is a test
This is a test
This is a test
The last line
"(/div)"
This is a test
This is test
......................
............................
................................

the other file contains punctuation and other special characters too..

[root@sistem1lnx ~]# cat file_to_add
tetreawd0-9-1212e312?"?{}_=--=0-00908][]989*()*&*&^&^%^$%#$$!@#!~@~`\'\/[;[][[]:
test12asas9ie23U&^I*&)*()(_*%&^$!@#~@~@~@~#$^%*&Y)(*_(_*()+)_+_
deneme34534tgergeg56u&*(*)()*_+*i�.",i22414"31<F2><F4>
test123�!'^'^�!'^+%&/())=?_�;:��><>�#${[}\|�`|<z<z:�,i

linecount=$( (sed -n '/"(div idm1)"/,/"(\/div)"/{;/"(div idm1)"/!p;}' testfile | sed -n '$!p' | wc -l) )
while [ $(( linecount -= 1 )) -gt -1 ]
  do
    sed -i '/^"(div idm1)"/ {n;d;q}' testfile
  done
  while read line
     do
       sed -i "/^\"(\/div)\"/ i$line" testfile
     done < file_to_add

[root@sistem1lnx ~]# cat testfile
This is a test
This is test
"(div idm1)"
tetreawd0-9-1212e312?"?{}_=--=0-00908][]989*()*&*&^&^%^$%#$$!@#!~@~`'/[;[][[]:
test12asas9ie23U&^I*&)*()(_*%&^$!@#~@~@~@~#$^%*&Y)(*_(_*()+)_+_
deneme34534tgergeg56u&*(*)()*_+*i�.",i22414"31<F2><F4>
test123�!'^'^�!'^+%&/())=?_�;:��><>�#${[}|�`|<z<z:�,i
"(/div)"
This is a test
This is test
......................
............................
................................

First example Ok!!

for another test
;;;; my testfile

"(div idm1)" and "(/div)" are in a line which contains punctuation and other special characters..

[root@sistem1lnx ~]# cat testfile
This is a test
This is test
"(div idm1)"The first line\'/'\.\'.[];][[+_)_()*&^&^&%^#%@#$!~[][[][=-00-99889763412`>?>`,';\';][p][{}]12`//'-0
This is a teste$%@#$!@#$!^$^&*))_\'/'/[[;;[,;,\z\d
This is a teste)_()&*&*%#$!~#!!~##!@#@#$#/
This is a test%^%<>?:"{P?{KI@#!@#!@!$~0[-[-][\[{}
This is a test3224234354()(_*)?.,.';';:::'\\]..][}{}312wefef
The last line$%@#$!@#$!^$^&*))_\'/'/[[;;[,;,\z\\\||||><><ALS[;12-3445]"(/div)"
This is a test
This is test
......................
............................
................................

 
sed -i 's/"(div idm1)"/"(div idm1)"\n/' testfile ; sed -i 's/"(\/div)"/\n"(\/div)"/' testfile
linecount=$( (sed -n '/"(div idm1)"/,/"(\/div)"/{;/"(div idm1)"/!p;}' testfile | sed -n '$!p' | wc -l) )
while [ $(( linecount -= 1 )) -gt -1 ]
do
sed -i '/^"(div idm1)"/ {n;d;q}' testfile
done
while read line
do
sed -i "/^\"(\/div)\"/ i$line" testfile
done < file_to_add
 
linecountaddfile=$( (cat file_to_add | wc -l)) # Other file line count for last line editing on testfile (my orginal file)
 
#Howmany line to read next line for sed and sed process on the org file
 
commandx=""
x="n;" # for read next line
 
while [ $(( linecountaddfile -= 1 )) -gt -1 ]
do
commandx="$commandx$x";
echo $linecount "$commandx"
done
 
# four line (4) in file_to_add so $commandx="n;n;n;n; "
 

# And for "(/div)" editing so to merge last line
sed -i " /(div idm1)/{ $commandx N; /\n/ {s///;} } " testfile
 
Now testfile is like seem

cat testfile
This is a test
This is test
"(div idm1)"
tetreawd0-9-1212e312?"?{}_=--=0-00908][]989*()*&*&^&^%^$%#$$!@#!~@~`'/[;[][[]:
test12asas9ie23U&^I*&)*()(_*%&^$!@#~@~@~@~#$^%*&Y)(*_(_*()+)_+_
deneme34534tgergeg56u&*(*)()*_+*i�.",i22414"31<F2><F4>
test123�!'^'^�!'^+%&/())=?_�;:��><>�#${[}|�`|<z<z:�,i"(/div)"
This is a test
This is test
......................
............................
................................

# more after "(div idm1)" editing so merge first line between our patterns ("(div idm1)" and "(/div)" )

 
sed -i '/"(div idm1)"/{N; /\n/ {s///;} } ' testfile

Now testfile is like seem

 
 
[root@sistem1lnx ~]# cat testfile
This is a test
This is test
"(div idm1)"tetreawd0-9-1212e312?"?{}_=--=0-00908][]989*()*&*&^&^%^$%#$$!@#!~@~`'/[;[][[]:
test12asas9ie23U&^I*&)*()(_*%&^$!@#~@~@~@~#$^%*&Y)(*_(_*()+)_+_
deneme34534tgergeg56u&*(*)()*_+*i�.",i22414"31<F2><F4>
test123�!'^'^�!'^+%&/())=?_�;:��><>�#${[}|�`|<z<z:�,i"(/div)"
This is a test
This is test
......................
............................
................................

The result is succesfull

Best Regards
ygemici

Hi, Dieter5.

Apologies for the long post.

I think that we often see a tool and a few features, and we think it will fit for a solution to a problem. It's possible that we see, for example, that sed has a range selection for lines:

line1,line2

or

/pattern1/,/pattern2/

and we think that we can perform some task based on that. However,we may be over-looking how a stream editor works.

The sed command, in normal operation, reads a single line, and applies all the operations on that line that are appropriate. So, if an operation has a range selection that includes the current line, then the operation proceeds. When all the operations are completed, the line is written out, and the next line is read in, operated upon, etc. So there are constraints on what can be done, and certainly on what can be done easily.

On the other hand, awk is flexible tool that may seem like it takes longer to master, but most people who take that time find it to be well spent.

The one advantage that sed has over awk is in the area of efficiency -- sed is 1/6 the size of awk (machine efficiency), and simple substitutions are easily coded (human efficiency).

I don't know if the code from ygemici is the best sed solution, but it is certainly better than I could do without a lot of extra work.

A few simple awk rules in the core of the solution below, seem simple and straight-forward. I'm sure that the awk can be improved upon, but this solution works as it is.

#!/usr/bin/env bash

# @(#) s1	Demonstrate block-replacement, preserve delimiters, awk.

# Uncomment to run script as external user.
# export PATH="/usr/local/bin:/usr/bin:/bin"
# Infrastructure details, environment, commands for forum posts. 
set +o nounset
pe() { for i;do printf "%s" "$i";done; printf "\n"; }
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe ; pe "Environment: LC_ALL = $LC_ALL, LANG = $LANG"
pe "(Versions displayed with local utility \"version\")"
c=$( ps | grep $$ | awk '{print $NF}' )
version >/dev/null 2>&1 && s=$(_eat $0 $1) || s=""
[ "$c" = "$s" ] && p="$s" || p="$c"
version >/dev/null 2>&1 && version "=o" $p specimen awk
set -o nounset

FILE1=${1-data1}
shift
FILE2=${1-data2}

# Sample data files with head / tail if specimen fails.
pe
specimen 7 $FILE1 $FILE2 \
|| { pe "(head/tail)"; head -n 5 $FILE1 $FILE2; pe " ||" ;\
     tail -n 5 $FILE1 $FILE2; }

pe
pe " Results:"
awk -v inside="$FILE2" '
BEGIN                     { looking = 0 }
looking == 1 && /(\/div)/ { looking = 0 ;
                            while ( ( getline t1 < inside > 0 ) ) {
                              print t1
                            }
                            close(inside)
                            print
                            next
                          }
/(div id=m1)/             { print ; looking = 1 ; next }
looking == 0              { print }
' $FILE1

exit 0

produces, by inserting file data2 into data1 between the markers:

% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0 
GNU bash 3.2.39
specimen (local) 1.17
GNU Awk 3.1.5

Whole: 7:0:7 of 13 lines in file "data1"
foo
(div id=m1)
bar
baz
(/div)
corge
warg
(div id=m1)
fred
plugh
xyzzy
(/div)
thud

Whole: 7:0:7 of 6 lines in file "data2"
  A little song
  A little dance
  A little seltzer down your pants
  Lots of specials: \'/'\.\'.[];][[+_)_()
  more: *&^&^&%^#%@#$!~[][[][=-00-99889763412
  still more: `>?>`,';\';][p][{}]12`//'-0

 Results:
foo
(div id=m1)
  A little song
  A little dance
  A little seltzer down your pants
  Lots of specials: \'/'\.\'.[];][[+_)_()
  more: *&^&^&%^#%@#$!~[][[][=-00-99889763412
  still more: `>?>`,';\';][p][{}]12`//'-0
(/div)
corge
warg
(div id=m1)
  A little song
  A little dance
  A little seltzer down your pants
  Lots of specials: \'/'\.\'.[];][[+_)_()
  more: *&^&^&%^#%@#$!~[][[][=-00-99889763412
  still more: `>?>`,';\';][p][{}]12`//'-0
(/div)
thud

cheers, drl

I took the message (awk) and got the solution (see above).
And now I know that there is no sed-awk program that can interpret

sed-awk s/(div id=m1).*?(\/div)/(div id=m1)$replacetextfromfile(\/div)/"

Thanks!

A little bit simple solution :

cat $file | grep -A $nb_line_to_suppress "$motif" > /tmp/tempfile.tmp
diff --suppress-common-lines --unchanged-line-format='' $file /tmp/tempfile.tmp > file.suppressed_block
rm -f /tmp/tempfile.tmp