Delete a block of text delimited by blank lines when pattern is found

Shell Programming and Scripting
1

I have a file which contains blocks of text - each block is a multi-lines text delimited by blank lines eg.

<blank line>
several lines of text
...
pattern found on this line
several more lines of text
...
<blank line>

How do you delete the block of text (including the blank lines) when the pattern is found and leave one blank line?

2

Something like this?

sed -n '
/^$/ b block
H
$ b block
b
:block
x
/pattern/!p' data
3

Or:

sed '/pattern/,/^$/d' file

Regards

4

I think the OP wants something different:

$ cat data
block1
block1
block1
pattern block1
block1

block2
block2
block2
block2

block3
block3
pattern block3
block3

$ sed '/pattern/,/^$/d' data
block1
block1
block1
block2
block2
block2
block2

block3
block3

$ sed -n '
/^$/ b block
H
$ b block
b
:block
x
/pattern/!p' data

block2
block2
block2
block2
5

Perhaps I misinterpreted the question, the OP could make it clear.

Regards

6

Yes. Thank you, Radoulov. You interpreted my question correctly. (Sorry, Franklin52 that it was none too clear).

I don't quite understand the above bit of code about the Set-Aside buffer (but I'll look it up later ...unless you or someone wants to explain for the benefit of others).

But, in any case, the final result with only lines of block2 is what I want. I will do a test run later ... before I use it in earnest on that 8 gig file at work. Thanks again.

7 
# more file
block1
block1
block1
pattern block1
block1

block2
block2
block2
block2

block3
block3
pattern block3
block3

# awk 'BEGIN{RS=""}/pattern/{next}1' file
block2
block2
block2
block2
8

Or even:

awk '!/pattern/' ORS="\n\n" RS= file

You need ORS to restore the blank lines between the blocks.

But it's quite possible to get (not GNU Awk) the "line too long" error ...

9

Sure,
check Bruce Barnett's Sed - An Introduction and Tutorial - Flow Control

10

Hi.

A perl script doesn't usually need to worry about line length, and telling perl about the definition of a block (I call it a visual paragraph) is simple:

#!/usr/bin/perl

# @(#) p1       Demonstrate visual paragraph skip if pattern found.

use warnings;
use strict;

my($debug);
$debug = 0;
$debug = 1;

my($lines) = 0;
my($pattern) = shift || die "usage: $0 pattern [files]\n";

$/ = "";        # block separator: any sequence of two or more newlines.

while ( <> ) {
        $lines++;
        print if !/$pattern/xms;        # always use xms on matches
}

print STDERR " ( Blocks read: $lines )\n";

exit(0);

Using the block test file of radoulov and ghostdog74 as "data1" produces:

% ./p1 pattern data1
block2
block2
block2
block2

 ( Blocks read: 3 )

Best wishes ... cheers, drl

11

Thanks.
I've also been reading the relevant bits in Sed & Awk by Dale Dougherty and Arnold Robbins (O'Reilly). I understand the code now.

12

The awk code looks straightforward but I could not get that code to work

gawk 'BEGIN {RS=""}/pattern/{next}' myfile

(I took out the 1 after next because I thought it was a typo... but in any case with or without the 1, the code gives me no output at all :frowning:

13

I can't get your version (nor ghostdog74) to output anything.
gawk '!/pattern/' ORS="\n\n" RS= myfile
Strange
... but at least the sed version works a treat!

14

Post a sample from your (real) file.

15 
gawk '!/initials: 1/' ORS="\n\n" RS= myfile

Here's a sample of myfile where the second block should be deleted because of the pattern - initials: 1

dn: cn=LMP0330,ou=People,o=myOrg
destinationIndicator: FR
mail: a.b@myOrg.com
initials: 0
givenName: LMP0330
fullName: LMP0340 LMP0330
Language: FR
title: Mr
sn: LMP0340
street: 
postalCode: B106
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: Person
objectClass: ndsLoginProperties
objectClass: Top
loginTime: 20070303204107Z
description: LMP0330
cn: LMP0330
ACL: 2#subtree#cn=LMP0330,ou=People,o=myOrg
 #[All Attributes Rights]
ACL: 6#entry#cn=LMP0330,ou=People,o=myOrg#l
 oginScript
ACL: 2#entry#[Public]#messageServer
ACL: 2#entry#[Root]#groupMembership
ACL: 6#entry#cn=LMP0330,ou=People,o=myOrg#p
 rintJobConfiguration
ACL: 2#entry#[Root]#networkAddress

dn: cn=LMP0339,ou=People,o=myOrg
destinationIndicator: BE
mail: a.b@myOrg.com
initials: 1
givenName: LMP0339
fullName: LMP0339 LMP0339
Language: NL
title: Mr
sn: LMP0339
street: bourget
postalCode: 1060
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: Person
objectClass: ndsLoginProperties
objectClass: Top
loginTime: 20070303204102Z
cn: LMP0339
ACL: 2#subtree#cn=LMP0339,ou=People,o=myOrg
 #[All Attributes Rights]
ACL: 6#entry#cn=LMP0339,ou=People,o=myOrg#l
 oginScript
ACL: 2#entry#[Public]#messageServer
ACL: 2#entry#[Root]#groupMembership
ACL: 6#entry#cn=LMP0339,ou=People,o=myOrg#p
 rintJobConfiguration
ACL: 2#entry#[Root]#networkAddress

dn: cn=LMP0340,ou=People,o=myOrg
destinationIndicator: GB
mail: a.b@myOrg.com
initials: 0
givenName: LMP0340
fullName: LMP0340 LMP0340
Language: EN
title: Mr
sn: LMP0340
street: Waverley
postalCode: SE25
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: Person
objectClass: ndsLoginProperties
objectClass: Top
loginTime: 20070303204107Z
description: LMP0340
cn: LMP0340
ACL: 2#subtree#cn=LMP0340,ou=People,o=myOrg
 #[All Attributes Rights]
ACL: 6#entry#cn=LMP0340,ou=People,o=myOrg#l
 oginScript
ACL: 2#entry#[Public]#messageServer
ACL: 2#entry#[Root]#groupMembership
ACL: 6#entry#cn=LMP0340,ou=People,o=myOrg#p
 rintJobConfiguration
ACL: 2#entry#[Root]#networkAddress
16

Using copy/paste from your post it gets deleted:

zsh-4.3.4-dev-4% cat data                                
dn: cn=LMP0330,ou=People,o=myOrg
destinationIndicator: FR
mail: a.b@myOrg.com
initials: 0
givenName: LMP0330
fullName: LMP0340 LMP0330
Language: FR
title: Mr
sn: LMP0340
street:
postalCode: B106
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: Person
objectClass: ndsLoginProperties
objectClass: Top
loginTime: 20070303204107Z
description: LMP0330
cn: LMP0330
ACL: 2#subtree#cn=LMP0330,ou=People,o=myOrg
#[All Attributes Rights]
ACL: 6#entry#cn=LMP0330,ou=People,o=myOrg#l
oginScript
ACL: 2#entry#[Public]#messageServer
ACL: 2#entry#[Root]#groupMembership
ACL: 6#entry#cn=LMP0330,ou=People,o=myOrg#p
rintJobConfiguration
ACL: 2#entry#[Root]#networkAddress

dn: cn=LMP0339,ou=People,o=myOrg
destinationIndicator: BE
mail: a.b@myOrg.com
initials: 1
givenName: LMP0339
fullName: LMP0339 LMP0339
Language: NL
title: Mr
sn: LMP0339
street: bourget
postalCode: 1060
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: Person
objectClass: ndsLoginProperties
objectClass: Top
loginTime: 20070303204102Z
cn: LMP0339
ACL: 2#subtree#cn=LMP0339,ou=People,o=myOrg
#[All Attributes Rights]
ACL: 6#entry#cn=LMP0339,ou=People,o=myOrg#l
oginScript
ACL: 2#entry#[Public]#messageServer
ACL: 2#entry#[Root]#groupMembership
ACL: 6#entry#cn=LMP0339,ou=People,o=myOrg#p
rintJobConfiguration
ACL: 2#entry#[Root]#networkAddress

dn: cn=LMP0340,ou=People,o=myOrg
destinationIndicator: GB
mail: a.b@myOrg.com
initials: 0
givenName: LMP0340
fullName: LMP0340 LMP0340
Language: EN
title: Mr
sn: LMP0340
street: Waverley
postalCode: SE25
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: Person
objectClass: ndsLoginProperties
objectClass: Top
loginTime: 20070303204107Z
description: LMP0340
cn: LMP0340
ACL: 2#subtree#cn=LMP0340,ou=People,o=myOrg
#[All Attributes Rights]
ACL: 6#entry#cn=LMP0340,ou=People,o=myOrg#l
oginScript
ACL: 2#entry#[Public]#messageServer
ACL: 2#entry#[Root]#groupMembership
ACL: 6#entry#cn=LMP0340,ou=People,o=myOrg#p
rintJobConfiguration
ACL: 2#entry#[Root]#networkAddress
zsh-4.3.4-dev-4% awk '!/initials: 1/' ORS="\n\n" RS= data
dn: cn=LMP0330,ou=People,o=myOrg
destinationIndicator: FR
mail: a.b@myOrg.com
initials: 0
givenName: LMP0330
fullName: LMP0340 LMP0330
Language: FR
title: Mr
sn: LMP0340
street:
postalCode: B106
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: Person
objectClass: ndsLoginProperties
objectClass: Top
loginTime: 20070303204107Z
description: LMP0330
cn: LMP0330
ACL: 2#subtree#cn=LMP0330,ou=People,o=myOrg
#[All Attributes Rights]
ACL: 6#entry#cn=LMP0330,ou=People,o=myOrg#l
oginScript
ACL: 2#entry#[Public]#messageServer
ACL: 2#entry#[Root]#groupMembership
ACL: 6#entry#cn=LMP0330,ou=People,o=myOrg#p
rintJobConfiguration
ACL: 2#entry#[Root]#networkAddress

dn: cn=LMP0340,ou=People,o=myOrg
destinationIndicator: GB
mail: a.b@myOrg.com
initials: 0
givenName: LMP0340
fullName: LMP0340 LMP0340
Language: EN
title: Mr
sn: LMP0340
street: Waverley
postalCode: SE25
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: Person
objectClass: ndsLoginProperties
objectClass: Top
loginTime: 20070303204107Z
description: LMP0340
cn: LMP0340
ACL: 2#subtree#cn=LMP0340,ou=People,o=myOrg
#[All Attributes Rights]
ACL: 6#entry#cn=LMP0340,ou=People,o=myOrg#l
oginScript
ACL: 2#entry#[Public]#messageServer
ACL: 2#entry#[Root]#groupMembership
ACL: 6#entry#cn=LMP0340,ou=People,o=myOrg#p
rintJobConfiguration
ACL: 2#entry#[Root]#networkAddress

zsh-4.3.4-dev-4%

What is the output from this command:

awk '!/initials: 1/' ORS="\n\n" RS= data|fgrep initial
17

What is the output from this command:

awk '!/initials: 1/' ORS="\n\n" RS= data|fgrep initial

nothing at all (after replacing data with myfile).
hmm...looks like my gawk (and awk) is broken on my linux box.

And thanks for all your patience and help. I will try it when I get back to work on Monday and hopefully it will work.

18

Hi,
Try follow one. It should be ok.
iuput

a A
b B
pattern
c C
d D

e E
f F
g G
h H

i I
j J
pattern
k K 
l L

a A
b B
c C
d D

output:

e E
f F
g G
h H


a A
b B
c C
d D

code:
awk 'BEGIN{
RS=""
FS="\r"
}
{
if ($0 !~ /pattern/)
	print 
print ""
}' filename