Match paragraph between two patterns, delete the duplicate paragraphs

Shell Programming and Scripting
1

Hello all

I have a file my DNS server where there are duplicate paragrapsh like below. How can I remove the duplicate paragraph so that only one paragraph remains.

BEGIN;
replace into domains (name,type) values ('225.168.192.in-addr.arpa','MASTER');
replace into records (domain_id, name,type,content,ttl,prio) select id ,'225.168.192.in-addr.arpa', 'SOA', 'ns1.test.com. hostmaster.test.com. 2013030511 600 7200 604800 345600', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'225.168.192.in-addr.arpa', 'NS', 'ns1.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'225.168.192.in-addr.arpa', 'NS', 'ns2.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'54.225.168.192.in-addr.arpa', 'PTR', 'somedomain1.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'117.225.168.192.in-addr.arpa', 'PTR', 'somedomain2.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
COMMIT;
BEGIN;
replace into domains (name,type) values ('225.168.192.in-addr.arpa','MASTER');
replace into records (domain_id, name,type,content,ttl,prio) select id ,'225.168.192.in-addr.arpa', 'SOA', 'ns1.test.com. hostmaster.test.com. 2013030511 600 7200 604800 345600', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'225.168.192.in-addr.arpa', 'NS', 'ns1.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'225.168.192.in-addr.arpa', 'NS', 'ns2.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'54.225.168.192.in-addr.arpa', 'PTR', 'somedomain1.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'117.225.168.192.in-addr.arpa', 'PTR', 'somedomain2.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
COMMIT;
BEGIN;
replace into domains (name,type) values ('225.168.192.in-addr.arpa','MASTER');
replace into records (domain_id, name,type,content,ttl,prio) select id ,'225.168.192.in-addr.arpa', 'SOA', 'ns1.test.com. hostmaster.test.com. 2013030511 600 7200 604800 345600', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'225.168.192.in-addr.arpa', 'NS', 'ns1.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'225.168.192.in-addr.arpa', 'NS', 'ns2.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'54.225.168.192.in-addr.arpa', 'PTR', 'somedomain1.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
replace into records (domain_id, name,type,content,ttl,prio) select id ,'117.225.168.192.in-addr.arpa', 'PTR', 'somedomain2.test.com', 86400, 0 from domains where name='225.168.192.in-addr.arpa';
COMMIT;

Note: pattern begins with "BEGIN;" and ends with "COMMIT;"

Any help please? I have changed the actual IP address and domain names. This is just one example. THere are other duplicate entries in the same format, only the IP address and domain names differ

TIA
sb

2

you basically want to remove duplicate lines right?

 
 awk '!x[$0]++'  filename
3

thanks vidyadhar85,

It removed all the duplicate lines. But the "BEGIN;" and "COMMIT;" were also needed which is needed in the file. The output came in below syntax.

BEGIN;
replace into domains (name,type) values ('blah blah 1,'MASTER');
COMMIT;
replace into domains (name,type) values ('blah blah 2','MASTER');
replace into domains (name,type) values ('blah blah 3','MASTER');
replace into domains (name,type) values ('blah blah 4','MASTER');

what I need is as below.

BEGIN;
replace into domains (name,type) values ('blah blah 1,'MASTER');
COMMIT;
BEGIN;
replace into domains (name,type) values ('blah blah 2','MASTER');
COMMIT;
BEGIN;
replace into domains (name,type) values ('blah blah 3','MASTER');
replace into domains (name,type) values ('blah blah 3','MASTER');
replace into domains (name,type) values ('blah blah 3','MASTER');
COMMIT;
BEGIN;
replace into domains (name,type) values ('blah blah 4','MASTER');
replace into domains (name,type) values ('blah blah 4','MASTER');
COMMIT;

Note: In the original file some paragraphs contain single line. Some contain multiple lines. The example I had pasted had multiple lines. Your script works perfectly for the example I had pasted.

TIA
sb