Given that you have asked us for help 61 times before with some of your problems being very similar to this one, it is disappointing that you are unable to guess that awk would be a good way to solve a problem like this and show us a start to a solution to this on your own.
On systems where it does work, this is a great solution. It doesn't work on all systems, and the standards state that the behavior is unspecified if RS is set to a string containing more than one character.
The standard doesn't place any limits like that on ORS ; only on RS .
#!/usr/bin/perl
use strict;
use warnings;
my $sep = $/ = "== :: ==\n";
my $pattern = "Permanent";
my $hat = 0;
while(<>) {
if(/$pattern/){
print $sep if $hat == 0 and ++$hat;
print;
}
}
Save as filter.pl
Run as perl filter.pl april_2015.txt
or perl filter.pl april_2015.txt > permanent.txt
or perl filter.pl april_2015.txt may_2015.txt june_2015.txt > permanent.txt
or perl filter.pl *_2015.txt > permanent.txt
I tried code suggested by Aia and its working as per expectations.
However, I would like to run this perl in loop as I have around 830 different patterns like "Permanent".
I tried following
cat filter.pl
#!/usr/bin/perl
use strict;
use warnings;
my $sep = $/ = "== :: ==\n";
my $pattern = `$1`;
my $hat = 0;
while(<>) {
if(/$pattern/){
print $sep if $hat == 0 and ++$hat;
print;
}
}
and made following loop.
for service in `cat input_patterns`
do
echo ${service}
perl filter.pl ${service} file>>pattern.out
done
however, its not working.
I am sure I am doing something wrong.
Kindly suggest.
The Perl program processes the data file to search for one pattern. Inside the loop, it runs as many times as there are patterns.
So, if your "input_patterns" file has 830 lines, then you process the data file 830 times, searching for one pattern each time!
To give you an analogy, let's say you want to go grocery shopping.
Do you do the following?
(1) Go to grocery store, buy eggs, come back.
(2) Then go to the same grocery store, buy milk, come back.
(3) Then go to the same grocery store, buy meat, come back.
(4) Then go to the same grocery store, buy drinks, come back.
...
I'm sure you see how inefficient this is, yet you're doing something similar in your code.
While this kind of code might work at a small scale (small data file, small pattern file), the inefficiency due to repeated scanning add up at a large scale.
Imagine searching for 10,000 patterns in a million line data file.
Do you want to scan a million line file 10,000 times, looking for one pattern each time?
This is not tested, however, if you want to use the Perl script in that way you need a different modification that what I highlighted in red.
#!/usr/bin/perl
use strict;
use warnings;
my $pattern = shift;
my $sep = $/ = "== :: ==\n";
my $hat = 0;
while(<>) {
if(/$pattern/){
print $sep if $hat == 0 and ++$hat;
print;
}
}
And then you can use the following shell script
#!/bin/bash
while read service; do
perl filter.pl "$service" file.txt >> pattern.out
done < input_patterns
Of course, that might be slow due to all the times the binary perl gets called, and the opening, appending and closing of pattern.out. Your mileage may vary there.
Here's a Perl script that might work alone.
Again, it is not tested but you can try with just a portion of your 800 plus patterns input_patterns. It assumes one pattern per line.
#!/usr/bin/perl
use strict;
use warnings;
my $pattern;
my $pattern_source = shift;
{
local $/ = undef;
open my $fh, '<', $pattern_source or die "$!\n";
$pattern = <$fh>;
$pattern =~ s/\R(?!$)/\|/g;
close $fh;
}
my $sep = $/ = "== :: ==\n";
my $hat = 0;
while(<>) {
if(/$pattern/){
print $sep if $hat == 0 and ++$hat;
print;
}
}
Use as perl filter.pl input_patterns april_2015.txt > pattern.out