Perl script: matching multiple lines error

Shell Programming and Scripting
1

Dear Perl users,

Could somebody help me how to fix my code so I can get my desired output.
Here is the data:

Pattern Gabriel
halo1
halo2
end
Pattern Andreas
halo1
halo2
end

I want to grep multiple lines between the pattern /Pattern Gabriel / and /end/.
Then I will store the output into two files (let's say Gabriel.log and Andres.log based on the name).

The Gabriel.log contains:


Pattern Gabriel
halo1
halo2
end

The Andreas.log contains:

Pattern Andreas
halo1
halo2
end

I saw there is a similar solution that I found but the desired output isn't valid yet.
here is my code so far:

#!/usr/bin/perl -w
use strict;

my @result;
my $flag = 0;
my @data = qw /Gabriel Andreas/;

while (my $line = shift @data)
{
  if (open (EXT, ">${line}.log"))
  {

  if (open (FILE, "<data_test.log"))
  {
  
  while (<FILE>) 
    {
      
      if (/Pattern\s+($line)/i) 
      {
       $flag = 1;
      } 
      elsif( /^end/ ) 
      {
       $flag = 0;
      }
      if ($flag) 
      {
        push @result, $_;
      }
    }

  print EXT "$_" for (@result);

  }
  }
}  
close (EXT);
close (FILE);
2

If your input file is not huge, then here's on way to do it:

$ 
$ ls -1 *
input
$ 
$ cat -n input
     1    Pattern Gabriel
     2    halo1
     3    halo2
     4    end
     5    Pattern Andreas
     6    halo1
     7    halo2
     8    end
$ 
$ 
$ perl -lne 'BEGIN {$/=undef} while(/^(Pattern (\S+).*?end)/msg) {open(FH, ">", "$2.log"); print FH $1; close(FH)}' input
$ 
$ # see if the logs have been created now
$ ls -1
Andreas.log
Gabriel.log
input
$ 
$ # check the logs
$ cat Andreas.log
Pattern Andreas
halo1
halo2
end
$ 
$ cat Gabriel.log
Pattern Gabriel
halo1
halo2
end
$ 
$
3

Hi durden_tyler,

Thanks a lot but unfortunately my file is huge. and also if your code above, I can not split into column directly (per each line), am I right?.

do you have other solution for this problem, hopefully just fixed my code because my code already huge so I do not need to change everything.

Thanks

4

I don't quite understand your question here.

5

if my original data is changed and added the additional parameter below. I have used your code above before but when I want to split it (e.g. halo1 data into two column by using regexp) is it possible in the while looping directly after the pattern is match?.

Pattern Gabriel
halo1 data
halo2 data
end
Pattern Andreas
halo1 data
halo2 data
end
6

I'll answer your second question first.

Of course it is possible. Almost every kind of text processing is possible in Perl, if you know how to do it.

As an example:

$ 
$ 
$ cat -n input
     1    Pattern Gabriel
     2    halo1 data
     3    halo2 data
     4    end
     5    Pattern Andreas
     6    halo1 data
     7    halo2 data
     8    end
$ 
$ 
$ perl -lne 'BEGIN {$/=undef}
             while(/^(Pattern (\S+).*?end)/msg) {
               @x = split("\n",$1);
               print "Begin:",$x[0];
               for $i (1..$#x-1) {
                 print "Line = ", $x[$i];
                 ($col1, $col2) = $x[$i] =~ /^(\S+)\s+(\S+)$/;
                 print "col1 = ", $col1;
                 print "col2 = ", $col2;
                 print "=" x 20;
               }
               print "End:",$x[$#x]
             }
            ' input
Begin:Pattern Gabriel
Line = halo1 data
col1 = halo1
col2 = data
====================
Line = halo2 data
col1 = halo2
col2 = data
====================
End:end
Begin:Pattern Andreas
Line = halo1 data
col1 = halo1
col2 = data
====================
Line = halo2 data
col1 = halo2
col2 = data
====================
End:end
$ 
$ 
$

Now for your first concern about the file being huge - that is a subjective term. What may be a huge file for you may not be huge for Perl. A good idea would be to benchmark your processing and go from there.

Finally, if you want to put chunks of text into separate log files but still want to do line-by-line regex processing, then a simpler approach be to just:

  • read your file line-by-line,
  • start logging to the file when you encounter the start pattern
  • use regexes for the following lines; do whatever you want to do with those
  • and stop logging when you encounter the end pattern
7

Hi durden_tyler,

Thanks for your code. I found the bug was the place of array variable initialization. After I put the "@result" array variable to the next line " if (open (EXT, ">${line}.log")) {",
The program already produced my desired output. Anyway thanks a lot for your code sharing.

Here is the complete code of mine.

#!/usr/bin/perl -w
use strict;


my $flag = 0;
my @data = qw /Gabriel Andreas/;

while (my $line = shift @data)
{
  if (open (EXT, ">${line}.log"))
  {
   my @result;
   
  if (open (FILE, "<data_test.log"))
  {
  
  while (<FILE>) 
    {
      
      if (/Pattern\s+($line)/i) 
      {
       $flag = 1;
      } 
      elsif( /^end/ ) 
      {
       $flag = 0;
      }
      if ($flag) 
      {
        push @result, $_;
      }
    }

  }
  print EXT "$_" for (@result);
  }
}  
close (EXT);
close (FILE)

__DATA__
Pattern Gabriel
halo1
halo2
end
Pattern Andreas
halo1
halo2
end