PERL: removing blank lines from multiple files

Hi Guru's , I have a whole bunch of files in /var/tmp that i need to strip any blank lines from, so ive written the following script to identify the lines (which works perfectly).. but i wanted to know, how can I actually strip the identified lines from the actual source files ??

	my @all = qx{ls /var/tmp}; # perform an ls on the folder
	
	foreach (@all) {
				
		open(IN, "/var/tmp/$_");
			chomp(my $file = $_);
			my $counter = "0";
			while  (<IN>) {
				$counter++;
				if ($_ =~ /^$/) {
				  print "blank line found in $file at line number $counter\n";
                                  #something here to go and fix it in the actual file ?
				}
			}
		close(IN);
	}

Any help or advice on this would be greatly appreciated

sed -i '/^$/d' /tmp/*

Careful, this will change any file in /tmp

thanks Scrutinizer for your response ... im ideally looking for a way in perl to do this

thank you anyway

I think Scrutinizer's method is a very simple and elegant way of doing what you want to do.
I fail to understand why you would want to open each file and read it line by line just to remove blank lines.
You have a loaded Uzi and you want to club someone to death with it ?

tyler_durden

There is no quick one or two liners native to perl to do this. If you want to use the perl program you have already written I would insert the sed statement Scrutinizer posted after the close file line in your foreach structure followed by a copy statement using the system function.

system("sed -i '/^$/d' $_ > ${_}.noblanks")
system("cp ${_}.noblanks ${_}")

Hi Tyler, the files in the /var/tmp folder are generated from another process which has been known (annoyingly) to produce output files with blank lines ..the reason i have to go through each file line by line is because i need to identify exactly which line number in which file is returning blank and send it to a log file. you are right, there is no need for me to actually fix these issues within this actual script but i thought it make sense to do so if possible

I read on another site that I can set my file location within my script to say $file and then run this

my $file = "/var/tmp/0629AN1208.net";
$file =~ s/^\n//g;

but that doesnt seem to work, whereas if i run the equivalent one liner from the command line (see below) it works perfectly .

perl -pi -e "s/^\n//" /var/tmp/0629AN1208.net

is there a way to convert that one liner into something I can run within a perl script ?

Look at my above post. It does just that.

Thanks, but i really cant believe a perl one liner exists to do this, but the only way to do this omside an actual perl script is to run system() to run sed, then run system() again to run cp to copy the file back over

surely ?

You can write something like this (untested).
Backup your files first!

#! /usr/bin/env perl

use warnings;
use strict;

my @files   = glob '/tmp/*'; # adjust the glob!
my $logfile = $0 . '.log';

open LH, '>', $logfile or die "open $logfile: $!\n";

for (@files) {
    {
        local ( $^I, @ARGV ) = ( '', $_ );
        while (<>) {
            print LH $ARGV, ": record ", $., "\n" and next if /^\s*$/;
            print;
        }
    }
}

close LH or warn "close $logfile: $!\n";

That's not a good enough reason.
You do not have to
(1) open
(2) read and
(3) close a file

just to find out which line number is blank. Sure, you can, but you don't have to.
If you do, then it's a convoluted way of doing things. The sed program didn't open, read, and close the file (at least explicitly).

Send the blank line to a log file ? This wasn't the requirement in your original post.

Not sure I understand this. If there is no need to fix it, then why do you even want to do it ?
As for the possibility, you opened the file for reading (the default). And you wanted to write to a file. They are not the same thing.

Of course it doesn't, if you understand what you are trying to do. "$file" is just a variable, not an actual file. And the "s///" operator simply works on the value of that variable. Your variable did not have a newline character at the beginning, so it wasn't removed. If it had, then it would've been removed -

$ 
$ perl -le '$file = "
/var/tmp/0629AN1208.net"; print "Before: $file"; $file =~ s/^\n//g; print "After : $file"'
Before: 
/var/tmp/0629AN1208.net
After : /var/tmp/0629AN1208.net
$ 
$ 

Perl is simply doing what it was told to do.
And if you are thinking that the "s///" operator will actually open a file and remove newlines, just because the variable was called "$file", then you need to brush up on the basics first.

That one-liner writes to the file because of that qualifier "-i". Check the perl documentation. You could make it work on a hierarchy of files, but the solution would be more verbose than that sed solution.

tyler_durden

Here is what you can do:

  1. A seperate script using sed to remove the blank lines
  2. Call sed from within your perl script with system
  3. The perl -i route in the above post
  4. Dump each line of the file to an array except blank lines, open the files again for writing, and clobber them with the contents of the array
  5. Create a new file, open it for writing, add an else to the if ( /^$/ ) that counts blank lines, and print the non blanks to the new file. Then overwrite the original with the new one.

1,2, and 3 are much easier than 4 or 5.

thank you for your responses, I have been learning perl for the grand total of 5 weeks now...so go easy on me :wink:

i will look at using the sed solution instead

thanks again