Efficiently altering and merging files in perl

Shell Programming and Scripting
1

I have two files

fileA
HEADER LINE A
CommentLine A
Content A
....
....
....
TAILER A

fileB
HEADER LINE B
CommentLine B
Content B
....
....
....
TAILER B

I want to merge these two files as

HEADER LINE A
CommentLine A
Content A
....
....
....
Content B
....
....
....
TAILER B

i.e. skip the TAILER line of file A and skip the HEADER and Comment Line of fileB

I am able to do it using the below perl code

        open ( FA, "$fileA" ) || die("can't open fileA $!");
        open ( FB, "$fileB" ) || die("can't open fileB $!");
        open ( TMP, ">> tmp_file" ) || die("can't open tmp_file $!");

        #reading both files in array
        my @fileA = <FA>;
        my @fileB = <FB>;

        #getting rid of HEADER, Comment line, in fileB
        shift @fileB;
        shift @fileB;

        #getting rid of TAILER in fileA
        pop @fileA;

        my @tmp_file=(@fileA,@fileB);

        foreach ( @tmp_file ){
            print TMP $_;
        }

        close(FA);
        close(FB);
        close(TMP);
        
        rename tmp_file, fileA || die("can't rename tmp_file to fileA);

This code works fine, however I doubt it's efficiency if fileA and fileB are going to be millions of lines (which is the case)
i.e. why read whole file in arrays just to get rid of three lines (will end up using lots of memory)

Can someone suggest a more efficient way of doing this
(answers in perl only)

2

Do not load it in memory, try:

open ( FA, "$fileA" ) || die("can't open fileA $!");
open ( FB, "$fileB" ) || die("can't open fileB $!");
open ( TMP, ">> tmp_file" ) || die("can't open tmp_file $!");


my $line;
foreach  (<FA>)  {
    #getting rid of TAILER in fileA
    print TMP $line if $line;
    $line = $_;
    }

$count = 0; 
foreach  (<FB>)  {
    #getting rid of HEADER, Comment line, in fileB
    next if ($++counter < 2) ;
    print TMP $_;
    }
}

close(FA);
close(FB);
close(TMP);
rename tmp_file, fileA || die("can't rename tmp_file to fileA);
3

Thanks KlashXX

Asking a little more help, can you please explain the below snippet

my $line; 
foreach  (<FA>)  
{     #getting rid of TAILER in fileA     
         print TMP $line if $line;
         $line = $_;
}
4

Of course , the line variable only gets value after the first iteneration of the loop, in other words the first line is printed in the second iteration and so on.

Finally the penultimate line will be write in the last loop. The last line value will be never used.

5

Thanks again