How to Eliminate first line of multiple files

hi gurus ,,

I have multiple files with same file pattern..in a particular directory

for ex: file20061101.trf
file20061102.trf
file20061103.trf

Each of the file has a header as column names..

My questions is how can i eliminate the first row of each of these files and write it into a single file..

Can anyone help uup with a shell script..

i was thinking of looping each file and get the count and then do tail of -1 to a file..

but was not sure whether its the best method..if anyone had come across the situation please let me know

Appreciate any kind of information or help

Thanks n Regards
Sish

#!/usr/bin/perl
$x=1; #first file who will open will be file20061.trf change if you need
$xmax=9999; #change this as you need is the limite of files
$patternb="file2006"; #begin of filename
$patterne=".trf"; #end of filename
while($x!=$xmax){
$y=0;
@lines=();
$file=$patternb . $x . $patterne;
open(FD,"< $file")|| $y=1;
@lines=<FD>;
close(FD);
@lines=reverse(@lines);   #\
pop(@lines);                  # - Remove First Line
@lines=reverse(@lines);   #/
open(FD,"> $file") if $y!=1;
print FD @lines if $y!=1;
close(FD);
$x++; #and loop 
}

:slight_smile:
ps: the script need to be at the same directory of the files

Much easier with pipes than perl:

# Print a list of files with 'find', feed it into while loop.
# For each filename, open file, read and discard first line,
# print rest of file, while redirecting all output into 'output'
find ./ -iname 'file2006*' |
        while read FILE
        do
                ( read LINE ; cat ) < "${FILE}"
        done > output

hey i need to learn more about sh :smiley:

Any reason that you can't use good old tail? It has a '+' option that you can use to print lines from that line onward:

# cat test.sh
#!/bin/ksh
echo $1
echo $?

#tail +2 test.sh
echo $1
echo $?

Run tail +2 on each file through the while loop.

Or...

awk 'FNR>1' file*.trf > outfile

On a side note, funny how it is always whose is bigger, yet, when it comes to scripting/coding, it is always whose is smaller :slight_smile:

Python alternative:

#!/usr/bin/python
#name:test.py
import glob
for file in glob.glob("*.trf"):
     r = open(file)
     print header = r.readline().strip()
     r.close()    

usage:

./test.py > outputfile

#!/usr/local/bin/bash ;# ksh 2nd choice if bash not installed.
HDRSTR="unique string in every header"
DIR=/path/to/data/dir
OFL=/path/to/output/file
TMPFL=/tmp/zzz
cat /dev/null > $OFL
cd $DIR
for i in ls
do
cat $i >> $TMPFL
done
cat $TMPFL|sort -u > $OFL
CHK=`cat -n $OFL|grep $HDRSTR|cut -f1 -d' '`
TTL=`cat $OFL|wc -l`
cat $OFL|head -n $(($CHK-1)) > $OFL
cat $OFL|tail -n $(($TTL-$CHK)) >>$OFL

Okay, this hasn't been tested. It looks right, but the -1 on the penultimate line may not be needed... Hmmm.

Have fun.

sed -n "1!p" file2006* > newfile

awk '{if(fn != FILENAME)NR=1; if(NR != 1 )print $0;fn=FILENAME} ' filename*

Note that any of these solutions that use shell-globbing instead of find will break on large numbers of files... I used find since I figured there might be thousands of those autogenerated files.