Creating a control file for a group of files

Shell Programming and Scripting
1

Hi,

We have almost 45,000 data files created by a script daily. The file names are of format-ODS.POS.<pharmacyid>.<table name>.<timestamp>.dat. There will be one data file like this for each pharmacy and each table.(Totally around 45,000)

The requirement is to create a control file for each pharmacy id with file name, No.of rows , table name, timestamp and MD5 value.

The data files look like this -

ODS.POS.ABC89.ADT_LOG.07272010_033303.dat
ODS.POS.AEC12.ADT_LOG.07272010_033303.dat
ODS.POS.ABC78.ADT_LOG.07272010_033303.dat 
 
ODS.POS.AEC12.TR_ITM_CPN_TND.07272010_033303.dat
ODS.POS.AEC13.TR_ITM_CPN_TND.07272010_033303.dat
ODS.POS.ABC89.TR_ITM_CPN_TND.07272010_033303.dat
ODS.POS.ABC78.TR_ITM_CPN_TND.07272010_033303.dat

The requirement is to create a control file for each pharmacyid

Controlfile 1 -> for pharmacy id - ABC89

Control file 1 (ODS.POS.<Pharmacyid>.timestamp.ctl )should contain the filename, no.of rows, table name, time stamp and MD5 value for the below files.

ODS.POS.ABC89.ADT_LOG.07272010_033303.dat
ODS.POS.ABC89.TR_ITM_CPN_TND.07272010_033303.dat

Control file 2->for pharmacy id - AEC12

Control file 2 ((ODS.POS.<Pharmacyid>.<timestamp>.ctl )should contain the filename, no.of rows, table name, time stamp and MD5 value for the below files.

ODS.POS.AEC12.ADT_LOG.07272010_033303.dat
ODS.POS.AEC12.TR_ITM_CPN_TND.07272010_033303.dat

can somebody help on this?

Thanks
Maya

2

Hi,

Try This,

#!/usr/bin/perl

`ls ODS.POS.*.dat > filelist`;
$timest=`date '+%m%d%Y_%H%M%S'`;
chop($timest);
open (FH,"<","filelist");
while (<FH>) {
chomp;
if (/(.+?)\.(.+?)\.(.+?)\.(.+?)\..*/) {
        $ctl='ODS.POS';
        $ctl="$ctl.$3.$timest.ctl";
        open (FW,">>","$ctl");
        print FW "File name - $_ \n";
        print FW "Table Name - $4 \n";
        $checksum=`md5sum $_ | cut -d " " -f1`;
        $row_cnt=`wc -l $_ | cut -d " " -f1`;
        chop($checksum,$row_cnt);
        print FW "Row Count - $row_cnt \n";
        print FW "md5Checksum - $checksum \n";
        print FW "--------------------------------------\n";
}
}
close(FH);
3

Thank you so much .. Its Perfect :slight_smile:

Maya