Read a File line by Line in unix

Shell Programming and Scripting
1

Hi ,

I have few files in a directory .
I need to read all the files in that direcory and search for certain criteria in each file and write into a outputfile.
I am also writing the each file name to the outputfile .
For the above criteria , i have written the script.
*****************************************************

#!/bin/sh
rm -f  LookupFile.txt
for i in `ls *.log`
do
echo -ne $i >> LookupFile.txt
awk  '/CMN_1716 Lookup/
     /Creating Lookup Cache/
     /Lookup cache creation completed/' $i >> LookupFile.txt
echo  $i >> LookupFile.txt
done

***********************************************************
o/p for the above script

LookupFile.txt--Filename
 

(sessionlogname1)TRANSF_1_1_1> CMN_1716 Lookup [lookuptransformationname1] uses database connection [Relational:DataWarehouse] in codepage [UTF-8 encoding of Unicode]
TRANSF_1_1_1> DBG_21079 Creating Lookup Cache : (Fri Aug 05 00:46:31 2011)
TRANSF_1_1_1> DBG_21294 Lookup cache creation completed : (Fri Aug 05 00:47:15 2011)
TRANSF_1_1_1> CMN_1716 Lookup [lookuptransformationname2] uses database connection [Relational:DataWarehouse] in codepage [UTF-8 encoding of Unicode]
TRANSF_1_1_1> DBG_21079 Creating Lookup Cache : (Fri Aug 05 00:47:16 2011)
TRANSF_1_1_1> DBG_21294 Lookup cache creation completed : (Fri Aug 05 00:47:16 2011)
sessionlogname1
(sessionlogname2)TRANSF_1_1_1> CMN_1716 Lookup [lookuptransformationname] uses database connection [Relational:DataWarehouse] in codepage [UTF-8 encoding of Unicode]
TRANSF_1_1_1> DBG_21079 Creating Lookup Cache : (Mon Aug 08 03:42:15 2011)
TRANSF_1_1_1> DBG_21294 Lookup cache creation completed : (Mon Aug 08 03:42:15 2011)
sessionlogname2

*****************************************
In the second stage, i need to read the LookupFile.txt file line by line and get the required information
for each lookup transformation into one line and write to a file.
So the file should be in the below mentioned format.

--------Final file structure------------

Filename1,lookuptransformationname1,cache creation time(Fri Aug 05 00:46:31 2011),cache completed time(Fri Aug 05 00:47:15 2011)
Filename1,lookuptransformationname2,Fri Aug 05 00:47:16 2011,Fri Aug 05 00:47:16 2011
Filename2,lookuptransformationname,Mon Aug 08 03:42:15 2011,Mon Aug 08 03:42:15 2011

So for each lookuptransformation in the file , it should write into one line with the assoiciated information.

Please help me in this issue .
Thanks,
Burwood

2

Try and adapt the following awk script that makes all the work (I hope) :

awk -v FS='[][()]' '

    FNR==1 {
        printLookup();
        FileName = FILENAME;
    }

    /CMN_1716 Lookup/ {
        printLookup();
        LookupTransName = (/^\(/ ? $4 : $2);
        next;
    }

    /Creating Lookup Cache/ {
        CreationTime = $2;
        next;
    }

    /Lookup cache creation completed/ {
        CompleteTime = $2;
        next;
    }

    END {
        printLookup();
    }

    function printLookup(from) {
        if (! LookupTransName) return;
        printf "%s,%s,%s,%s\n", FileName, LookupTransName, CreationTime, CompleteTime;
        LookupTransName = CreationTime = CompleteTime = "";
    }


' *.log </dev/null 2>/dev/null

Jean-Pierre.

1 Like