Script to validate header in a csv file

Shell Programming and Scripting
1

Hi All;

I am struggling to write a script that validates file header.

Header file would be like below with TAB separated

TRX #	TYPE	REF #	Source Piece Code	Destination Piece Code

every time I need to check the txt file if the header was same as above fields if validation success process the file else exit 1

I really appreciate if someone can provide me the script

Thanks

2

Welcome to the forum.

Any attempts / ideas / thoughts from your side?

3

The question is incomplete:

Do you require that the header is EXACTLY like the one you posted? In this case, there is nothing CSV-specific in your question. Instead, the problem boils down to finding out, whether the first line of a file has a certain content.

Or do you require that the header is STRUCTURALLY like the one you posted? For example, the CSV-file format allows fields to be quoted; or, you might consider trailing spaces after the header names insignificant; or you might allow the fields to occur in any order. In this case, you need to specify, what exactly are the conditions to regard the header valid.

4

RudiC, I tried to write below code, but not able write the logic how to check below columns with header file columns.

TRX #    TYPE    REF #    Source Piece Code    Destination Piece Code

cd $MOUNT_DIR 
for file in $(ls ${MOUNT_DIR}/Test*.txt)
do
    file_name=$(basename $file)
    echo $name
    chmod -R 777 $MOUNT_DIR$name
    match_header=awk NR==1 $file_name
    echo "the header file from the input file is" $match_header
    
    if [ comm $match_header =] ** not able the script how to compare the header columns to mentioned columns **
    then
        echo $name:"Header file mismatch, terminate the flow " >> $JOB_LOG
        exit 1
    else
        echo $name:"Header file match, We can process this file " >> $JOB_LOG
    fi
done

---------- Post updated at 02:09 AM ---------- Previous update was at 01:58 AM ----------

rovf, Sorry for confusion. it was txt file (I have edited the post).

Desired columns (A): TRX # TYPE REF # Source Piece Code

columns we get from header file (B): TRX # TYPE *** # Source Piece Code

if columns from header file doesn't matches to Desired columns then it need to through terminate exit 1. else process the file

5

Something like below:

head -1 fileA | grep "$(cat fileB)" 
if [[ $? -ne 0 ]] ;
then
echo "fileA header does not match with fileB"
exit 1;
fi

Note: even if there is a difference with "spaces" match wont happen.

6

How about

if awk '{exit !(/TRX #\tTYPE\tREF #/)}' $file_name
   then	# process file
   else	exit 1
   fi

What's the difference between $name and $file_name ?

7

What do you mean by "matching". Being literally equal? Or are you thinking of some kind of pattern matching?

Also, specify which shell you want the script to run in. This might be an important point if you are looking for some kind of pattern matching instead of just testing equality (which is trivial).