Awk+colpos+negative if for 20 values

rajubollas · July 7, 2014, 3:01pm

Hi All,

i have 20 Obligors when ever i dont find all of them on particular row/line from each in put i need to print it in different file.

using below command but it is not working please help at earliest.

Steps:

        set -A FILENAME   $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $1 }')
        set -A COLPOS     $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $4 }')

#Carrs Extract File                                             Column Position     Control File column position	GSWID Column Possition
##############################################################################################################################################
Assessment_Extract.txt                                       4                     0-17					3

$ cat GERMANY_Obligor_filter_ctl
##Obligor information file
##For new obligor for search contion add "|| $colpos != xxxxxx" in format
##make sure all condition values in single line
$colpos  != 228070 || $colpos  != 228071 || $colpos  != 228072 || $colpos  != 228073 || $colpos  != 228074 || $colpos  != 228075 || $colpos  != 228076 || $colpos  != 228077 || $colpos  != 228078 || $colpos  != 228079 || $colpos  != 228080 || $colpos  != 228081 || $colpos  != 228082 || $colpos  != 228083 || $colpos  != 228084 || $colpos  != 228085 || $colpos  != 228086 || $colpos  != 228196 || $colpos  != 228199$


ifcondition_val=$( grep -v "^#" $config_file_path/GERMANY_Obligor_filter_ctl)

command_construction=$( echo "cat \$SOURCE_DIR/\${FILENAME[\$x]}  | awk -F\"|\" -v colpos=\"\${COLPOS[\$x]}\"  'if (\"\$ifcondition_val\") { print \$0 }'  > \$NEWFILENAME")

eval $command_construction

please help me how to achieve it. i am trying to assign values dynamically. how to compare column position value in "if then else" dynamically?

pilnet101 · July 7, 2014, 5:10pm

Can you please provide sample input and output and re-state your problem a bit clearer if it is possible.

Don_Cragun · July 7, 2014, 9:17pm

I see several things that are likely to fail, but I don't understand what you're trying to do so I don't know how to fix them. Here are a few things for you to consider:

Shell variables are not expanded inside single quoted strings.
Inside awk programs, awk variables don't need a $ unless you are trying to get data from the field in the current input line specified by the value that variable expands to.
Anytime you have an expression starting with variable != value || variable != another_value , the evaluation of that expression is TRUE . Adding more || expression sequences to that expression will never change its value to FALSE .
If an input line that you are processing with awk or grep is longer than LINE_MAX bytes, the results are undefined. You can get the value of LINE_MAX for your system by running the command: getconf LINE_MAX . If colpos != 228070 means that your input lines are 228,070 bytes or more in length, you will not be able to use awk or grep to process those files on some systems.

rajubollas · July 9, 2014, 7:57am

Hi Don,

Thanks for looking into my requirement let me summerize what i am trying achieve.

i have source data in Germany_Source_1.txt, Germany_Source_2.txt ...nth.txt files which contains business keys.
Obligor_ID is business key. using given 18 Obligor_ID i am dividing Germay_Source files for Greece and Germany when Given Obligor_ID not found in each row then i need that row/line as Greece_Source_1.txt .*_2.txt...._nth.txt file found Obligor_ID information i am keeping Greece_Source_1 ...*_2.txt...._nth.txt.
Note: Obligor_ID column position is not a fixed position on every file

i would like to achieve based on source file i have to check Obligor_ID coulm position on the column then i have to write out put to another file whenever list of 19 Obligors not found on the row/line.

Source Data sample

$ Grermany_Source_1.txt file data
1|ACTIVE|FINANCIAL_RI|3456-89-12|456789|RTNG_DLR|RI_FIN_STMT|234
67|ACTIVE|FINANCIAL_RI|9012-45-78|012345|RTNG_DLR|RI_FIN_STMT|890
2|ACTIVE|FINANCIAL_RI|4567-90-23|567890|RTNG_DLR|RI_FIN_STMT|345
78|ACTIVE|FINANCIAL_RI|0123-56-89|123456|RTNG_DLR|RI_FIN_STMT|901
3|ACTIVE|FINANCIAL_RI|5678-01-34|678901|RTNG_DLR|RI_FIN_STMT|456
89|ACTIVE|FINANCIAL_RI|1234-67-90|234567|RTNG_DLR|RI_FIN_STMT|012
4|ACTIVE|FINANCIAL_RI|6789-12-45|789012|RTNG_DLR|RI_FIN_STMT|567
9|ACTIVE|FINANCIAL_RI|1234-67-90|234567|RTNG_DLR|RI_FIN_STMT|012
45|ACTIVE|FINANCIAL_RI|7890-23-56|890123|RTNG_DLR|RI_FIN_STMT|678
01|ACTIVE|FINANCIAL_RI|3456-89-12|456789|RTNG_DLR|RI_FIN_STMT|234
6|ACTIVE|FINANCIAL_RI|8901-34-67|901234|RTNG_DLR|RI_FIN_STMT|789
1|ACTIVE|FINANCIAL_RI|3456-89-12|456789|RTNG_DLR|RI_FIN_STMT|234
6|ACTIVE|FINANCIAL_RI|8901-34-67|901234|RTNG_DLR|RI_FIN_STMT|789
1|ACTIVE|FINANCIAL_RI|3456-89-12|456789|RTNG_DLR|RI_FIN_STMT|234
67|ACTIVE|FINANCIAL_RI|9012-45-78|012345|RTNG_DLR|RI_FIN_STMT|890
23|ACTIVE|FINANCIAL_RI|5678-01-34|678901|RTNG_DLR|RI_FIN_STMT|456
8|ACTIVE|FINANCIAL_RI|0123-56-89|123456|RTNG_DLR|RI_FIN_STMT|901
3|ACTIVE|FINANCIAL_RI|5678-01-34|678901|RTNG_DLR|RI_FIN_STMT|456
8|ACTIVE|FINANCIAL_RI|0123-56-89|123456|RTNG_DLR|RI_FIN_STMT|901
34|ACTIVE|FINANCIAL_RI|6789-12-45|789012|RTNG_DLR|RI_FIN_STMT|567
9|ACTIVE|FINANCIAL_RI|1234-67-90|234567|RTNG_DLR|RI_FIN_STMT|012
4|ACTIVE|FINANCIAL_RI|6789-12-45|789012|RTNG_DLR|RI_FIN_STMT|567
9|ACTIVE|FINANCIAL_RI|1234-67-90|234567|RTNG_DLR|RI_FIN_STMT|012

$$Obligor Test filter file
GERMANY_Obligor_filter_ctl file data

##Obligor information file
##For new obligor for search contion add "&& $colpos !=xxxxxx" in format
##make sure all condition values in single line
$colpos != 228070 && $colpos != 228071 && $colpos != 228072 && $colpos != 228073 && $colpos != 228074 && $colpos != 228075 && $colpos != 228076 && $colpos != 228077 && $colpos != 228078 && $colpos != 228079 && $colpos != 228080 && $colpos != 228081 && $colpos != 228082 && $colpos != 228083 && $colpos != 228084 && $colpos != 228085 && $colpos != 228086 && $colpos != 228196 && $colpos != 228299

##Files Structure on which position we do get Obligor/GSW ID for each file :
crt_extracts_branch_gsw_cntl file data
#################################################
#Carrs Extract File                                         GSWID Column Possition
#################################################
Grermany_Source_1.txt						3
Grermany_Source_2.txt						3
Grermany_Source_3.txt						6			
Grermany_Source_4.txt						1
Grermany_Source_5.txt						1
Grermany_Source_6.txt						4

Script Part:

if [[ -f $config_file_path/crt_extracts_branch_gswid_cntl ]]
then
        set -A FILENAME   $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $1 }')
        set -A COLPOS     $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $4 }')
        set -A CTLCOLPOS  $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $3 }')

	SOURCE_DIR=$( echo "/tgrrappkg3/DataStage/CONSUMER_BI/datap5/WSLGER")
	x=0
        while (( $x <  ${#FILENAME[*]}  ))
        do
                if [[ -f $SOURCE_DIR/${FILENAME[$x]} ]]
                then
                        MSG="Processing file $SOURCE_DIR/${FILENAME[$x]}:INFO"
                        echo $MSG;

                        ##DISTINCT_BRNCH_NBRS=$( cat $SOURCE_DIR/${FILENAME[$x]} | awk -F"|" -v colpos="${COLPOS[$x]}" '{ print $colpos }' | sort -u )
			##DISTINCT_OBLIGOR_NBRS=$( cat $config_file_path/Greece_Obligor_filter.txt	| grep -v "^#"| awk '{ print $1 }' | sort -u )

                        ##for OBLIGOR_NBRS in $( echo $DISTINCT_OBLIGOR_NBRS )
                        ##do
                                ##MSG="Processing branch $OBLIGOR_NBRS:$INFO"
                                ##echo $MSG;
                                PART1FILENAME=$( echo ${FILENAME[$x]} | awk -F"." '{ print $1 }')
                                PART2FILENAME=$( echo ${FILENAME[$x]} | awk -F"." '{ print $2 }')
                                NEWFILENAME=$( echo "$SOURCE_DIR/GERMANY_$PART1FILENAME.$PART2FILENAME")
				touch $NEWFILENAME
                                ##cat $SOURCE_DIR/${FILENAME[$x]}                                                                |
                                ##awk -F"|" -v colpos="${COLPOS[$x]}" -v obligor_nbr="${OBLIGOR_NBRS}" '
                                ##          ( $colpos==obligor_nbr ) { print $0 }'                                          > $NEWFILENAME
				##Reading GERMANY_Obligor_filter_ctl file
				##set -x
				if [[ -f $config_file_path/GERMANY_Obligor_filter_ctl ]]
				then 
					##this an if then else condition values for command_construction variable
					ifcondition_val=$( grep -v "^#" $config_file_path/GERMANY_Obligor_filter_ctl)
					command_construction=$( echo "cat $SOURCE_DIR/${FILENAME[$x]}  | awk -F\"|\" -v colpos=\"${COLPOS[$x]}\"  '{if ("$ifcondition_val") { print \$0 }}'  > $NEWFILENAME")
					##Executing cunstructed command with EVAL command
					eval $command_construction
                                	RC=$?
                                	if [[ $RC -ne 0 ]]
                                	then
                                        MSG="Error in creating file $NEWFILENAME:ALERT"
                                        echo $MSG;
                                        return -1
                                	else

                                        MSG="$NEWFILENAME file created:INFO"
                                        echo $MSG;
                                	fi
				fi
				##set +x
                         ##done
			MSG="Completed processing file ${FILENAME[$x]}"
                        echo $MSG;
                else
                        MSG="File not found $SOURCE_DIR/${FILENAME[$x]}.     Proceeding to next file:WARN"
                        echo $MSG;
                fi
                x=$(echo "$x+1" | bc)
        done
else

###

I have corrected command_construction variable properly, now my script is working fine. but i am still looking for better solution on below part of script to maintain it easily.

Can you please suggest with good solution in below part of script how maintain it easy.

					##this an if then else condition values for command_construction variable
					ifcondition_val=$( grep -v "^#" $config_file_path/GERMANY_Obligor_filter_ctl)
					command_construction=$( echo "cat $SOURCE_DIR/${FILENAME[$x]}  | awk -F\"|\" -v colpos=\"${COLPOS[$x]}\"  '{if ("$ifcondition_val") { print \$0 }}'  > $NEWFILENAME")
					##Executing cunstructed command with EVAL command
					eval $command_construction

rajubollas · July 15, 2014, 12:49am

Hi All,

I am still waiting for other work around.
Please suggesst me

Don_Cragun · July 20, 2014, 6:49pm

I have been trying off an on to figure out how your code works and have been unable to do so. I assume that this is the contents of $config_file_path/crt_extracts_branch_gswid_cntl :

##Files Structure on which position we do get Obligor/GSW ID for each file :
crt_extracts_branch_gsw_cntl file data
#################################################
#Carrs Extract File                                         GSWID Column Possition
#################################################
Grermany_Source_1.txt						3
Grermany_Source_2.txt						3
Grermany_Source_3.txt						6			
Grermany_Source_4.txt						1
Grermany_Source_5.txt						1
Grermany_Source_6.txt						4

So after running the commands in your script:

        set -A FILENAME   $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $1 }')
        set -A COLPOS     $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $4 }')
        set -A CTLCOLPOS  $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $3 }')

FILENAME is an array containing crt_extracts_branch_gsw_cntl and Grermany_Source_1.txt through Grermany_Source_6.txt , COLPOS is an array containing nothing, and CTLCOLPOS is an array containing the string data . Moving on from there nothing makes any sense to me. It would seem that you would be better off running awk once to extract the three fields you want and read the output in a while loop instead of creating three arrays to be processed later.

I'm sorry, but if I can't understand how this code is working, I don't know how to make it work faster.

Obviously you could get rid of the three calls to cat and the three calls to grep in the above pipelines and just use:

        arrayname=($(awk '!/^#/{print $n}))

for various values of arrayname and n to speed up the code above, and you could change:

	SOURCE_DIR=$( echo "/tgrrappkg3/DataStage/CONSUMER_BI/datap5/WSLGER")

to the simpler and faster:

	SOURCE_DIR="/tgrrappkg3/DataStage/CONSUMER_BI/datap5/WSLGER"

but these things are only done once; so they won't have a big effect on the speed of the inner loop.

So, since your code is working for you as is, I don't think I can help much.

You have lots of command substitutions that could be replaced by simple (and faster) parameter expansions. You have a command construction segment and an eval that seem like they could be replaced with a simple command invocation. But I hesitate to make any suggestions here because I can't figure out what is going on with the data you've shown us.

We might have a better chance of understanding what you were doing if you would show us the actual contents of the files named in your script and label those files with their names (as used in your script). AND, please use CODE tags for you sample input files so we can see what is actually in the files versus what may just be descriptive text before or after the contents of those files.

rajubollas · July 20, 2014, 11:58pm

Thanks Don, i will take your inputs for faster and simple scripts going forward.
Please find here my actual code.

#!/bin/ksh


#########################################################################
# Date:	25 June 2014							#
# Author : Raju Bolla							# 
# Email: raju.bolla@hp.com						#
# This script will Segregate data from CARRs Germany source files	#
# Germany Files For data load activities				#
# 									#
# Sustain will execute this Script as a last pre-load activty step	#
#									#
#########################################################################



#Function to check the return code of the command executed
chckRC()
	{
		if [ $1 -ne 0 ]
		then
				echo "$2" >/dev/null
				exit 1
		fi	
	}

##Copy Germany Files to GERMANY Folder
##Copy Germany Files to Greece Folder
#Bkp Path Germany Path
SOURCE_COMMON=$( echo "/pgrrappkg3/DataStage/CONSUMER_BI/datap5/WSLGER_WSLGREECE")
TARGET_GREECE=$( echo "/pgrrappkg3/DataStage/CONSUMER_BI/datap5/WSLGER") 
cd $SOURCE_COMMON
cp Crs_*Extract.txt $TARGET_GREECE/
cp Crs_*CTL.txt $TARGET_GREECE/
RC=$?
chckRC $RC "Aborted while executing Copy Germany Files to Greece Folder Part"
echo "Completed the execution of Copy Germany Files to Greece Folder Part" >/dev/null

cd $TARGET_GREECE
config_file_path=$( echo "/pgrrappkg3/DataStage/CONSUMER_BI/etlisp5/cfg")
if [[ -f $config_file_path/crt_extracts_branch_gswid_cntl ]]
then
        set -A FILENAME   $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $1 }')
        set -A COLPOS     $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $4 }')
        set -A CTLCOLPOS  $( cat $config_file_path/crt_extracts_branch_gswid_cntl | grep -v "^#" | awk '{ print $3 }')

	SOURCE_DIR=$( echo "/pgrrappkg3/DataStage/CONSUMER_BI/datap5/WSLGER")
	x=0
        while (( $x <  ${#FILENAME[*]}  ))
        do
                if [[ -f $SOURCE_DIR/${FILENAME[$x]} ]]
                then
                        MSG="Processing file $SOURCE_DIR/${FILENAME[$x]}:INFO"
                        echo $MSG;

                                PART1FILENAME=$( echo ${FILENAME[$x]} | awk -F"." '{ print $1 }')
                                PART2FILENAME=$( echo ${FILENAME[$x]} | awk -F"." '{ print $2 }')
                                NEWFILENAME=$( echo "$SOURCE_DIR/GERMANY_$PART1FILENAME.$PART2FILENAME")
				touch $NEWFILENAME
				##Reading GERMANY_Obligor_filter_ctl file
				##set -x
				if [[ -f $config_file_path/GERMANY_Obligor_filter_ctl ]]
				then 
					##this an if then else condition values for command_construction variable
					ifcondition_val=$( grep -v "^#" $config_file_path/GERMANY_Obligor_filter_ctl)
					command_construction=$( echo "cat $SOURCE_DIR/${FILENAME[$x]}  | awk -F\"|\" -v colpos=\"${COLPOS[$x]}\"  '{if ("$ifcondition_val") { print \$0 }}'  > $NEWFILENAME")
					##Executing cunstructed command with EVAL command
					eval $command_construction
                                	RC=$?
                                	if [[ $RC -ne 0 ]]
                                	then
                                        MSG="Error in creating file $NEWFILENAME:ALERT"
                                        echo $MSG;
                                        return -1
                                	else

                                        MSG="$NEWFILENAME file created:INFO"
                                        echo $MSG;
                                	fi
				fi
				##set +x

				##CTL File generation
				for CTL in $( echo ${CTLCOLPOS[$x]} | awk '{ FldCnt=split($0,Arr,"-"); print FldCnt }' )
				do
					CTLCTR=1
					DISPLAYVAL=""
					while (( $CTLCTR <= $CTL ))
					do
						VAL=$( echo ${CTLCOLPOS[$x]} | awk -v ctr="${CTLCTR}" '{ FldCnt=split($0,Arr,"-"); print Arr[ctr] }')
						if [[ $VAL = 0 ]];
						then
						    ROWCNT=$(cat $NEWFILENAME | wc -l ) 
						    DISPLAYVAL="$DISPLAYVAL""$ROWCNT|"
						else
						    SUMVAL=$(cat $NEWFILENAME | awk -F"|" -v val="${VAL}" ' 
										BEGIN { tot=0 } {tot=tot+$val } END { printf "%.2f",tot }')
						    DISPLAYVAL="$DISPLAYVAL""$SUMVAL|"
						fi
						CTLCTR=$(echo "$CTLCTR+1" | bc)
					done
					CTLNEWFILENAME=$( echo $NEWFILENAME 	| sed s/Extract/CTL/g )
					CTLORGFILENAME=$( echo ${FILENAME[$x]}	| sed s/Extract/CTL/g )
					if [[ -f $SOURCE_DIR/$CTLORGFILENAME ]]
					then
						cat $SOURCE_DIR/$CTLORGFILENAME | head -1  	> $CTLNEWFILENAME
					else
						echo "No header for CTL file found" 		> $CTLNEWFILENAME
						MSG="$CTLORGFILENAME file not found :WARN" 
						echo $MSG;
					fi		
					echo $DISPLAYVAL | sed s/\|$//g				>> $CTLNEWFILENAME
				done
                        MSG="Completed processing file ${FILENAME[$x]}"
                        echo $MSG;
                else
                        MSG="File not found $SOURCE_DIR/${FILENAME[$x]}. Proceeding to next file:WARN"
                        echo $MSG;
                fi
                x=$(echo "$x+1" | bc)
        done
else
  MSG="Invalid file $config_file_path/crt_extracts_branch_gswid_cntl:ALERT"
  echo $MSG;
fi;
## End of Generating Greece and Germany Files GREECE_* and Names

##Modifying Germany Data with Greece data and writing to Sed_*GREE
##Greece Files Path
cd $SOURCE_DIR
##Renaming file to Atcual name
for o in `ls -1 GERMANY_Crs_*_Extract.txt`
do
Actual_FILE_Name=`echo "$o" | sed -e 's/GERMANY\_//'`; ##Indetifying Extract File Name
mv $o $Actual_FILE_Name
RC=$?
chckRC $RC "Aborted while executing Renaming file to Atcual name Part"
echo "Completed the execution of Renaming file to Atcual name Part" >/dev/null
done;

##Renaming file to Atcual name
for p in `ls -1 GERMANY_Crs_*_CTL.txt`
do
Actual_FILE_Name=$( echo "$p" | sed -e 's/GERMANY\_//'); ##Indetifying Extract File Name
mv $p $Actual_FILE_Name
RC=$?
chckRC $RC "Aborted while executing Renaming file to Atcual name Part"
echo "Completed the execution of Renaming file to Atcual name Part" >/dev/null
done;
##set +x
###End of Script

Please find control files data.

cat GERMANY_Obligor_filter_ctl
##Obligor information file
##For new obligor for search contion add "&& $colpos !=xxxxxx" in format
##make sure all condition values in single line
$colpos != 228070 && $colpos != 228071 && $colpos != 228072 && $colpos != 228073 && $colpos != 228074 && $colpos != 228075 && $colpos != 228076 && $colpos != 228077 && $colpos != 228078 && $colpos != 228079 && $colpos != 228080 && $colpos != 228081 && $colpos != 228082 && $colpos != 228083 && $colpos != 228084 && $colpos != 228085 && $colpos != 228086 && $colpos != 228196 && $colpos != 228299

cat crt_extracts_branch_gsw_cntl

################################################################################
#
#  carrs_extracts_branch_gswid_cntl
#
################################################################################
#
# carrs_extracts_branch_cntl is a control file containing carrs extract file names
# and column position of Branch nbr in that extract file
#
################################################################################
#
# Column Definition:
# 1st Column     Carrs Extract file name
# 2nd Column     Colump position of "Branch Number"
# 4th Column     Colump position of "GSW ID or Obligor ID"
##############################################################################################################################################
#Carrs Extract File                                             Column Position     Control File column position	GSWID Column Possition
##############################################################################################################################################
Crs_Assessment_Extract.txt                                       4                     0-17					3
Crs_Involved_Party_Risk_Rating_Extract.txt                       4                        0					3
Crs_Model_Input_Extract.txt                                      7                     0-12					6			
Crs_Involved_Party_Watchlist_Extract.txt                         2                        0					1
Crs_Involved_Party_Default_Extract.txt                           2			0-9					1
Crs_AR_X_IP_Extract.txt                                          3                        0					4
Crs_Oprtnl_Risk_Assessment_Extract.txt                           4                        0					3
Crs_Financial_Resource_Item_Extract.txt                         11                      0-5					8
#Crs_County_Extract.txt                                           5                        0
#Crs_State_Extract.txt                                            4                        0
#Crs_City_Extract.txt                                             6                        0
Crs_Involved_Party_Involved_Party_Extract.txt                    6                        0					1
Crs_Customer_Obligor_Extract.txt                                 6                        0					15	
#Crs_Post_Addr_Extract.txt                                       13                        0
Crs_Involved_Party_Post_Address_Extract.txt	                 23                       0					3
#Crs_IP_Product_Plan_Grp_Rating_Extract.txt                       8                        0					1
Crs_Financial_Resource_Item_Hdr_Extract.txt                      8                        0					5
Crs_IP_Credit_Facility_Rating_Extract.txt                        2                     0-10					1
Crs_Txn_Ev_Dlr_Charges_Extract.txt                               1                     0-10					2

Please provide your suggestion on above code

command_construction=$( echo "cat $SOURCE_DIR/${FILENAME[$x]}  | awk -F\"|\" -v colpos=\"${COLPOS[$x]}\"  '{if ("$ifcondition_val") { print \$0 }}'  > $NEWFILENAME")

especially here i am looking for a refine my script and simple use of

cat GERMANY_Obligor_filter_ctl

file

Don_Cragun · July 21, 2014, 5:52pm

We have given you several suggestions.

We have asked you to show us the contents of the files your script is using (and you may have hidden some of those file under the heading "please find control files data"), but I don't want to try to guess at what parts of what you have shown us correspond to the file $config_file_path/crt_extracts_branch_gswid_cntl what parts of what you have shown us correspond to $SOURCE_DIR/${FILENAME[$x]} , what parts of what you have shown us correspond to the other files used by your script.

If you don't want to make use of those suggestions and just want us to rewrite your script for you, you have come to the wrong place.

If you want to learn how to write better code, clearly show us that you have tried to use our suggestions, show us clearly the data you're trying to process, and show us what isn't working.

Michael_Stora · July 21, 2014, 5:58pm

I learned something today.

Are you saying all 20 must be on the line or only one of the 20?

If it is one of 20, something like this should work:

grep -f obligors.txt myfile

obligors.txt

Obligor1
Obligor2
Obligor3

Mike

rajubollas · July 21, 2014, 11:44pm

Hi Don,

i have given control files data not the actual files data as it is huge in size.
here i am attaching one sample for your reference.
i have modified file extension *.txt as was unable to upload control files

rajubollas · July 21, 2014, 11:59pm

before script run file data

cat Crs_Assessment_Extract.txt
NA|ASSESSMENT|567890|234|6789-12-45 78:01:34|678901|ASSESSMENT|OPRTNL|PENDING|9012-45-78|fake|name|wzsf0d||45||8.01234567890123456||RTNG_DLR||901234
NA|ASSESSMENT|012345|789|1234-67-90 23:56:89|123456|ASSESSMENT|OPRTNL|PENDING|4567-90-23|who|knows|xzhqmv|CARRS rating of the 78/0123 operating report which are the financials of the merged companies and the RE purchase |23||6.89012345678901234||RTNG_DLR|678|012345
NA|ASSESSMENT|123456|890|2345-78-01 34:67:90|234567|ASSESSMENT|OPRTNL|PENDING|5678-01-34|some|one|xzhqmv|CARRS rating of the 89/1234 operating report which are the financials of the merged companies and the RE purchase |34||7.90123456789012345||RTNG_DLR|789|123456

after script run my output should be as below on every line i have to check 20 values on each line, none of the value not found in

GERMANY_Obligor_filter_ctl.txt

row/line i am writing to output

cat Crs_Assessment_Extract.txt
NA|ASSESSMENT|567890|234|6789-12-45 78:01:34|678901|ASSESSMENT|OPRTNL|PENDING|9012-45-78|fake|name|wzsf0d||45||8.01234567890123456||RTNG_DLR||901234

cat GERMANY_Obligor_filter_ctl
##Obligor information file
##For new obligor for search contion add "&& $colpos !=xxxxxx" in format
##make sure all condition values in single line
$colpos != 228070 && $colpos != 228071 && $colpos != 228072 && $colpos != 228073 && $colpos != 228074 && $colpos != 228075 && $colpos != 228076 && $colpos != 228077 && $colpos != 228078 && $colpos != 228079 && $colpos != 228080 && $colpos != 228081 && $colpos != 228082 && $colpos != 228083 && $colpos != 228084 && $colpos != 228085 && $colpos != 228086 && $colpos != 228196 && $colpos != 228299