Compare Values between column in the same file

Input File:-

 
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0
VEGE Onion S W 17 0 17 0 0 0 0 0 0
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 

The value in Column 3 should determine that among columns $7(SMS),$8(EMAIL),$9(AO),$10(Mail),$11(P),$12(N) and $13(cell), which column will have a non-zero integer value and which will have value zero.

eg.
example 1- if column 3 has value E then code should check only column 7,9,10,11,12,13 and sum of these columns (7+9+10+11+12+13) should be zero . If it is Zero then add GOOD else add WARNING to Newcolumn.
like in row 1 in the above file the sum of the 7,9,10,11,12,13 columns is zero so added Good in the out put file.
example 2 -if Column 3 has value SE then code should only check columns 9,10,11,12,13 and sum of these columns (9+10+11+12+13) should be zero.If it is Zero then add GOOD else add WARNING to Newcolumn.
like in row 4 the sum of columns 9,10,11,12,13 is not zero so added warning to the output file.

Now, single character value like E, S, A etc is fine but when there is a string like SE or AM or EAMS then there is a problem.

Like row 5 column 3 has values EAMS so it should only check column 11,12,13 and if those columns are greater then zero then add WARNING to Newcolumn.

Column 3 can be combination of any predefined values in no particular sequence eg. EM,ES,E,S,M,EAMS,EMAS,EP,P,SEC,SEA and more.

Conditions based on which code will decide GOOD or WARNING:-

if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= M then check column 7,8,9,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= P then check column 7,8,9,10,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= N then check column 7,8,9,10,11,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= C then check column 7,8,9,10,11,12 (values should be zero in each) and hence sum of these columns should be zero.

I know its is very confusing. Please help. For single values(can be E or ES or EAM or ...) i am using below code. I have taken E as an example.

awk 'BEGIN{FS=OFS="\t"} NR<2{print $0,"NewColumn";next} {(($3 == "E") && ((($7+$9+$10+$11+$12+$13) >0))) ?c="WARNING"$14:c="GOOD"$14;printf "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,c;}' Input_File

Desired OUTPUT

 
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell Newcolumn
VEGE Potato E W 396 12 0 384 0 0 0 0 0 GOOD
VEGE Onion S W 17 0 17 0 0 0 0 0 0 GOOD
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 GOOD
 FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 WARNING
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0 GOOD
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 WARNING
 

You are correct. This is very confusing. You show us code, but do not tell us if it is doing what you want for the single case it seems to try to handle, and for all of the other cases you don't even say what output you do want. How do you expect us to help you write code if you don't clearly explain what that code is supposed to do?

Please show us the exact output that you want your script to produce for the sample input you provided, and explain in detail what output is supposed to be produced for any given value found in field 3 in your input file.

Is there supposed to be a header line in the output produced? If so, what header is supposed to appear in the output?

1 Like

@Don. Thank you for reading such a long post and your feedback. I tried to edit it.Please let me know if it is readable to you now.Otherwise I will try to edit it more

Hello Nina2910,

Could you please try following and let me know if this helps you.

awk 'function sum_check(a,b,c,d,e,f){if(($a+$b+$c+$d+$e+$f)==0){value="Good"} else {value="warning"}}  function cal(w){if(length($w)==1){if($w=="E"){sum_check(7,9,10,11,12,13)};if($3=="S"){sum_check(8,9,10,11,12,13)};if($3=="A"){sum_check(7,8,10,11,12,13)};if($3=="M"){sum_check(7,8,9,11,12,13)};if($3=="N"){sum_check(7,8,9,10,12,13)};if($3=="C"){sum_check(7,8,9,10,11,12)};$(NF+1)=value};if($3 ~ /[SEAMPC][SEAMPC]/){sum_check(11,12,13);$(NF+1)=value}} {cal(3);print}'  Input_file

Output will be as follows.

COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0 Good
VEGE Onion S W 17 0 17 0 0 0 0 0 0 Good
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 warning
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 Good
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 Good

Also not sure how you output shown in last row and 3rd last row has warning in them as I could see their 11st, 12th and 13th field's sum is 0 only.
If it was a typo then please try above and let me know if this helps. Also rushing to somewhere so will add non-one liner form little later.
EDIT: Adding a non-one liner form of solution too now.

awk 'function sum_check(a,b,c,d,e,f){
                                        if(($a+$b+$c+$d+$e+$f)==0){
                                                                        value="Good"
                                                                  }
                                        else                      {
                                                                        value="warning"
                                                                  }
                                    }
     function cal(w)                {
                                        if(length($w)==1)         {
                                                                        if($w=="E"){
                                                                                        sum_check(7,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="S"){
                                                                                        sum_check(8,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="A"){
                                                                                        sum_check(7,8,10,11,12,13)
                                                                                   };
                                                                        if($w=="M"){
                                                                                        sum_check(7,8,9,11,12,13)
                                                                                   };
                                                                        if($w=="N"){
                                                                                        sum_check(7,8,9,10,12,13)
                                                                                   };
                                                                        if($w=="C"){
                                                                                        sum_check(7,8,9,10,11,12)
                                                                                   };
                                                                        $(NF+1)=value
                                                                  };
                                        if($w ~ /[SEAMPC][SEAMPC]/){
                                                                        sum_check(11,12,13);
                                                                         $(NF+1)=value;
                                                                   }
                                    }
                                    {
                                        cal(3);
                                        print
                                    }
    '    Input_file
 

EDIT2: Improving the solution above by putting logic where once the match found in any of if condition then it shouldn't execute further conditions to save time of execution here.

awk 'function sum_check(a,b,c,d,e,f){
                                        if(($a+$b+$c+$d+$e+$f)==0){
                                                                        value="Good"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                        else                      {
                                                                        value="warning"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                    }
     function cal(w)                {
                                        if(length($w)==1)         {
                                                                        if($w=="E"){
                                                                                        sum_check(7,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="S"){
                                                                                        sum_check(8,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="A"){
                                                                                        sum_check(7,8,10,11,12,13)
                                                                                   };
                                                                        if($w=="M"){
                                                                                        sum_check(7,8,9,11,12,13)
                                                                                   };
                                                                        if($w=="N"){
                                                                                        sum_check(7,8,9,10,12,13)
                                                                                   };
                                                                        if($w=="C"){
                                                                                        sum_check(7,8,9,10,11,12)
                                                                                   };
                                                                        
                                                                  };
                                        if($w ~ /[SEAMPC][SEAMPC]/){
                                                                        sum_check(11,12,13);
                                                                        
                                                                   }
                                    }
                                    {
                                        cal(3);
                                        print
                                    }
    '    Input_file
 

Thanks,
R. Singh

1 Like
#!/bin/bash

INPUT_FILE=input.txt

awk 'BEGIN{
E_Len=split("7,9,10,11,12,13",E_Arr,",");
}
{	
	if($3 == "E")
	{
		for(i=1;i<=E_Len;i++)
		{
			col_num=E_Arr;
			sum+=$col_num;
		}
		if ( sum == 0)
		{
			print $0",Good";
		}
		else
		{
			print $0",Bad";
		}
	}
	if(length($3)>1)
	{
		sum=$11+$12+$13
		if (sum>0)
		{
			print $0",Good";
		}
		else
		{
			print $0",Bad";
		}
	}
}' ${INPUT_FILE}

you have to repeat the read color coding for other checks as well

1 Like

Please STOP changing your posts. If you keep updating post #1 in your threads, no one can follow the conversation we are having trying to get a clear specification of your problem. In the future, please stop and think about what you are asking us to help you do before you submit your first post!

If you need to add more details, do so in subsequent posts; don't keep changing post #1 in a thread. During the time that I have been trying to compose this message, you have edited post #1 at least a dozen times. How can you expect anyone to help you if you keep changing the description of what you want done!

You have now told us what columns need to be added when column 3 is one of A, C, E, M, N, P, S, SE, or EAMS. And you have told us that column 3 can have numerous other values with absolutely no specification of what is supposed to happen in those cases. (Unless you have changed things again and I missed it.)

Please give us a clear description of what needs to be done and then ask for help. I'm tired to trying to keep up with a constantly changing, incomplete description of what you want done.

You constantly state that a list of fields have to be added, but from your sample data that seems like a waste of time. Since none of your input fields seem to have negative values, can your script just add warning (which is what is in your description) or Warning (which is what is in your code) or WARNING (which is what is in your sample output) instead of Good (which is in your description) or Goodd (which is what is in your code) or GOOD (which is what is in your sample output) when any of the specified fields contains a non-zero value (instead of adding them all together and then looking for a non-zero sum)?

---------- Post updated at 12:59 AM ---------- Previous update was at 12:58 AM ----------

@Don... Sorry for numerous updates . Trust me I was just trying to make it as clear as possible and I am done updating .. Sorry for inconvenience today.

Great. I am glad that you are done updating.

Now, please answer all of the questions I asked in post #6 in this thread. And, please give us a clear description of what fields need to be checked for all of the possible combinations that might appear in field #3 in your input.

1 Like

@Don ...I have to answer your questions in blue color.
You have now told us what columns need to be added when column 3 is one of A, C, E, M, N, P, S, SE, or EAMS. And you have told us that column 3 can have numerous other values with absolutely no specification of what is supposed to happen in those cases. (Unless you have changed things again and I missed it.)
column 3 can be any of these Values A, C, E, M, N, P, S or combinations of these characters eg. AS,SE,MP,PS,SAE,EAM, CEPS and more
I updated the intial post yesterday to clarify what columns should be added when column 3 have any values among A, C, E, M, N, P, S.
for the combination, code should check only common columns for all single character in combination.
example 1-
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
so if it is ES or SE, then code should check common columns which in this case are 9,10,11,12,13 and sum of these should zero.

example 2-
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.

so if column 3 Value is SAE, EAS,ASE, then code should check common columns which in this case are 10,11,12,13 and sum of these should zero.

Please give us a clear description of what needs to be done and then ask for help. I'm tired to trying to keep up with a constantly changing, incomplete description of what you want done.
You constantly state that a list of fields have to be added, but from your sample data that seems like a waste of time. Since none of your input fields seem to have negative values.

Row 4 in the INPUT file in which column 3 has Values SE , so code should check 9,10,11,12,13 columns and sum of should be zero which is not the case.
Input:-
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0
sum of the column 9+10+11+12+13 >0 in the above case. That's the reason output file has WARNING in the Newcolumn.
Output:-
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell Newcolumn
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 WARNING

can your script just add warning (which is what is in your description) or Warning (which is what is in your code) or WARNING (which is what is in your sample output) instead of Good (which is in your description) or Goodd (which is what is in your code) or GOOD (which is what is in your sample output) when any of the specified fields contains a non-zero value (instead of adding them all together and then looking for a non-zero sum)?

Updated it already in the initial post.

---------- Post updated at 11:20 AM ---------- Previous update was at 10:47 AM ----------

@Ravinder..Thank you for the code however for row number 4 and 6 the code should have output

COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell Newcolumn
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 WARNING
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 WARNING

column 3 can be any of these Values A, C, E, M, N, P, S or combinations of these characters eg. AS,SE,MP,PS,SAE,EAM, CEPS and more

Mainly it would be in A ,E, M,P,S,C,N,AM,EA,EM,SC,SM,ES,SE ,AS,SA,ESA ,SEA, EMS,ESM,SEM ,SCE,SEC, EAM,SAM,EAMS,EASM,ESAM,SEAM,ESCM.
I updated the intial post yesterday to clarify what columns should be added when column 3 have any values among A, C, E, M, N, P, S.
for the combination, code should check only common columns for all single character in combination.
example 1-
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
so if it is ES or SE, then code should check common columns which in this case are 9,10,11,12,13 and sum of these should zero.

example 2-
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.

so if column 3 Value is SAE, EAS,ASE, then code should check common columns which in this case are 10,11,12,13 and sum of these should zero.

---------- Post updated at 11:26 AM ---------- Previous update was at 11:20 AM ----------

@Don ..One more update

I Dig more and found that the expected column 3 values would be either of these ::- A ,E, M,P,S,C,N,AM,EA,EM,SC,SM,ES,SE ,AS,SA,ESA ,SEA, EMS,ESM,SEM ,SCE,SEC, EAM,SAM,EAMS,EASM,ESAM,SEAM,ESCM.

however if the script is able to Calculate the values based on Dynamic combinations that would really helpful.

Thanks again

---------- Post updated at 01:28 PM ---------- Previous update was at 11:26 AM ----------

@Ravinder I have update the code you according to me. however it will work only if I know values of column $3 in advance.

 
 awk 'BEGIN{FS=OFS="\t"} NR<2{print $0,"NewColumn";next} function sum_check(a,b,c,d,e,f){ 
                                        if(($a+$b+$c+$d+$e+$f)==0){
                                                                        value="GOOD"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                        else                      {
                                                                        value="WARNING"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                    }
     function cal(w)                {
                                        if(length($w)==1)         {
                                                                        if($w=="E"){
                                                                                        sum_check(7,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="S"){
                                                                                        sum_check(8,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="A"){
                                                                                        sum_check(7,8,10,11,12,13)
                                                                                   };
                                                                        if($w=="M"){
                                                                                        sum_check(7,8,9,11,12,13)
                                                                                   };
                                                                        if($w=="N"){
                                                                                        sum_check(7,8,9,10,12,13)
                                                                                   };
                                                                        if($w=="C"){
                                                                                        sum_check(7,8,9,10,11,12)
                                                                                   };
                                                                        
                                                                  }
if(length($w)==2)         {
                                                                        if($w=="SE"||$w=="ES"){
                                                                                        sum_check(9,10,11,12,13)
                                                                                   };
                                                                  }
if(length($w)==3)         {
                                                                        if($w=="SEA"){
                                                                                        sum_check(10,11,12,13)
                                                                                   };
                                                                  }
if(length($w)==4)         {
                                                                        if($w=="EAMS"){
                                                                                        sum_check(11,12,13)
                                                                                   };
                                                                  };
                                        if($w ~ /[SEAMPC][SEAMPC]/){
                                                                        sum_check(11,12,13);
                                                                        
                                                                   }
                                    }
                                    {
                                        cal(3);
                                        print
                                    }
    '    INPUT_file

 

Thank you. I was thinking if I can have more dynamic values.

How about

awk '
BEGIN   {MX = split ("      SEAMPNC", CH, _)
        }
NR > 1  {SUM = 0
         for (i = 7; i<=13; i++) if ($3 !~ CH) SUM += $i
         $(NF+1) = SUM?"WARNING":"GOOD"
        }
1
 ' file
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0 GOOD
VEGE Onion S W 17 0 17 0 0 0 0 0 0 GOOD
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 GOOD
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 WARNING
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0 GOOD
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 WARNING

Please be aware that your conditions in post#1 seem to be inconsistent for "P" and "N".

1 Like

Hello Nina2910,

Let's say I have edited your Input_file as follows.

cat Input_file
COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0
VEGE Onion S W 17 0 17 0 0 0 0 0 0
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 10
FRUIT APPLE EA W 397 32 309 56 309 309 0 0 0
FRUIT APPLE MS W 397 32 309 56 309 309 0 0 10
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0
FRUIT APPLE EAS W 397 32 309 56 309 309 0 0 0
FRUIT APPLE EAM W 397 32 309 56 309 309 0 0 0
FRUIT APPLE SMEAC W 808 58 663 87 488 20 0 10 0
FRUIT APPLE CSEAN W 808 58 663 87 488 20 0 10 0
FRUIT APPLE MCSEA W 808 58 663 87 488 20 0 01 0

Then when I run my code posted in POST#3 as follows is the result.

COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell
VEGE Potato E W 396 12 0 384 0 0 0 0 0 Good
VEGE Onion S W 17 0 17 0 0 0 0 0 0 Good
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 warning
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 10 warning
FRUIT APPLE EA W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE MS W 397 32 309 56 309 309 0 0 10 warning
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 Good
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE EAS W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE EAM W 397 32 309 56 309 309 0 0 0 Good
FRUIT APPLE SMEAC W 808 58 663 87 488 20 0 10 0 warning
FRUIT APPLE CSEAN W 808 58 663 87 488 20 0 10 0 warning
FRUIT APPLE MCSEA W 808 58 663 87 488 20 0 01 0 warning

So could you please let me know about following question's answers here.
i- Is above code working as per your expectations or not?
ii- So as per your requirements only strings AEMCSN or specific strings mentioned by you could be in $3, if this is true then in code for single character conditions are there(for different single character different fields we need to add and compare so they are different in conditions there), you have mentioned it needs to be more dynamic, could you please elaborate it more.
Please try to rephrase your requirements in a single post with code tags please, with sample Input_file and expected output sample too.

EDIT: BY seeing your try to edit my code got a little pulse of your objective here(may be), so could you try follwing and let us know if this helps you.

awk 'function sum_check(a,b,c,d,e,f){
                                        if(($a+$b+$c+$d+$e+$f)==0){
                                                                        value="Good"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                        else                      {
                                                                        value="warning"
                                                                        $(NF+1)=value;
                                                                        print $0;
                                                                        next;
                                                                  }
                                    }
     function cal(w)                {
                                        if(length($w)==1)         {
                                                                        if($w=="E"){
                                                                                        sum_check(7,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="S"){
                                                                                        sum_check(8,9,10,11,12,13)
                                                                                   };
                                                                        if($w=="A"){
                                                                                        sum_check(7,8,10,11,12,13)
                                                                                   };
                                                                        if($w=="M"){
                                                                                        sum_check(7,8,9,11,12,13)
                                                                                   };
                                                                        if($w=="N"){
                                                                                        sum_check(7,8,9,10,12,13)
                                                                                   };
                                                                        if($w=="C"){
                                                                                        sum_check(7,8,9,10,11,12)
                                                                                   };
                                                                        
                                                                  };
                                        if(length($w)==2 && $w ~ /[SEAMPC][SEAMPC]/)         {
												sum_check(9,10,11,12,13);
											     }
					if(length($w)==3 && $w ~ /[SEAMPC][SEAMPC]/)          { 
											 	sum_check(10,11,12,13);
											     }	
									
					if(length($w)==4 && $w ~ /[SEAMPC][SEAMPC]/){
                                                                        sum_check(11,12,13);
                                                                        
                                                                   }
                                    }
                                    {
                                        cal(3);
                                        print
                                    }
    '   Input_file

Thanks,
R. Singh

1 Like

@Ravinder your code is partially satisfying the expectation.
I am stating the conditions again on which code will decide GOOD or WARNING for Single character Values of column 3 (A, C, E, M, N, P, S).

if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= M then check column 7,8,9,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= P then check column 7,8,9,10,11,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= N then check column 7,8,9,10,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= C then check column 7,8,9,10,11,12 (values should be zero in each) and hence sum of these columns should be zero.

as column 3 can be any of these Values A, C, E, M, N, P, S or combinations of these characters eg. AS,SE,MP,PS,SAE,EAM, CEPS and more.The code should be able to Calculate the values based on Dynamic combinations.

for the combination, code should check only common columns for all single character in combination.
example 1-
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
so if it is ES or SE, then code should check common columns which in this case are 9,10,11,12,13 and sum of these should zero.
example 2-
if Column 3 has value= S then check column 8,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= A then check column 7,8,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
if Column 3 has value= E then check column 7,9,10,11,12,13 (values should be zero in each) and hence sum of these columns should be zero.
so if column 3 Value is SAE, EAS,ASE, then code should check common columns which in this case are 10,11,12,13 and sum of these should zero.

---------- Post updated at 03:06 PM ---------- Previous update was at 02:54 PM ----------

@Rudi ...row 3,4,6 should have "WARNING" in the output file. I have fixed the my output file in post #1.

If you ignore the heading and start counting from the row after it, row 4 and 6 in my result HAVE "WARNING".
Are you sure that "P" and "N" need to ignore the SAME column (=11)? Or, shouldn't "N" ignore col 12?

1 Like

@Rudi...yes you are right and code is working perfectly fine..Thank you.

would it be possible if you can explain the code ?

I agree with RudiC that your specification (in both post #1 [after 20 edits] and in post #12) saying:

appears to be a mistake. And to get the output you said you want, it would seem that if field #3 is N , you really want to check 7, 8, 9, 10, 11, and 13 instead of checking exactly the same fields for both P and N .

Assuming that that is correct, here is another awk script using the same underlying logic as the code RudiC suggested. This script builds the array of fields to be skipped for each letter than appears in the string in field #3 manually. RudiC's script depends on an extension to the standards (using an empty ERE in split() to create an array from single characters in a string) that is available in GNU awk but is not often supported on versions of awk available on UNIX systems and BSD-based systems. The standards say that the behavior of awk is unspecified when split() is called with an empty string specified as the field separator.

Unlike RudiC's code, this adds a new column to the header line as shown in the output header you said you wanted. And, while RudiC's code adds up all of the fields being checked, this code looks at each field individually and breaks out of the loop immediately if a non-zero value is found in a field that is to be checked.

The following awk script:

awk '
BEGIN {	# Initialize skip array: S[fn] = char
	# If field #3 contains the character specified by char, do NOT check
	# the contents of field #fn.
	S[7] = "S"
	S[8] = "E"
	S[9] = "A"
	S[10] = "M"
	S[11] = "P"
	S[12] = "N"
	S[13] = "C"
}
NR < 2 {# Add the requested heading field to the header line...
	print $0, "Newcolumn"
	# and skip to the next input line.
	next
}
{	# For all other input lines, check fields 7-13 inclusive:
	for(i = 7; i <= 13; i++)
		# If the character corresponding to the field is not prsent in
		# field #3 AND the field contains a non-zero value...
		if($3 !~ S && $i) 
			# break out of the loop.
			break
	# Print the input line followed by "WARNING" if a non-zero value was
	# found in a field to be checked; otherwise, print the input line
	# followed by "GOOD".
	print $0, (i <= 13) ? "WARNING" : "GOOD"
}' file

when file contains the sample input you provided, produces the output:

COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell Newcolumn
VEGE Potato E W 396 12 0 384 0 0 0 0 0 GOOD
VEGE Onion S W 17 0 17 0 0 0 0 0 0 GOOD
FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 GOOD
FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 WARNING
FRUIT APPLE EAMS W 397 32 309 56 309 309 0 0 0 GOOD
FRUIT APPLE SEA W 808 58 663 87 488 20 0 0 0 WARNING

which seems to match the output you said you wanted.

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk .

1 Like

@Don ..Yes you and Rudi were right about P and N . Your code is working fine only one issue when I am running it, it is giving me list of files in that directory and gives the output.

doing something like below

> }
> {# For all other input lines, check fields 7-13 inclusive:
> for(i = 7; i <= 13; i++)
>
.csv                          20160405_StarData/            BCSreplay/                    archive/                      neha.txt                      sync_query_results_26668.txt
20140327_ISP_AAID_migrations/ 25049_CT_DELIVERY2.txt        DSLOrderWTN/                  data_for_demos/               neha2.txt                     sync_query_results_29460.txt
20150818_AppleCreate/         32706_CT_DELIVERY2.txt        MIGRATION_DATA_DIRECTORIES/   dp332j/                       neha3.txt                     sync_query_results_6635.txt
20151008_OMSIssue_DeTitanize/ ActivateHSIA/                 OldProjectDirs/               igate_data/                   other_logs/                   text.txt
20151014_DetitanizeMobility/  AdHoc_Requests/               ProcessEDDFiles/              junk/                         replays/                      tmpUniqAsMob.out
20160314_FULCRUM/             AddOrder_data/                ProjectDirs/                  kr9850/                       sh2818/                       tmpUniqAsMob.sql
> # If the character corresponding to the field is not prsent in
>
.csv                          20160405_StarData/            BCSreplay/                    archive/                      neha.txt                      sync_query_results_26668.txt
20140327_ISP_AAID_migrations/ 25049_CT_DELIVERY2.txt        DSLOrderWTN/                  data_for_demos/               neha2.txt                     sync_query_results_29460.txt
20150818_AppleCreate/         32706_CT_DELIVERY2.txt        MIGRATION_DATA_DIRECTORIES/   dp332j/                       neha3.txt                     sync_query_results_6635.txt
20151008_OMSIssue_DeTitanize/ ActivateHSIA/                 OldProjectDirs/               igate_data/                   other_logs/                   text.txt
20151014_DetitanizeMobility/  AdHoc_Requests/               ProcessEDDFiles/              junk/                         replays/                      tmpUniqAsMob.out
20160314_FULCRUM/             AddOrder_data/                ProjectDirs/                  kr9850/                       sh2818/                       tmpUniqAsMob.sql
> # field #3 AND the field contains a non-zero value...
>
.csv                          20160405_StarData/            BCSreplay/                    archive/                      neha.txt                      sync_query_results_26668.txt
20140327_ISP_AAID_migrations/ 25049_CT_DELIVERY2.txt        DSLOrderWTN/                  data_for_demos/               neha2.txt                     sync_query_results_29460.txt
20150818_AppleCreate/         32706_CT_DELIVERY2.txt        MIGRATION_DATA_DIRECTORIES/   dp332j/                       neha3.txt                     sync_query_results_6635.txt
20151008_OMSIssue_DeTitanize/ ActivateHSIA/                 OldProjectDirs/               igate_data/                   other_logs/                   text.txt
20151014_DetitanizeMobility/  AdHoc_Requests/               ProcessEDDFiles/              junk/                         replays/                      tmpUniqAsMob.out
20160314_FULCRUM/             AddOrder_data/                ProjectDirs/                  kr9850/                       sh2818/                       tmpUniqAsMob.sql
> if($3 !~ S && $i)
>

Hello Nina2910,

Could you please create a script for example script.ksh , paste script there then give it executable permissions eg--> chmod 755 and then run Rudi's/Don's code, it should fly without showing anything else then.

Thanks,
R. Singh

1 Like

@Ravinder...Rudi's code is working without script that's was only for Don's code

Yes:

awk '
BEGIN   {MX = split ("      SEAMPNC", CH, _)                    # create the CH array : CH[1] = " ",...,CH[7] = "S",...,CH[13] = "C" by splitting a string constant at the empty string ("_")
        }                                                       # MX variable could be skipped in this context; provided only for totally "dynamic data"
NR > 1  {SUM = 0                                                # initialize SUM for every input line
         for (i = 7; i<=13; i++) if ($3 !~ CH) SUM += $i     # for the to be checked fields: test if the relevant char (CH[fieldNr]) is found in $3
                                                                # if yes, DON"T sum the field
         $(NF+1) = SUM?"WARNING":"GOOD"                         # add the respective info as the "last plus one" filed.
        }
1                                                               # default action: print
 ' file
1 Like

Pasting a script that contains tabs into a shell that uses tabs to trigger command completion is not going to work. As Ravinder suggested, copy my script into a file and execute the file.

You haven't told us what operating system or shell you're using, but I test it out with the following in a file named tester :

#!/bin/ksh
awk '
BEGIN {	# Initialize skip array: S[fn] = char
	# If field #3 contains the character specified by char, do NOT check
	# the contents of field #fn.
	S[7] = "S"
	S[8] = "E"
	S[9] = "A"
	S[10] = "M"
	S[11] = "P"
	S[12] = "N"
	S[13] = "C"
}
NR < 2 {# Add the requested heading field to the header line...
	print $0, "Newcolumn"
	# and skip to the next input line.
	next
}
{	# For all other input lines, check fields 7-13 inclusive:
	for(i = 7; i <= 13; i++)
		# If the character corresponding to the field is not prsent in
		# field #3 AND the field contains a non-zero value...
		if($3 !~ S && $i) 
			# break out of the loop.
			break
	# Print the input line followed by "WARNING" if a non-zero value was
	# found in a field to be checked; otherwise, print the input line
	# followed by "GOOD".
	print $0, (i <= 13) ? "WARNING" : "GOOD"
}' "${1:-file}"

make it executable with:

chmod +x tester

and then executing it with:

./tester

produces the output I showed you in post #15 in this thread. If you invoke it with an operand that is the pathname of a different file to be processed:

./tester otherfile

it will process the data in otherfile instead of the data in a file named file .

1 Like