AWK Looping. How can I get expected result?

Shell Programming and Scripting
1

Can Anyone help with looping...

awk 'FNR==1{i++} 
{for(k=1; k<=NF; k++) A[i,FNR,k]=$k}                                # 3 Dimension Array 
END{
for(i=1;i<=217;i++)                                                             # For loop 2nd File 1st and 2nd column
x=0;y=0                                                                           # Intialize x and y value 0
{
for(j=1;j<=2440;j++)                                                   # For loop for 1st file 1st and 2nd column
if(A[1,j,2]>=A[2,i,2]-0.1 && A[1,j,2]<=A[2,i,2]+0.1)            # Comparison starts here
{
if(A[1,j,1]>=A[2,i,1])x=x+1;                           
if(A[1,j,1]<=A[2,i,1])y=y+1;
}                                  
}                                                                             # Internal loop ends here
if(x>=1 && y>=1)
print i,j,A[2,i,1],A[2,i,2]}' OFS="\t" file1.pl file2.pl                  # 1st for loop ends here and printing


Result : coming
218 2441 75.5 10.07

Expected result :
1 2441 75.5   10.05
2 2441 75.02  10.15
3 2441 74.52  10.23
2

It helps allot if you also post the input data.

3

Dear Jotne Kindly look at attachment

4

And how do you like to calculate this two file to get your result?

5

problem here is I could not able to trace where internal loop is ending and where outer loop is ending .

here it was suppose to do like this repeat following statements 2440 times, after that come out of loop, print if statement is true else increment outer loop(i).. again repeat following statements 2440 times...

if(A[1,j,2]>=A[2,i,2]-0.1 && A[1,j,2]<=A[2,i,2]+0.1)            # Comparison starts here
{
if(A[1,j,1]>=A[2,i,1])x=x+1;                           
if(A[1,j,1]<=A[2,i,1])y=y+1;
}  

instead of printing result after every increment of i its printing result at last value of i (thats i=218)
except if(x>=1 && y>=1) is false

[/CODE]I hope you will understand

This is that logic, I had posted earlier unfortunately nobody helped, Thank God at least you asked me..what you want to do

1 . First read column 1 and column 2 of file1.plo store it in array(reference)

2 . Now read column 1 and column 2 of file2.plo store it in another array,to be compared with reference

3 . for loop : - for(i=1;i<=file2.plo_length;i++)

4 . x=0; y=0 this will get clear as and when i will increment

5 . nested loop :- for(j=1;j<=file1.plo_lenght;j++)

6 . if(col1_f1[j]>=col1_f2-0.1 [i]&& col1_f1[j]<=col1_f2+0.1[i])

if above if statement is true then,go to following statement else check condition with j++

7 . if(col2_f2[i]>=col2_f1[j]) then x=x+1

8 . if(col2_f2[i]<=col2_f1[j]) then y=y+1

repeat 5,6,7 and 8 till j reaches file1.txt_length(NR),Once j reaches NR following

9 . if(x>=1 && y>=1) then print column1_f2 [i]column2_f2

[i]repeat statements after number 4 to 9 till i reaches file2.plo_length(NR)

6

What immediately jumps to my eyes (after rearranging / indenting your code for better readability) is that your first for loop sets x=0 217 times, then y=0 once, then continues to the rest:

END     {for(i=1;i<=217;i++)                                                    # For loop 2nd File 1st and 2nd column
         x=0;y=0                                                                # Intialize x and y value 0
                 {for(j=1;j<=2440;j++)                                          # For loop for 1st file 1st and 2nd column
                    if(A[1,j,2]>=A[2,i,2]-0.1 && A[1,j,2]<=A[2,i,2]+0.1)        # Comparison starts here
                        {if(A[1,j,1]>=A[2,i,1])x=x+1;                           
                         if(A[1,j,1]<=A[2,i,1])y=y+1;
                        }                            
                 }                                                              # Internal loop ends here
         if(x>=1 && y>=1) print i, j, A[2,i,1], A[2,i,2]
        }

You want to move that second opening brace in front of the x assignment and retry...

7

Do you mean like this

END {for(i=1;i<=217;i++)x=0{for(j=1;j<=2440;j++)

I want to clear x and y value after the end of internal j loop

Sir I still not getting result as I expected

8
9

Sir I think I am doing something wrong here, as I do program most of time in assembly language, like that I tried to clear x and y after the end of internal loop...but its resulting something else, I really have no idea that whats wrong here...same logic is working in FORTRAN with do loop and label for repetition.

---------- Post updated at 04:45 AM ---------- Previous update was at 04:26 AM ----------

I didn't get it sir..if you have time...please help me in modifying it.

---------- Post updated at 05:06 AM ---------- Previous update was at 04:45 AM ----------

You might have remember I had posted this before also..

do 20 i=1,217      
        x=0
        y=0
        do 21 j=1,2440
        if(column2_1(j).ge.column1_1(i)-0.1.and.column2_1(j).le.column1_1(i)+0.1)then
        if(column1_1(i).ge.column1_2(j))then
        x=x+1
        endif        
        if(column1_1(i).le.column1_2(j))then
        y=y+1
        endif
        endif
 21     continue      
        if(x.ge.1.and.y.ge.1)then
        write(13,'(a70)')dat(i)
        endif       
 20     continue 
    stop
    end
10 
END     {for(i=1;i<=217;i++)                                                    # For loop 2nd File 1st and 2nd column
         x=0;y=0                                                                # Intialize x and y value 0
         ^-------{for(j=1;j<=2440;j++)                                          # For loop for 1st file 1st and 2nd column
                    if(A[1,j,2]>=A[2,i,2]-0.1 && A[1,j,2]<=A[2,i,2]+0.1)        # Comparison starts here
                        {if(A[1,j,1]>=A[2,i,1])x=x+1;                           
                         if(A[1,j,1]<=A[2,i,1])y=y+1;
                        }                            
                 }                                                              # Internal loop ends here
         if(x>=1 && y>=1) print i, j, A[2,i,1], A[2,i,2]
        }

resulting in

END     {for (i=1;i<=217;i++)                                       # For loop 2nd File 1st and 2nd column
           {x=0                                                     # Intialize x and y value 0
            y=0
            for (j=1;j<=2440;j++)                                   # For loop for 1st file 1st and 2nd column
              if(A[1,j,2]>=A[2,i,2]-0.1 && A[1,j,2]<=A[2,i,2]+0.1)  # Comparison starts here
                 {if(A[1,j,1]>=A[2,i,1])x=x+1;       
                  if(A[1,j,1]<=A[2,i,1])y=y+1;
                 }                                   
           }                                                        # Internal loop ends here
         if(x>=1 && y>=1) print i, j, A[2,i,1], A[2,i,2]
        }

As you reset x and y in every iteration, your final if will consider the last iteration's values only. So I guess you should reverse those lines as well:

            if(x>=1 && y>=1) print i, j, A[2,i,1], A[2,i,2]
           }                                                        # Internal loop ends here
        }
1 Like
11

Thanks a lot, its printing, now I need to verify with FORTRAN result.
I wish it will be fine...

and one more thing I wanted to ask you, for inner loop I want to assign NR of file 1 and for outer loop NR of fie2,

if I assign simply for(i=1;i<=NR;i++)

which NR it will take ? by default

c=`awk 'END{print NR}' file1`
d=`awk 'END{print NR}' file2`
for(i=1;i<='$c';i++)

till date I am assigning counter value from bash variable like above

Thanks a lot RudyC, for using your valuable time to help me....I just processed 413513 lines, 5855826 characters fille

12

For your file[12] line count, in the first lines of the program try sth. like

FNR==1 {i++}
       {LC=NR}

The respective LC element will hold the last NR for each file.

1 Like
13

Thank you so much sir...for solving my big problem,

Sir I am trying one more thing with same file to display unmatched records

my problem is like this.. for instance say file1 has 1000 records, file 2 has 750 records, file1's 750 records are matching with file 2

I want to display remaining 250 records from file 1

I could able to display matching records, using following script

awk 'FNR==1{i++}{for(k=1;k<=NF;k++) A[i,FNR,k]=$k}
END{
for(i=1;i<='$d';i++)
{
for(j=1;j<='$c';j++)
if(A[2,i,2]==A[1,j,1] && A[2,i,1]==A[1,j,2])
print A[2,i,2], A[2,i,1], "<==F2 equal F1==>", A[1,j,1],A[1,j,2]
}}' OFS="\t" file1 file2
14

Not sure I understand. If the first 750 records in both files match and you want to display records 751 - 1000 of file1, simply reverse the files' order in awk's file list and insert sth. like FNR>LC[1] into your program.
If the mismatches occur anywhere in the files, use the unix diff command.

1 Like
15 
diff  <(cut -f2- < file2) <(cut -f1- < file1)

this is printing something....how can I assign FS here.. file 1 is tab separated file 2 is space separated

16

man cut ?
Seems you are looking for matching fields instead of matching entire records (= rows, lines). cut -f2- will render field 2 up to end of line, -f1- the entire line. Drop the - - sign if you go for single fields only. And, use -d" " for the space delimiter.

17 
diff file1 file2

I tried above one after sorting of file

18

there is something displaced

just re-typed the code with appropraite braces

~# awk 'FNR==1{i++}
{ for(k=1; k<=NF; k++){ A[i,FNR,k]=$k } }
END { for (i=1;i<=217;i++) { x=0;y=0; for(j=1; j<=2440;j++) { if(A[1,j,2]>=A[2,i,2]-0.1 && A[1,j,2]<=A[2,i,2]+0.1) { if(A[1,j,1]>=A[2,i,1])x=x+1;  if(A[1,j,1]<=A[2,i,1])y=y+1; }  }
if(x>=1 && y>=1) { print i,j,A[2,i,1],A[2,i,2]} } }' OFS="\t"  file1.pl file2.pl
awk 'FNR==1{i++}
{ for(k=1; k<=NF; k++){ A[i,FNR,k]=$k } }
END { for (i=1;i<=217;i++) { x=0;y=0; for(j=1; j<=2440;j++) { if(A[1,j,2]>=A[2,i,2]-0.1 && A[1,j,2]<=A[2,i,2]+0.1) { if(A[1,j,1]>=A[2,i,1])x=x+1;  if(A[1,j,1]<=A[2,i,1])y=y+1; }  }
if(x>=1 && y>=1) { print i,j,A[2,i,1],A[2,i,2]} } }' OFS="\t"  file1.pl file2.pl
1       2441    75.5    10.05
2       2441    75.02   10.15
3       2441    74.52   10.23
4       2441    74.02   10.33
5       2441    73.45   10.43
6       2441    73.15   10.48
8       2441    73.52   10.23
9       2441    73.98   10.13
10      2441    74.5    10.08
11      2441    74.98   10.05
12      2441    75.5    10
13      2441    75.5    10.05
14      2441    75      10.15
15      2441    74.5    10.23
16      2441    74      10.32
17      2441    73.5    10.42
18      2441    73      10.5
19      2441    75.5    10.07
20      2441    75      10.15
21      2441    74.5    10.23
22      2441    74      10.33
23      2441    73.5    10.42
24      2441    73      10.52
25      2441    72.7    10.57
26      2441    75.5    10.07
27      2441    75      10.15
28      2441    74.5    10.23
29      2441    74.02   10.32
30      2441    73.53   10.43
31      2441    73.02   10.5
32      2441    72.75   10.55
33      2441    75.83   9.98
34      2441    75.5    10.05
35      2441    75      10.15
36      2441    74.5    10.23
37      2441    74      10.32
38      2441    73.5    10.42
39      2441    73      10.52
40      2441    72.73   10.57
41      2441    75.83   9.97
42      2441    75.5    10.05
43      2441    75      10.15
44      2441    74.5    10.22
45      2441    74      10.32
46      2441    73.5    10.42
47      2441    73      10.5
48      2441    72.67   10.57
49      2441    72.75   10.55
50      2441    73.08   10.5
51      2441    73.5    10.4
52      2441    74      10.33
53      2441    74.5    10.23
54      2441    75.42   10.07
55      2441    75.83   9.97
56      2441    75.5    10
57      2441    74.92   10
58      2441    74.42   10.05
59      2441    74      10.05
60      2441    73.5    10.23
61      2441    73      10.42
62      2441    72.77   10.53
63      2441    75.83   9.8
64      2441    72.77   10.52
65      2441    73      10.42
66      2441    73.5    10.23
67      2441    74      10.05
68      2441    74.5    10.03
69      2441    75.25   10
70      2441    75.5    9.97
71      2441    76      9.95
72      2441    72.73   10.53
73      2441    73      10.48
74      2441    73.5    10.4
75      2441    74      10.32
76      2441    74.5    10.22
77      2441    75      10.12
78      2441    75.5    10.05
79      2441    75.67   10.02
80      2441    75.83   9.98
81      2441    75.83   10
82      2441    75.5    10.07
83      2441    75      10.17
84      2441    74.47   10.23
85      2441    74      10.33
86      2441    73.5    10.42
87      2441    72.97   10.5
88      2441    72.67   10.57
89      2441    75.83   10
90      2441    75.4    10.13
91      2441    74.98   10.33
92      2441    74.47   10.53
93      2441    73.98   10.72
94      2441    73.47   10.12
95      2441    73      10.37
96      2441    72.72   10.52
97      2441    75.83   9.98
98      2441    75.5    10.05
99      2441    75      10.13
100     2441    74.5    10.23
101     2441    74      10.32
102     2441    73.5    10.42
103     2441    73      10.5
104     2441    72.75   10.55
111     2441    89.53   19
112     2441    88.83   20
113     2441    88.47   20.5
114     2441    88.6    20.5
122     2441    89.6    19
123     2441    88.82   20
124     2441    88.47   20.5
132     2441    89.53   19
133     2441    88.8    20
134     2441    88.82   20
142     2441    89.78   19
143     2441    88.82   20
151     2441    89.5    19
152     2441    88.75   20
160     2441    89.53   19
161     2441    88.83   20
168     2441    89.6    19
169     2441    88.88   20
170     2441    75.83   9.98
171     2441    75.5    10.05
172     2441    75      10.13
173     2441    74.5    10.23
174     2441    74      10.32
175     2441    73.5    10.42
176     2441    73      10.48
177     2441    72.75   10.55
185     2441    89.17   19
186     2441    88.82   20
187     2441    75.82   9.92
188     2441    75.48   9.93
189     2441    75.02   9.97
190     2441    74.45   10
191     2441    74.02   10.03
192     2441    73.62   9.93
193     2441    72.7    10.57
194     2441    73      10.5
195     2441    73.5    10.42
196     2441    74      10.32
197     2441    74.5    10.23
198     2441    75      10.15
199     2441    75.5    10.05
200     2441    75.83   9.98
201     2441    72.67   10.57
202     2441    73.5    10.5
203     2441    73.5    10.42
204     2441    74      10.33
205     2441    74.5    10.23
206     2441    75      10.13
207     2441    75.5    10.05
208     2441    75.83   9.98
209     2441    75.83   9.98
210     2441    75.5    10.05
211     2441    75      10.13
212     2441    74.5    10.23
213     2441    74      10.33
214     2441    73.5    10.42
215     2441    73      10.52
216     2441    72.67   10.55
217     2441    75.83   10

---------- Post updated at 06:32 PM ---------- Previous update was at 06:31 PM ----------

sorry gues, wrong window selected...my apologies..

---------- Post updated at 06:33 PM ---------- Previous update was at 06:32 PM ----------

Just replaced and re-typed to appropriate places for the braces...

~# awk 'FNR==1{i++}
{ for(k=1; k<=NF; k++){ A[i,FNR,k]=$k } }
END { for (i=1;i<=217;i++) { x=0;y=0; for(j=1; j<=2440;j++) { if(A[1,j,2]>=A[2,i,2]-0.1 && A[1,j,2]<=A[2,i,2]+0.1) { if(A[1,j,1]>=A[2,i,1])x=x+1;  if(A[1,j,1]<=A[2,i,1])y=y+1; }  }
if(x>=1 && y>=1) { print i,j,A[2,i,1],A[2,i,2]} } }' OFS="\t"  file1.pl file2.pl
awk 'FNR==1{i++}
{ for(k=1; k<=NF; k++){ A[i,FNR,k]=$k } }
END { for (i=1;i<=217;i++) { x=0;y=0; for(j=1; j<=2440;j++) { if(A[1,j,2]>=A[2,i,2]-0.1 && A[1,j,2]<=A[2,i,2]+0.1) { if(A[1,j,1]>=A[2,i,1])x=x+1;  if(A[1,j,1]<=A[2,i,1])y=y+1; }  }
if(x>=1 && y>=1) { print i,j,A[2,i,1],A[2,i,2]} } }' OFS="\t"  file1.pl file2.pl
1       2441    75.5    10.05
2       2441    75.02   10.15
3       2441    74.52   10.23
4       2441    74.02   10.33
5       2441    73.45   10.43
6       2441    73.15   10.48
8       2441    73.52   10.23
9       2441    73.98   10.13
10      2441    74.5    10.08
11      2441    74.98   10.05
12      2441    75.5    10
13      2441    75.5    10.05
14      2441    75      10.15
15      2441    74.5    10.23
16      2441    74      10.32
17      2441    73.5    10.42
18      2441    73      10.5
19      2441    75.5    10.07
20      2441    75      10.15
21      2441    74.5    10.23
22      2441    74      10.33
23      2441    73.5    10.42
24      2441    73      10.52
25      2441    72.7    10.57
26      2441    75.5    10.07
27      2441    75      10.15
28      2441    74.5    10.23
29      2441    74.02   10.32
30      2441    73.53   10.43
31      2441    73.02   10.5
32      2441    72.75   10.55
33      2441    75.83   9.98
34      2441    75.5    10.05
35      2441    75      10.15
36      2441    74.5    10.23
37      2441    74      10.32
38      2441    73.5    10.42
39      2441    73      10.52
40      2441    72.73   10.57
41      2441    75.83   9.97
42      2441    75.5    10.05
43      2441    75      10.15
44      2441    74.5    10.22
45      2441    74      10.32
46      2441    73.5    10.42
47      2441    73      10.5
48      2441    72.67   10.57
49      2441    72.75   10.55
50      2441    73.08   10.5
51      2441    73.5    10.4
52      2441    74      10.33
53      2441    74.5    10.23
54      2441    75.42   10.07
55      2441    75.83   9.97
56      2441    75.5    10
57      2441    74.92   10
58      2441    74.42   10.05
59      2441    74      10.05
60      2441    73.5    10.23
61      2441    73      10.42
62      2441    72.77   10.53
63      2441    75.83   9.8
64      2441    72.77   10.52
65      2441    73      10.42
66      2441    73.5    10.23
67      2441    74      10.05
68      2441    74.5    10.03
69      2441    75.25   10
70      2441    75.5    9.97
71      2441    76      9.95
72      2441    72.73   10.53
73      2441    73      10.48
74      2441    73.5    10.4
75      2441    74      10.32
76      2441    74.5    10.22
77      2441    75      10.12
78      2441    75.5    10.05
79      2441    75.67   10.02
80      2441    75.83   9.98
81      2441    75.83   10
82      2441    75.5    10.07
83      2441    75      10.17
84      2441    74.47   10.23
85      2441    74      10.33
86      2441    73.5    10.42
87      2441    72.97   10.5
88      2441    72.67   10.57
89      2441    75.83   10
90      2441    75.4    10.13
91      2441    74.98   10.33
92      2441    74.47   10.53
93      2441    73.98   10.72
94      2441    73.47   10.12
95      2441    73      10.37
96      2441    72.72   10.52
97      2441    75.83   9.98
98      2441    75.5    10.05
99      2441    75      10.13
100     2441    74.5    10.23
101     2441    74      10.32
102     2441    73.5    10.42
103     2441    73      10.5
104     2441    72.75   10.55
111     2441    89.53   19
112     2441    88.83   20
113     2441    88.47   20.5
114     2441    88.6    20.5
122     2441    89.6    19
123     2441    88.82   20
124     2441    88.47   20.5
132     2441    89.53   19
133     2441    88.8    20
134     2441    88.82   20
142     2441    89.78   19
143     2441    88.82   20
151     2441    89.5    19
152     2441    88.75   20
160     2441    89.53   19
161     2441    88.83   20
168     2441    89.6    19
169     2441    88.88   20
170     2441    75.83   9.98
171     2441    75.5    10.05
172     2441    75      10.13
173     2441    74.5    10.23
174     2441    74      10.32
175     2441    73.5    10.42
176     2441    73      10.48
177     2441    72.75   10.55
185     2441    89.17   19
186     2441    88.82   20
187     2441    75.82   9.92
188     2441    75.48   9.93
189     2441    75.02   9.97
190     2441    74.45   10
191     2441    74.02   10.03
192     2441    73.62   9.93
193     2441    72.7    10.57
194     2441    73      10.5
195     2441    73.5    10.42
196     2441    74      10.32
197     2441    74.5    10.23
198     2441    75      10.15
199     2441    75.5    10.05
200     2441    75.83   9.98
201     2441    72.67   10.57
202     2441    73.5    10.5
203     2441    73.5    10.42
204     2441    74      10.33
205     2441    74.5    10.23
206     2441    75      10.13
207     2441    75.5    10.05
208     2441    75.83   9.98
209     2441    75.83   9.98
210     2441    75.5    10.05
211     2441    75      10.13
212     2441    74.5    10.23
213     2441    74      10.33
214     2441    73.5    10.42
215     2441    73      10.52
216     2441    72.67   10.55
217     2441    75.83   10
19

Anyways thank you....RudyC has given me solution for this problem