Can Anyone help me..to do my task in simple way..

Shell Programming and Scripting
1

Hi...

I am trying extract data using 2 files..

  1. Data file - DATA.TXT containing data to be extracted
  2. Column file - LIST.TXT header column to be extracted

output not coming properly..

if the header field name matches with data file header extract data...this actually I wanted ...

#!/bin/bash

DATAFILE=${1:-DATA.TXT}
COLUMNFILE=${2:-LIST.TXT}

awk -F, -v colsFile="$COLUMNFILE" '
   BEGIN {
     j=1
     while ((getline < colsFile) > 0) {
        col[j++] = $1
     }
     n=j-1;
     close(colsFile)
     for (i=1; i<=n; i++) s[col]=i
   }
   NR==1 {
     for (f=1; f<=NF; f++)
       if ($f in s) c[s[$f]]=f
     next
   }
   { sep=""
     for (f=1; f<=n; f++) {
       printf("%c%s",sep,$c[f])
       sep=FS
     }
     print ""
   }
' "$DATAFILE"

cat DATA.TXT
ID, head1, head2, head3, head4
1, 25.5, 1364.0, 22.5, 13.2
2, 10.1, 215.56, 1.15, 22.2

cat LIST.TXT
ID
head1
head4

 sh extract.sh DATA.TXT LIST.TXT
1,1, 25.5, 1364.0, 22.5, 13.2,1, 25.5, 1364.0, 22.5, 13.2
2,2, 10.1, 215.56, 1.15, 22.2,2, 10.1, 215.56, 1.15, 22.2

and I want to display it as space separated ..

awk '{print $3,$2,$1}' FS="," DATA.TXT
whether here some changes can be done to get desired output

please help..

awk 'NR==1{for(i=1;i<=NF;i++)if($i~/ID/)f[n++]=i}{for(i=0;i<n;i++)printf"%s%s",i?" ":"",$f;print""}' DATA.TXT

I could able to get output from above script but, I have so many columns, how can I search, and display as space or tab separated.

2

Try this:

$ awk   'NR==FNR {Cols=Cols(Cols?"|":"")$1; next}
         FNR==1 {for (i=1;i<=NF;i++) if (match($i,Cols)) Ar[++n]=i}
         {for (i=1;i<=n;i++) printf "%s ", $(Ar); printf "\n"}
        ' FS=", " list data
ID head1 head4 
1 25.5 13.2 
2 10.1 22.2
3

worked fine, for file which I attached, but when I tried with another file...its listing all the columns...

4

Then, please, provide a meaningful sample. What do you expect? If the sample files do not represent your real life data, why should the program work on the latter?

5

Hi...here I am attaching new sample file...pls..see...

6

Works perfectly for me - once I've eliminated or corrected for the errors in your files.
a) field separator in data.txt is just a comma, not comma space as in your sample
b) extra empty line at the end of the list.txt file ruins the creation of the Cols regex

And, you should remove the windows control chars from your file when working in unix environments. Doesn't hurt here, but problems will come soon.

1 Like
7

Thank you so much RudiC, I wasn't know about this.

8

Hi RudiC I just tried your script(#2) with top2 data files(#1)...when I run your script I got following error

 awk   'NR==FNR {Cols=Cols(Cols?"|":"")$1; next}
         FNR==1 {for (i=1;i<=NF;i++) if (match($i,Cols)) Ar[++n]=i}
         {for (i=1;i<=n;i++) printf "%s ", $(Ar); printf "\n"}
        ' FS=", " LIST.TXT DATA.TXT

awk: (FILENAME=LIST.TXT FNR=1) fatal: function `Cols' not defined

I didn't understand the reason behind this..