Gnu parallel and awk syntax quotes

UNIX for Beginners Q & A UNIX for Dummies Questions & Answers
1

I have many files that has the following structure.

CHR         SNP         BP   A1       TEST    NMISS         OR       SE      L95      U95         STAT            P 
   1   rs2980319     766985    A        ADD     4948      1.068  0.08356   0.9065    1.258       0.7853       0.4323
   1   rs2980319     766985    A     VAR1     4948      1.109  0.02484    1.057    1.165        4.177    2.952e-05
   1   rs2980319     766985    A        VAR2     4948  1.567e+27    2.441 1.31e+25 1.875e+29        25.65   4.009e-145
   1   rs2980319     766985    A        VAR3     4948  3.031e+12    2.221 3.902e+10 2.355e+14        12.94     2.63e-38

I want to output certain columns and lines which only contain "ADD" in the 5th column.
I am trying to use GNU parallel and am having no luck getting output.

>parallel ""awk 'NR==1{print $2, $1, $3, $4, $7, $9, $10, $12; next} $5=="ADD"{print $2, $1, $3, $4, $7, $9, $10, $12 | "sort -gk8"}' OFS="\t" {1}_fileA.assoc > {1}_fileA.assoc.ADD

The syntax seems too dense for me.
Any help is appreciated.

2

It looks extremely bizzare to me, too. I have no idea why they'd "parallel" a single file, or even how...

3

I just gave an exmple of a single file .
The {1} can be substituted by any number of variables that act as prefixes for FileA

parallel - build and execute shell command lines from standard input in parallel

I gind GNU parallel to be very useful especially without access to a cluster for processing big data. However, yet to learn to make awk work with it. :frowning:

Thanks for looking.

4

the redirection would be for the entire run of 'parallel' (which i have no experience with). you'd be better off creating a small shell script to run, or possibly something like

parallel sh -c 'awk '\''program''\' OFS="\t" $1_fileA.assoc > $1_fileA.assoc.ADD' _ {1}

maybe... :confused:

5

Try this:

... parallel awk \'NR == 1 \{ \
  print \$2, \$1, \$3, \$4, \$7, \$9, \$10, \$12 \
  next \
  \} \
  \$5 == \"ADD\" \{ \
    print \$2, \$1, \$3, \$4, \$7, \$9, \$10, \$12 \| \"sort -gk8\" \
    \}' OFS=\'\\t\' {1}_fileA.assoc \> {1}_fileA.assoc.ADD

Or just put your awk code in a script file and make your life easier :slight_smile: