subsetting data

UNIX for Beginners Q & A UNIX for Dummies Questions & Answers
1

I have a file where the data is stored in 6 columns, I would like to subset only lines with the fourth column is blank.
Can anybody help me with this?
Thanks
Joseph

2

Provide a sample input and output

3

input:

infile:
1 B 1 chr5 VF
2 C 2 CF
3 D a CA
4 E c chrM AC

outfile:
2 C 2 CF
3 D a CA

4

There would be one limitation here..
that system will not identify whether the netry for the 4th column is missing or not, for it will consider the 5th column as 4th..One approch is that if we know that the 4th column length is 4 and fifth column length is 2 (as shown in ur example), it would be easy..

awk '{ if(length($4)==2) print; }'  filename
5

I am sorry the formatting did not come out right to reflect what I meant.
here is another trial:

input

#col1 #col2 #col3 #col4 #col5 #col6
   a        b       c       d       e      f

output

#col1 #col2 #col3 #col4 #col5 #col6
   a        b       c                e      f

I want data that looks like the second row where col4 is blank

6

Hm, could you post a bigger sample from your input using code tags?
Is the field separator a fixed number of spaces?

7

I hope I got it right this time.
In my real data, the length of both fourth and fifth columns is variable

input

#col1 #col2 #col3 #col4 #col5 #col6
   a        b       c       d       e      f
   a        b       c                e      f

output

#col1 #col2 #col3 #col4 #col5 #col6
   a        b       c                e      f
8

One approch...

egrep $(cut -d" " -f4 filename | grep -v '^$' | xargs | tr " " "|") filename
9

in my real data the field separator is a tab.
I honestly don't know anything about code tags

10

Edit: Just saw your reply.

Use nawk or /usr/xpg4/bin/awk on Solaris:

awk -F'\t' '!$4' filename

or:

perl -F'\t' -ane'print unless $F[3]' filename
11

awk -F'\t' '!$4' filename
got me what I wanted
Thanks everybody

12

I used the following to subset lines in which the forth column is blank:
awk -F'\t' '!$4' filename
How can I change it to subset lines in which the forth column is NOT blank. In other words I would like to split the file into two files based on whether or not the fourth column is blank or NOT blank.
Thanks
Joseph

13 
awk -F'\t' '{print > "out_" ($4 ? "4" : "no_4") }' filename

It's important to mention that the value of 0 evaluates to false!
If the fourth field can have a value of 0, you should force string context:

awk -F'\t' '{print > "out_" ($4_ ? "4" : "no_4") }' filename
14

Hi radoulov
are there any hidden differences between what you suggeted and the following:
awk '{ if(($4)==" ") print; }' filename #to get lines with forth col blank
awk '{ if(($4)!=" ") print; }' filename #to get lines with forth col NOT blank

15

You're missing the field separator(typo?).
You can get what you want using the style above with one command:

awk -F'\t' '{
  if ($4 == "")
    print > FILENAME "_no"
  else
    print > FILENAME "_yes"
    }' filename

No particular semantic differences, but more typing :slight_smile:

16

awk -F'\t' '{
if ($4 == "")
print > FILENAME "_no"
else
print > FILENAME "_yes"
}' filename

in the above code, is FILENAME the output and filename the input?

17

FILENAME is a special awk variable containing the name of the file currently being processed, i.e. "filename". So the output files will be "filename_no" and "filename_yes".

18

I got an error:

$ awk -F'\t' '{
> if ($4 == "")
> print > FILENAME "_no"
> else
> print > FILENAME "_yes"
> }' liver
awk: syntax error at source line 3
context is
print > FILENAME >>> "_no" <<<
awk: illegal statement at source line 3
awk: syntax error at source line 5

19

Try this slight adjustment:

awk -F'\t' '
  {
    nofile=FILENAME "_no"
    yesfile=FILENAME "_yes"
    if ($4 == "")
      print > nofile
    else
      print > yesfile
  }
' filename
20

Yep, sorry.
This should work with your awk implementation:

awk -F'\t' '{
  if ($4 == "")
    print > (FILENAME "_no")
  else
    print > (FILENAME "_yes")
    }' file