how to get one particular section (using awk)?

Shell Programming and Scripting
1

Hey,
I have a problem about how to get one section of a file?
I'm new to shell, but by reading some tutorial, I think I can use awk to do this.

my input file:

>ref|ZP_04937576.1|
ECRINAEDPKTFMPSPGKVKHFHAPGGNGVRVDSHLYSGYSVPPNYDSLVGKVITYGAD
DEALARMRNALDELIVDGIKTNTELHKDLVRDAAFCKGGVNIHYLE
>ref|NP_253535.1|
ECRINAEDPKTFMPSPGKVKHFHAPGGNGVRVDSHLYSGYSVPPNYDSLVGKVITYGAD
DEALARMRNALDELIVDGIKTNTELHKDLVRDAAFCKGGVNIHYLE
>ref|YP_002442811.1|
ECRINAEDPKTFMPSPGKVKHFHAPGGNGVRVDSHLYSGYSVPPNYDSLVGKVITYGAD
DEALARMRNALDELIVDGIKTNTELHKDLVRDAAFCKGGVNIHYLE
>pdb|2VQD|A
ECRINAEDPKTFMPSPGKVKHFHAPGGNGVRVDSHLYSGYSVPPNYDSLVGKVITYGA
DEALARMRNALDELIVDGIKTNTELHKDLVRDAAFCKGGVNIHYLE

-----------------
say I have the name "NP_253535.1", how do I return the whole thing?
note each section are begin with symbol ">"

expected output:
--------

>ref|NP_253535.1|
ECRINAEDPKTFMPSPGKVKHFHAPGGNGVRVDSHLYSGYSVPPNYDSLVGKVITYGAD
DEALARMRNALDELIVDGIKTNTELHKDLVRDAAFCKGGVNIHYLE

Thanks a lot!!

2 
awk -vRS=">" '/NP_253535\.1/' file
3

I think sed is a more regular and friendly tool that is usually faster and easy to use. Awk has more strength in delimited many-field records. I am ot much of an awk user, and I don't think either is great about multi-line blocks, but sed is pretty good, as you can pile up lines in the buffer.

sed '
  :loop
  /^>ref|'$input'| *$/!d
  :loop2
  $b
  N
  P
  s/.*\n//
  /^>...|.*| *$/b loop
  b loop2
 ' $in_file

Narrative: sed opens $in_file and examines every line.

  1. Set a branch tag called 'loop'
  2. If we do not find the desired $input in a header line, delete this line (which starts the script over on the next line).
  3. Set a second branch tag 'loop2',
  4. if at EOF, branch (to end of script = print and exit),
  5. get the next line into the buffer after the first and \n,
  6. print the first line,
  7. delete the first line and \n,
  8. if there is a header in the buffer, loop way back to :loop,
  9. branch to loop2 to continue printing the block.

You can see the simple commands in sed allow easy reuse. I call this sort of sed a looper script. Some sed scripts just work on one line, so I call them filter-transform scripts. If you need both at once, you can pipe one sed to the next. Sed is fast and has no size limits.

I see an awk guy bartus11 has given us a one line awk script. I could one line, I guess, but I like to make it easy to read and maintain. The '.' in your $input is the sed metachar for any char, but that does not seem a very big weakness. I used more definition on the header line, so you can have <>| in the text without side effects. Is every '^>[^|]\{3\}|[^\]\{1,99\}| *$' line a header? I may have used too much definition, with /^>ref| so I will revise -- poof! Always >...|?

4

I like that one.
A slight variation on a theme:

nawk -v str='NP_253535.1' '$0~str{print RS $0}' RS='>' myFile
5

Can RS= take a regex?

6

My testing machine's GNU AWK can take some kind of regex in the form of character set "" , for example:

[bart@linux ~]$ awk -vRS="[>|]" 'NR==2' a
ref
[bart@linux ~]$ awk -vRS="[>|]" 'NR==3' a
ZP_04937576.1
[bart@linux ~]$ awk -vRS="[>|]" 'NR==4' a

ECRINAEDPKTFMPSPGKVKHFHAPGGNGVRVDSHLYSGYSVPPNYDSLVGKVITYGAD
DEALARMRNALDELIVDGIKTNTELHKDLVRDAAFCKGGVNIHYLE

After testing some more, I think any regex can go there :slight_smile:

7

Great, you an make the RS proof against a > in the text somewhere! Does it take *, too? Do you lose the RS if you expand it?

8

Thanks for all your replies!!

I ran the one line awk script and turn out to be symbol">" is missing in the output file. How can I can the ">" remained in the very beginning like ">ref|NP_253535.1|"?

I'll also be nice if you can explain to me what is "vRS".

Thanks!!

9

gawk can, but not awk/nawk (at least not on Solaris).

---------- Post updated at 07:48 AM ---------- Previous update was at 07:45 AM ----------

see my previous post.

10

Well, UNIX optarg can be confusing, especially when awk has multi-letter commands in non sh style mixed in. Probably equivalent to "-v RS=...". I am not an optarg fan, as it makes things confusing to save a key stroke or two: training and mistakes are more expensive. For instance, "-abcde" might mean "-a 'bcde'" or "-a -b 'cde'" OR . . . "-a -b -c -d -e" depending on which if any one letter option takes an argument.

Looks like xpg4 awk or nawk (new awk) or the wrong O/S man page:

Man Page for awk (OpenSolaris Section 1) - The UNIX and Linux Forums

	

awk(1)				 User Commands				awk(1)

NAME
       awk - pattern scanning and processing language

SYNOPSIS
       /usr/bin/awk [-f progfile] [-Fc] [' prog '] [parameters]
	    [filename]...

       /usr/xpg4/bin/awk [-FcERE] [-v assignment]... 'program' -f progfile...
	    [argument]...
11

That woks! By the way, I don't have nawk on my mac and also linux red hat, so I use awk instead.

A further question, I have a list of names in one file, how do I get each of those and put together into one file by your "nawk" command?

I tried it by a naive guess, but it didn't return anything in outfile. The "mylist" file is the name list of all the entry name.

for i in mylist ; do awk -v str='$i' '$0~str{print RS $0}' RS='>' fasta-outCT-beta-q1; done > outfile
12 
while read i
do
  awk -v str="$i" '$0~str{print RS $0}' RS='>' fasta-outCT-beta-q1
done < mylist > outfile