Extract specific content from data and rename its header problem asking

Shell Programming and Scripting
1

Input file 1:

>pattern_5
GAATTCGTTCATGTAGGTTGASDASFGDSGRTYRYGHDGSDFGSDGGDSGSDGSDFGSDF
ATTTAATTATGATTCATACGTCATATGTTATTATTCAATCGTATAAAATTATGTGACCTT
SDFSDGSDFKSDAFLKJASLFJASKLFSJAKJFHASJKFHASJKFHASJKFHSJAKFHAW
>pattern_1
AAGTCTTAAGATATCACCGTCGATTAGGTTTATACAGCTTTTGTGTTATTTAAATTTGAC
ASDASRFSARFASFEDEGSDGHSDHWDYTQATWQRQOPTPEOTPWRIYRHIRGOPEIWRA
.
.

Input file 2:

pattern_5    5   15
pattern_5    18  25 
pattern_1    10  19
pattern_1    22  27
.
.

Desired output:

>pattern_5_0.01
TCGTTCATGTA
>pattern_5_0.02
TTGASDAS
>pattern_1_0.01
GATATCACCG
>pattern_1_0.02
GATTAG
.
.

I got a long list of input file 1 and input file 2. Input file 1 is the raw data while input file 2 is the range of input file 1 data that I'm interested to extract and generate the output result file. The column 2 and column 3 of input file 2 is the position that I interested to extract from the data of input file 1. The output file I will rename with the header like "pattern_*_0.0*"
It seems like awk or perl scripts able to archive these goal.
Thanks a lot for any advice.

2

Can you explain how you get the below line in red from your desired output..

>pattern_5_0.01
TCGTTCATGTA
>pattern_5_0.02
TTGASDAS
>pattern_1_0.01
GATATCACCG
>pattern_1_0.02
GATTAG
3

Hi malcomex999,
I just edited my question and explain the usage of column 2 and column 3 inside input file 2.
Thanks for your advice.

---------- Post updated 03-24-10 at 01:53 AM ---------- Previous update was 03-23-10 at 04:32 AM ----------

Hi malcomex999,
do you got any idea to archive the desired goal?
Thanks a lot.

4

Hi, patrick87:

awk 'FNR==NR {if (/^>/) p=substr($0,2); else a[p]=a[p] $0; next} {printf(">%s_0.%02u\n%s\n", $1, ++i[$1], substr(a[$1], $2, $3-$2+1))}' f1 f2

While processing the first file (FNR==NR), if a line begins with ">", grab everything that follows it and store it in p, the pattern name. If a line does not begin with a ">", then it is data for the current pattern, p; append the line to a[p], that pattern's entry in array a. Repeat until done with the first file.

For the second file, we use the pattern name in the first field and the index values in the second and third fields to extract the required substring from a[$1], while incrementing a counter for each pattern name seen, in the i array, i[$1].

Cheers,
Alister

5

Thanks alister,
I'm trying apply your awk code to my case now :slight_smile:
Besides that, thanks a lot for your further explanation of your awk code too.
I very appreciate and thanks for your help and advice.
Thanks again ^^

---------- Post updated at 04:19 AM ---------- Previous update was at 03:43 AM ----------

Hi Alister,
Your awk code worked perfectly in my case. Thanks a lot.
Can I ask you if my input file 2 change like this:

pattern_5    15  5
pattern_5    18  25 
pattern_1    10  19
pattern_1    22  27
.
.

How I can edit the awk code that you suggested to give the same output result as above?
Is it I need to add the "if" condition in the awk code for this problem?
Thanks again for your advice.

6

You can modify alister code like this...

awk 'FNR==NR {if (/^>/) p=substr($0,2); 
else a[p]=a[p] $0; next} 
{printf(">%s_0.%02u\n%s\n", $1, ++i[$1], substr(a[$1], $2, ($2>=$3?$3:$3-$2+1)))}' f1 f2
7

Hi, malcomeex999:

That tweak is incorrect, if I understand the modification to f2 correctly. If the second field is greater than the third, then it instead of being treated as the beginning index of the substring, it should be considered the end index (and the interpretation of the third field should be complementarily swapped). The correct solution requires that the second argument to substr() be modified as well, since in the case of $2 > $3, it should be $3 not $2.

By the way, malcomeex999 and rdcwayx, thank you very much for your bit awards. It's appreciated :slight_smile:

Hi, patrick87:

One solution to handle both cases (even if they appear within the same file2):

awk 'FNR==NR {if (/^>/) p=substr($0,2); else a[p]=a[p] $0; next}
     {if ($2>$3) {t=$2; $2=$3; $3=t}; printf(">%s_0.%02u\n%s\n", $1, ++i[$1], substr(a[$1], $2, $3-$2+1))}' f1 f2

It works identically to my earlier solution except that it tests the second and third fields in f2. If the first index is greater than the second, their values are swapped before the substr() call.

Regards,
Alister

8

Hi Alister,
i guess we both understood the changes to f2 differently.
From what i understood,if it is like below

pattern_5    15  5

it should fetch 5 characters starting from 15th character. But in your way, you are swapping it so it will fetch 15 characters starting 5th character. But as long as the OP didn't show the desired output, we have to wait and see his reply.

9

I took that to mean that it should give an identical result if the column indexes were swapped. Of course, I could be wrong.

Regards,
Alister

10

Hi alister,
Your awk code work perfectly for both of my cases :slight_smile:

awk 'FNR==NR {if (/^>/) p=substr($0,2); else a[p]=a[p] $0; next}
     {if ($2>$3) {t=$2; $2=$3; $3=t}; printf(">%s_0.%02u\n%s\n", $1, ++i[$1], substr(a[$1], $2, $3-$2+1))}' f1 f2

Really thanks a lot for your sharing. It is fantastic and work perfectly.
Thanks again.

---------- Post updated at 01:47 AM ---------- Previous update was at 01:39 AM ----------

Hi malcomex999,
Thanks a lot for your help too.
Sorry if my second problem case leads to your misunderstanding.
Actually for both problem case, I would like to obtain the same output result:

>pattern_5_0.01
TCGTTCATGTA
>pattern_5_0.02
TTGASDAS
>pattern_1_0.01
GATATCACCG
>pattern_1_0.02
GATTAG

No matter how I alter the range of column 2 and column 3 from the input file 2.
Anywhere I appreciate your awk sharing too :slight_smile:
Thanks again, malcomex999 ^^

11

You're very welcome, patrick87. :slight_smile: