# Execution problem with repeat the same program with two set of same data

**URL:** <https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992>\
**Category:** Shell Programming and Scripting\
**Created:** [October 9, 2009, 6:40am UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992 "2009-10-09T06:40:13Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![patrick87](https://community.unix.com/letter_avatar/patrick87/32/6_22734026db12803ebb3e059622ac3534.png) [@patrick87](https://community.unix.com/u/patrick87)\
**Post date:** [October 9, 2009, 6:40am UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/1 "2009-10-09T06:40:13Z")

</div>

I got a program named as "fastq\_to\_fasta".  
I got a long list of file all named as AB1 and AB2.

My input file is :

```nohighlight
071022_L1_AB1.fq
012121_L1_AB1.fq
021213_L1_AB1.fq
012153_L1_AB1.fq
071022_L1_AB2.fq
012121_L1_AB2.fq
021213_L1_AB2.fq
012153_L1_AB2.fq
.
.
.
.
.
.

```

My desired output is write a work script or a text file look like this:

```nohighlight
fastq_to_fasta 071022_L1_AB1.fq 071022_L1_AB2.fq
fastq_to_fasta 012121_L1_AB1.fq 012121_L1_AB2.fq
fastq_to_fasta 021213_L1_AB1.fq 021213_L1_AB2.fq
fastq_to_fasta 012153_L1_AB1.fq 012153_L1_AB2.fq
.
.
.
.
.
.

```

Actually all the long list of file need to do the same program like:

```nohighlight
fastq_to_fasta <XXXXAB1.fq> <XXXXAB2.fq>
Same XXXXAB1 must work with same XXXXAB2

```

Does anybody know by using awk command or any other programming language can easy for me to get my desired output result.

Thanks a lot.

---

<div class="post-metadata">

**Author:** ![Franklin52](https://community.unix.com/user_avatar/community.unix.com/franklin52/32/500_2.png) [@Franklin52](https://community.unix.com/u/Franklin52)\
**Post date:** [October 9, 2009, 7:26am UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/2 "2009-10-09T07:26:48Z")

</div>

Something like this?

```nohighlight
awk -F_ 'a[$1]{print "fastq_to_fasta "a[$1]" "$0; next} {a[$1]=$0}' file

```

---

<div class="post-metadata">

**Author:** ![patrick87](https://community.unix.com/letter_avatar/patrick87/32/6_22734026db12803ebb3e059622ac3534.png) [@patrick87](https://community.unix.com/u/patrick87)\
**Post date:** [October 9, 2009, 7:42am UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/3 "2009-10-09T07:42:44Z")

</div>

Hi Franklin52  
I try your code just now.  
Unfortunately, It can't work ☹  
Do you have any other idea?  
Actually I just want to produce a long list of below (based on same XXX\_AB1 with XXX\_AB2):

```nohighlight
fastq_to_fasta 071022_L1_AB1.fq 071022_L1_AB2.fq
fastq_to_fasta 012121_L1_AB1.fq 012121_L1_AB2.fq
fastq_to_fasta 021213_L1_AB1.fq 021213_L1_AB2.fq
fastq_to_fasta 012153_L1_AB1.fq 012153_L1_AB2.fq
.
.
.

```

.  
.  
.

Thanks a lot for your help.

---

<div class="post-metadata">

**Author:** ![Franklin52](https://community.unix.com/user_avatar/community.unix.com/franklin52/32/500_2.png) [@Franklin52](https://community.unix.com/u/Franklin52)\
**Post date:** [October 9, 2009, 8:07am UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/4 "2009-10-09T08:07:08Z")

</div>

> [@patrick87](#):
>
> Hi Franklin52  
> I try your code just now.  
> Unfortunately, It can't work ☹  
> Do you have any other idea?

Your comment doesn't make any sense, I can't guess what's wrong.

This is my output with your input file:

```plaintext
$ cat file
071022_L1_AB1.fq
012121_L1_AB1.fq
021213_L1_AB1.fq
012153_L1_AB1.fq
071022_L1_AB2.fq
012121_L1_AB2.fq
021213_L1_AB2.fq
012153_L1_AB2.fq
$ awk -F_ 'a[$1]{print "fastq_to_fasta "a[$1]" "$0; next} {a[$1]=$0}' file
fastq_to_fasta 071022_L1_AB1.fq 071022_L1_AB2.fq
fastq_to_fasta 012121_L1_AB1.fq 012121_L1_AB2.fq
fastq_to_fasta 021213_L1_AB1.fq 021213_L1_AB2.fq
fastq_to_fasta 012153_L1_AB1.fq 012153_L1_AB2.fq
$

```

---

<div class="post-metadata">

**Author:** ![patrick87](https://community.unix.com/letter_avatar/patrick87/32/6_22734026db12803ebb3e059622ac3534.png) [@patrick87](https://community.unix.com/u/patrick87)\
**Post date:** [October 9, 2009, 9:33pm UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/5 "2009-10-09T21:33:20Z")

</div>

Thanks a lot Frankin52 🙂  
I think I understand what you mean now.  
Maybe I type some error and let you misunderstanding it.  
Actually all the \*\_AB1.fq and \*\_AB2.fq all are long list of "file" not the list of "contents" in a file.  
But it should be not a problem.  
I just do

```plaintext
 
ls *.fq > file
cat file
awk -F_ 'a[$1]{print "fastq_to_fasta "a[$1]" "$0; next} {a[$1]=$0}' file
 

I believe it should be work nice.
Thanks a lot and again, Frankin52 :)
$ cat file
071022_L1_AB1.fq
012121_L1_AB1.fq
021213_L1_AB1.fq
012153_L1_AB1.fq
071022_L1_AB2.fq
012121_L1_AB2.fq
021213_L1_AB2.fq
012153_L1_AB2.fq
$ awk -F_ 'a[$1]{print "fastq_to_fasta "a[$1]" "$0; next} {a[$1]=$0}' file
fastq_to_fasta 071022_L1_AB1.fq 071022_L1_AB2.fq
fastq_to_fasta 012121_L1_AB1.fq 012121_L1_AB2.fq
fastq_to_fasta 021213_L1_AB1.fq 021213_L1_AB2.fq
fastq_to_fasta 012153_L1_AB1.fq 012153_L1_AB2.fq
$

```

[/quote]

---------- Post updated at 08:33 PM ---------- Previous update was at 07:35 PM ----------

Thanks again Franklin52.  
Your suggestion code look cool 🙂  
I will go and find out the reason why you write this code.  
Thus able to edit it next time to solve another similar problem.  
In between, can I roughly know the explanation of your code written?  
Sorry if bring you any inconvenience.

> [@franklin52](#):
>
> ```plaintext
> awk -F_ 'a[$1]{print "fastq_to_fasta "a[$1]" "$0; next} {a[$1]=$0}' file
> 
> ```

Your code is fantastic and work fast:b:

---

<div class="post-metadata">

**Author:** ![Franklin52](https://community.unix.com/user_avatar/community.unix.com/franklin52/32/500_2.png) [@Franklin52](https://community.unix.com/u/Franklin52)\
**Post date:** [October 10, 2009, 7:13am UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/6 "2009-10-10T07:13:34Z")

</div>

```nohighlight
awk -F_ 'a[$1]{print "fastq_to_fasta "a[$1]" "$0; next} {a[$1]=$0}' file

```

Explanation:

```nohighlight
awk -F_

```

Change the default field separator

```nohighlight
a[$1]{print "fastq_to_fasta "a[$1]" "$0; next}

```

If element a[$1] of array a has a value, print a[$1], a space and the current line. Read the next line

```nohighlight
{a[$1]=$0}

```

a[$1] is empty on this line. Here the element is filled with the value of the current line

---

<div class="post-metadata">

**Author:** ![patrick87](https://community.unix.com/letter_avatar/patrick87/32/6_22734026db12803ebb3e059622ac3534.png) [@patrick87](https://community.unix.com/u/patrick87)\
**Post date:** [October 10, 2009, 8:09pm UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/7 "2009-10-10T20:09:06Z")

</div>

Thanks a lot for your explanation, Franklin52:)  
I feel more comfortable and understanding about the reason of your code now.  
I really appreciate for your explanation.  
Thanks again :b:

---

<div class="post-metadata">

**Author:** ![patrick87](https://community.unix.com/letter_avatar/patrick87/32/6_22734026db12803ebb3e059622ac3534.png) [@patrick87](https://community.unix.com/u/patrick87)\
**Post date:** [October 11, 2009, 9:55pm UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/8 "2009-10-11T21:55:41Z")

</div>

Hi, Franklin52.  
The code that you suggestion:

```nohighlight
awk -F_ 'a[$1]{print "fastq_to_fasta "a[$1]" "$0; next} {a[$1]=$0}' file

```

[/quote]  
It facing a problem if my input is look the following:  
01234\_ABC\_L1\_CDE\_PE1.fq  
01234\_ABC\_L1\_CDE\_PE2.fq  
01234\_ABC\_L2\_CDE\_PE1.fq  
01234\_ABC\_L2\_CDE\_PE2.fq  
01234\_ABC\_L3\_CDE\_PE1.fq  
01234\_ABC\_L3\_CDE\_PE2.fq  
05512\_ABC\_L1\_CDE\_PE1.fq  
05512\_ABC\_L1\_CDE\_PE2.fq  
05512\_ABC\_L2\_CDE\_PE1.fq  
05512\_ABC\_L2\_CDE\_PE2.fq  
05512\_ABC\_L3\_CDE\_PE1.fq  
05512\_ABC\_L3\_CDE\_PE2.fq

The code can't give the desired output like:  
fastq\_to\_fasta 01234\_ABC\_L1\_CDE\_PE1.fq 01234\_ABC\_L1\_CDE\_PE2.fq  
fastq\_to\_fasta 01234\_ABC\_L2\_CDE\_PE1.fq 01234\_ABC\_L2\_CDE\_PE2.fq  
fastq\_to\_fasta 01234\_ABC\_L3\_CDE\_PE1.fq 01234\_ABC\_L3\_CDE\_PE2.fq  
fastq\_to\_fasta 05512\_ABC\_L1\_CDE\_PE1.fq 01234\_ABC\_L1\_CDE\_PE2.fq  
fastq\_to\_fasta 05512\_ABC\_L2\_CDE\_PE1.fq 01234\_ABC\_L2\_CDE\_PE2.fq  
fastq\_to\_fasta 05512\_ABC\_L3\_CDE\_PE1.fq 01234\_ABC\_L3\_CDE\_PE2.fq

Unfortunately, the code give me another result ☹  
Franklin52, do you have any better idea to get my desired output result?  
Actually I got try a way like this:

```nohighlight
awk '/PE1/{print $_}' file_name > file_name.txt
awk '/PE2/{print $_}' file_name > file_name2.txt
paste -d ' ' file_name.txt file_name2.txt > file_name.output.txt

```

At last, I vi and add the "fastq\_to\_fasta" inside file\_name.output.txt one line by one line:(  
It seem like quite troublesome and wasting time ☹  
I think it should got faster way. Just I still can't get it:(  
Franklin52, thanks again for your help and sorry if bring you any inconvenience.

---

<div class="post-metadata">

**Author:** ![jp2542a](https://community.unix.com/letter_avatar/jp2542a/32/6_22734026db12803ebb3e059622ac3534.png) [@jp2542a](https://community.unix.com/u/jp2542a)\
**Post date:** [October 11, 2009, 10:45pm UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/9 "2009-10-11T22:45:26Z")

</div>

```nohighlight
awk '{a = $0; getline; print "fastq_to_fasta " a " " $0}' file

```

---

<div class="post-metadata">

**Author:** ![danmero](https://community.unix.com/letter_avatar/danmero/32/6_22734026db12803ebb3e059622ac3534.png) [@danmero](https://community.unix.com/u/danmero)\
**Post date:** [October 11, 2009, 10:46pm UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/10 "2009-10-11T22:46:49Z")

</div>

That's what you want ?

```nohighlight
# cat file
01234_ABC_L1_CDE_PE1.fq
01234_ABC_L1_CDE_PE2.fq
01234_ABC_L2_CDE_PE1.fq
01234_ABC_L2_CDE_PE2.fq
01234_ABC_L3_CDE_PE1.fq
01234_ABC_L3_CDE_PE2.fq
05512_ABC_L1_CDE_PE1.fq
05512_ABC_L1_CDE_PE2.fq
05512_ABC_L2_CDE_PE1.fq
05512_ABC_L2_CDE_PE2.fq
05512_ABC_L3_CDE_PE1.fq
05512_ABC_L3_CDE_PE2.fq
# awk 'NR%2{printf "fastq_to_fasta" FS $0 FS;next}1' file
fastq_to_fasta 01234_ABC_L1_CDE_PE1.fq 01234_ABC_L1_CDE_PE2.fq
fastq_to_fasta 01234_ABC_L2_CDE_PE1.fq 01234_ABC_L2_CDE_PE2.fq
fastq_to_fasta 01234_ABC_L3_CDE_PE1.fq 01234_ABC_L3_CDE_PE2.fq
fastq_to_fasta 05512_ABC_L1_CDE_PE1.fq 05512_ABC_L1_CDE_PE2.fq
fastq_to_fasta 05512_ABC_L2_CDE_PE1.fq 05512_ABC_L2_CDE_PE2.fq
fastq_to_fasta 05512_ABC_L3_CDE_PE1.fq 05512_ABC_L3_CDE_PE2.fq

```

---

<div class="post-metadata">

**Author:** ![patrick87](https://community.unix.com/letter_avatar/patrick87/32/6_22734026db12803ebb3e059622ac3534.png) [@patrick87](https://community.unix.com/u/patrick87)\
**Post date:** [October 11, 2009, 10:53pm UTC](https://community.unix.com/t/execution-problem-with-repeat-the-same-program-with-two-set-of-same-data/250992/11 "2009-10-11T22:53:34Z")

</div>

thanks a lot, danmero:)  
Your code work excellent and solve my trouble nice.  
Can you roughly explain the reason of your code?  
Thanks again.