Two variables in output file name nested for loops

Shell Programming and Scripting
1

I am trying to use two nested for loops to process some files and then create a new file using both variables in the output file name. I have several files in this naming style:

S1_L3_all_R1.fastq
S1_L3_all_R2.fastq
S1_L4_all_R1.fastq
S1_L4_all_R2.fastq
.
.
S1_L8_all_R1.fastq
S1_L8_all_R1.fastq

I need to apply a process to each file and rename the files like this:

S1_L3_all_R1.fasta
S1_L3_all_R2.fasta
.
.
S1_L8_all_R1.fasta
S1_L8_all_R2.fasta

I am using BASH. Here is what I have tried:

for((n=3;n<=8;n++))
do 
for((k=1;k<=2;k++))
do 
for f  in S1_L$n_all_filt_R*.fastq
do 
echo $f
cat $f | perl -e my file processing > S1_L$n_all_filt_R$k.fasta
done
done
done

This runs through the file processing for each file but the file name is S1_L1.fasta and this file name is recycled for each of my original files. It seems to me that everything after the first variable in my output file name is ignored.

How can I use two variables in a file name to get the output file names I am looking for?

2

You get a useless use of cat award :wink:

This is the problem:

$ n_=1234
$ echo $n_

1234

$

_ is being taken as part of the variable name here, and will do so inside strings too. To make it not do so, do ${n} instead of $n. I try to do this habitually, even.

In short:

for((n=3;n<=8;n++))
do 
        for((k=1;k<=2;k++))
        do 
                for f  in S1_L${n}_all_filt_R*.fastq
                do 
                        echo $f
                        perl -e my file processing <"$f" > S1_L${n}_all_filt_R${k}.fasta
                done
        done
done

Though I might just do:

for n in 3 4 5 6 7 8
do
        for k in 1 2
        do
        ...
        done
done

That way, this code could run in any Bourne shell, not just BASH.

3

Corona,

Thank you for your reply. I knew that using cat in that way wasn't necessary but I don't have enough experience to know a better way to do what I needed to do. Your revisions work great! I did have to change the * that I was using to ${k}: the * was running the process on the correct file but was writing the output to the same file name (reusing file names).

I learned three things from you today; the use of {} with a variable, how to not use cat in this instance and an alternative way to acomplish the same thing.

Thanks!

4

It's not a special case. :slight_smile: Anywhere at all you have cat file | command , you can do command < file instead. You can even do so for entire shell statements, loops, and subshells the same way -- put the redirections on the end.

# This entire loop of code will have 'filename' as its standard input
while read LINE
do
...
done <filename

# This entire subshell will read from 'filename' as its standard input and write to 'outfile' as its standard output
# 'read' will read the first line.  Then 'echo' will write 'something' into 'outfile'.
# Then 'grep' will hunt for 'word' in the rest of the file and print any results after 'something' in 'outfile'.
( read FIRSTLINE ; echo "something" ; grep "word" ) <filename >outfile

The special case is whether you could do command filename instead of command <filename . Whether that works depends entirely on the program you're giving the filename to.

This is one of the big differences between the Bourne shell and the C shell, incidentally. Bourne lets you put complicated redirections wherever it's convenient. C shell only lets you put very simple redirections, in very specific places.