loop through numbered filenames

Shell Programming and Scripting
1

Hi I'm very new to this script thing, so please be gentle.

I am trying to get a command - the mach2qtl command in the code below - to loop through a set of files.
Each command should take the same two .dat and .ped files, but the .mlinfo and .mlprob files with filenames including 'chrom1' should be run together and output to a file PC_full_1.txt

I have run the command fine by retyping it using the appropriate file names, but would like to run this as a script that runs through files 1 - 22.

When I run the script below I get a segmentational error.

What am I doing wrong?

Thanks,

Polly

 
#! /bin/bash
# program to run through each chromosome carrying out association using mach2qtl
 

# increment through chromosome numbers
i=0
while [ $i -lt 23 ]
        do
                i=$(( $i +1 ))
#list info and prob files 1 - 22
                infofile=chrom$i_step2.mlinfo
                probfile=chrom$i_step2.mlprob
# run mach2qtl using the above specified files
                mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile > PC_full_$i.txt

done
exit 0
2

Hi
The option "--probfile" is correct or you have to use a single symbol "-"?

can you provide us the error message returned by the script ?

3

Here is the list of errors, --probfile is correct, as I said this command works on its own, just not when in the script.

./assoc.sh: line 24: 31017 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31018 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31019 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31020 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31021 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31022 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31023 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31024 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31025 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31026 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31027 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31028 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31029 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31030 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31031 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31032 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31033 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31034 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31035 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31036 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31037 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31038 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
./assoc.sh: line 24: 31039 Segmentation fault mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt

4

I've realised the - while [ $i -lt 23 ] - should probably be -lt 22 instead.

However, I'm still getting the same fault message.

5

Ok - I've put some 'echo's in so I could diagnose the problem

1
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4075 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
2
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4076 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
3
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4077 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
4
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4078 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
5
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4079 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
6
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4080 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
7
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4081 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
8
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4082 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
9
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4083 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
10
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4084 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
11
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4087 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
12
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4088 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
13
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4089 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
14
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4090 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
15
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4091 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
16
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4092 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
17
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4093 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
18
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4094 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
19
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4095 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
20
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4096 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
21
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4097 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt
22
chrom.mlinfo
chrom.mlprob
./assoc.sh: line 27:  4098 Segmentation fault      mach2qtl -d PC_full.dat -p PC_full.ped -i $infofile --probfile $probfile >PC_full_$i.txt

so my incremental numbering is working put the value of 'i' is not being used in the filenames.

1
chrom.mlinfo
chrom.mlprob

should be:

1
chrom1.mlinfo
chrom1.mlprob

what am I doing wrong?

I've tried with and without double quotes (") around filenames.

6

Ok solved it myself. I think.
I needed curly brackets around the variable i within the filename

:slight_smile:

7

The error message "segmentation fault" comes directly from the OS. Somehow it tries to execute the program "mach2qtl" and this program crashes. This is definitely not a problem coming out of your script, at least not directly.

A little hint: if you have a loop which behaves strangely try to "echo" out all the commands in question. This way you will see (instead of execute) what is to be execute:

i=0
while [ $i -lt 10 ] ; do
     command_in_question someparameter $i some_more_param
     (( i += 1 ))
done

Modify this to:

i=0
while [ $i -lt 10 ] ; do
     echo command_in_question someparameter $i some_more_param
     (( i += 1 ))
done

to see if what you think is executed is really executed. In loops more complicated than this example this can widely differ.

Another possibility is to insert "set -xv" at the start of the loop and "set +xv" at the end. This sends every line as it is interpreted to stderr. ("set -xv" turns this on, "set +xv" turns it off). You can monitor the script as it is executed like this:

script param1 param2 .... 2>&1 1>/dev/null | more

Lets talk about your script itself now:

infofile=chrom$i_step2.mlinfo

While not directly incorrect this could be misleading to the shell: suppose you have two variables, "$i" and "$i_". How should the shell find out which one you mean? If you want to expand a variable in the middle of a string write it this way:

infofile=chrom${i}_step2.mlinfo

This will remove all ambiguity. Further, make it a habit to ALWAYS QUOTE your strings, because a misplaced (or unexpected) space character could well lead to disaster, because it is a separator to the shell. Therefore always write

infofile="chrom${i}_step2.mlinfo"

and similarly for the other line in your script.

Btw., it seems that your variables "$infofile" and "$probfile" are not getting expanded, because otherwise there would be the contents of them in the error message.

I hope this helps.

bakunin

8

Thanks bakunin - the quotes and {} definitely were causing the problem in this instance.

I'll use your tips for my future programming :slight_smile: