FASTEST way to loop a script 10k times

Shell Programming and Scripting
1

Is there any FASTEST way to loop a script 10k times

my script works likes this

c-randomnumbers-script -i input1.bed -g g19 -e DB >> output1

I need to run this 10k times by using consecutive outputs to get my final output i.e, output10000

c-random-script -i input1.bed -g g19 -e DB >> output1
c-random-script -i output1 -g g19 -e DB >> output2
...................
c-random-script -i output9999 -g g19 -e DB >> output10000
2

not fastest, maybe shortest.

seq 1 10000 |while read line
do
  c-randomnumbers-script -i output${line} -g g19 -e DB >> output${line}
done
3

From your text I get the impression you are concerned only with the final file and not the 9,999 intermediate ones, and from that it doesn't seem that you are appending to existing output files as your example suggests, but are creating new intermediate files to be used only for the next input. If I assume correctly then this should work:

input=input1.bed
for (( i = 0; i < 10000; i++ ))
do
    c-random-script -i $input -g g19 -e DB >output
    mv output last_output
    input=last_output
done
mv last_output final-desired-name

The speed will depend completely on the time it takes "random script" to execute 10K times.

4

More clear explanation and a small change in the request.

c-randomnumbers-script -i input1.bed -g g19 -e DB >> outputX1
intesect-script -a outputX1 -b input2.bed -wa -wa  -f1 |awk '!a[$4]++' |awk '{print $1"\t"$2"\t"$3"\t"$4"\t"$5"\t"$6"\t"$10}' >> outputY1
rm outputX1

# I need to run this 10k times by using same input to get final 10k outputs i.e outputY1 to outputY10000

# Then output a table having 4th column and 10th column in a file named stats.txt and calculate mean and standard deviation of 7th column 0f 10k outputY* files

c-randomnumbers-script -i input1.bed -g g19 -e DB >> outputX1;
intesect-script -a outputX1 -b input2.bed -wa -wa  -f1 |awk '!a[$4]++' |awk '{print $1"\t"$2"\t"$3"\t"$4"\t"$5"\t"$6"\t"$10}' >> outputY1;
rm outputX1 ;

c-randomnumbers-script -i input1.bed -g g19 -e DB >> outputX2;
intesect-script -a outputX2 -b input2.bed -wa -wa  -f1 |awk '!a[$4]++' |awk '{print $1"\t"$2"\t"$3"\t"$4"\t"$5"\t"$6"\t"$10}' >> outputY2;
rm outputX2;

.........

c-randomnumbers-script -i input1.bed -g g19 -e DB >> outputX10000
intesect-script -a outputX10000 -b input2.bed -wa -wa  -f1 |awk '!a[$4]++' |awk '{print $1"\t"$2"\t"$3"\t"$4"\t"$5"\t"$6"\t"$10}' >> outputY10000
rm outputX10000

stats.txt

4thcolumn    10thcol_total    mean    SD
id1_outputY1-id_outputY10k    2000    20    2
5 
seq 1 10000 |while read line
do
  c-randomnumbers-script -i input1.bed -g g19 -e DB >> outputX${line}
  intesect-script -a outputX${line} -b input2.bed -wa -wa  -f1 |awk '!a[$4]++ {print $1, $2, $3, $4, $5, $6, $10}' OFS="\t" >> outputY${line};
  rm outputX${line}
done

You need explain the second rules more detail, I don't see 10th column in your sample stats.txt

Then output a table having 4th column and 10th column in a file named stats.txt and calculate mean and standard deviation of 7th column 0f 10k outputY* files
6

Yeah sorry for explaining clear. I spend some time explain thing more clearly and also included some coded that i needed to add.

I need to calculate the mean and standard deviation of the 7th column keys by using output1 to output10000.
For ex:

output1
col1 col2 col3 col4 col5 col6 col7
abc1 10 20 x.a.1 0 + key1
abc1 15 20 x.a.2 0 + key1
abc1 10 20 y.a.1 0 + key2
abc1 10 20 c.a.1 0 + key2
abc1 10 20 r.a.1 0 + key2
abc1 50 80 m.a.1 0 - key_3

output2
col1 col2 col3 col4 col5 col6 col7
abc1    5    20    x.a.1    0    +    key1
abc1    105    200    x.a.2    0    +    key1
abc1    150    250 y.a.1    0    +    key2
abc1    130    240    c.a.1    0    +    key3
abc1    130    220    r.a.1    0    +    key3
abc1    500    800    m.a.1    0    -    key_3

output10k
....................................

The count the keys based on output

output1_key1    2
outut1_key2    3
output1_key_3    1
output1_key1    2
outut1_key2    3
output1_key_3    1
output10k_key..........

Then calculate mean using

awk �{s+=$2} END{print �Sum: �s, �\nNumber of lines: �NR, �\nAverage: �s/(NR)}�

Then calculate standard deviation using

awk �{sum+=$2;sumsq+=$2*$2} END {print sqrt(sumsq/NR � (sum/NR)^2)}�

final output

key    mean    s.d
key1
key2
key_3
......
key_16

---------- Post updated at 06:37 AM ---------- Previous update was at 12:44 AM ----------

I wrote more clear explanation. Please let me know it is clear enough. Thanx