Shell Script average runtime

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!

  1. The problem statement, all variables and given/known data:
    Make a bash script that calculates average runtime for the first two scripts you made. The average should be based on 100 runs of the scripts. For measuring the runtime you shall use the built in GNU time(1) function (/usr/bin/time). For calculating the average (by addition and floatdivition) use bc(1).

  2. Relevant commands, code, scripts, algorithms:
    bash
    /usr/bin/time
    bc

return values from the built in time:

0.00user 0.01system 0:00.11elapsed 13%CPU (0avgtext+0avgdata 0maxresident)k0inputs+0outputs (0major+529minor)pagefaults 0swaps

  1. The attempts at a solution (include all code and scripts):

script 1

#!/bin/bash
for I in {1..6}
do
        echo "downloading 0$I.html"
        wget filefromurlhere    <-(not adding this as it requires a password and stuff)
 echo "deleting 0$I.html"
        rm 0$I.html
done

script2

#!/bin/bash
 for I in {1..6}
do
        echo "downloading 0${I}.html"
        wget "file from url here"
        wait
        echo "deleting 0${I}.html"
 rm "0${I}.html"
 done

script3 for average runtime.

#!/bin/bash
 
#/usr/bin/time
for I in {1.100}
do
        echo $(time script1.sh 2>&1)+
done | sed 's/+$//\n/' | bc
  1. Complete Name of School (University), City (State), Country, Name of Professor, and Course Number (Link to Course):
    Norwegian University of Science and Technology NTNU, Trondheim, Norway, Svein Erik Bratsberg, TDT4186 Operativsystemer, h t t p : / / w w w . i d i . n t n u . n o / e m n e r / t d t 4 1 8 6 / (can't post urls, remove spaces ;D )

Note: Without school/professor/course information, you will be banned if you post here! You must complete the entire template (not just parts of it).

I'm in need of the elapsed return value from the time (atleast that's what makes sense to me), added up for 100 run throughs and divided by 100 to find the average time, using bc to do it all. just not sure how to get exactly those numbers from all the junk I get returned with the built in time function.

I know my 3rd script might be far off, but I'm a real newbie to linux scripting :slight_smile:

---------- Post updated at 10:54 PM ---------- Previous update was at 06:00 PM ----------

Well I found I could use awk like this, but I really need it to do the calculations in bc. can I just replace awk with bc in this and have it do what it's supposed to? D:

usr/bin/time -p {myScript} | grep elapsed | awk '{print $2}' >> datafile.dat
awk 'BEGIN {t=0.0;c=0;a=0.0;} {t+=$1; c++;} END {printf("Total %f Count %d Avg %f\n", t, c, t/c);}' < datafile.dat

---------- Post updated 10-12-11 at 06:06 AM ---------- Previous update was 10-11-11 at 10:54 PM ----------

I'll update in a new post instead of editing, I've had a pretty major breakthrough, just being stuck trying to get my stdin into my calculations.

script1

!#/ in/bash
for I in {1..6}
do
wget --quiet myUrl/file$I.html
done
rm 0{1..6}.html

script2

#!/bin/bash
for I in {1..6}
do
wget  --quiet myUrl/file$I.html &
done
wait
rm 0{1..6}.html

script3 for runtime calculations

#!/bin/bash
for I in {1..5}
do
/usr/bin/time -f "%E (elapsed)" ~/script1.sh 2>&1
echo 'x+stdout' <--- how to get the value of stdout for calculations?
done
echo 'scale=20;x/100' | bc
echo "gjennomsnittstid for skript1a er $x"
 
for I in {1..5}
do
/usr/bin/time -f "%E (elapsed)" ~/script2.sh 2>&1
echo 'y+stdout' | bc <--- how to get the value of stdout for calculations?
done
echo 'scale=20;y/100' | bc
 

so yeh. I've rerouted the elapsed time from my time call into stdout, but how do I get it back so I can calculate on it?

redirect the output of 'done' into 'bc' with a pipe. That's valid for entire loops and such in the bourne shell, not just individual commands.

1 Like

ok, not really got a clue how, but thanks. trying to look into it :smiley:

It's simpler than whatever you're thinking. I only mean exactly what I'm saying:

for I in {1..5}
do
/usr/bin/time -f "%E (elapsed)" ~/script2.sh 2>&1
done | program1 | program2 ...

You'll need to organize the text somehow before sending it into bc, but not knowing what your 'time' program prints, I can't say how.

awk would probably be better than bc, but if they insist on you using the worst program for the job... :wall:

well with that format I print 0:00.00elapsed , asuming minutes:seconds.tenth of a second. I'd be interested in adding up all the seconds.tenths of seconds during the loop, then averaging by dividing on number of runs. supposed to run it 100 times, but my test code runs 5 for simplicity till I get it going.

Does it really print that? Or does it actually print "0:00.00 (Elapsed)"? Be specific, if we make a program for the wrong input it won't give the right output.

You can probably make that simpler, by just feeding it "%F" instead of "%F (elapsed)", so it prints 0:00.00 per loop.

Then feed it through another loop so you can transform it into an expression to feed into bc.

Here's an incomplete example:

(

echo "(" # Start of bracket

while whatever
do
...
done | while IFS=":." read MIN SEC TENTH
do
        echo "( ( $MIN * 60 ) * $SEC) +"
done

echo "0 )" # End of bracket.  So we get ( A + B + C + D + 0 )
echo "/ 100" # Divide the whole mess by 100
 ) | bc

0:00.05 (elapsed) indeed, forgot the space there. changed it to "%E" to only get the numbers.

---------- Post updated at 11:42 AM ---------- Previous update was at 11:22 AM ----------

#!/bin/bash
(
echo "("
for I in {1..5}
do
/usr/bin/time -f "%E"  ~/oblig1/oblig1a.sh 2>&1
done | while IFS=":." read MIN SEC TENTH
do
echo "( ( $MIN * 60) * $SEC ) +"
done
echo "0 )"
echo "/ 100"
) | bc

getting standard_in errors tho, syntax error and variable occurances of illegal character. :confused:

(standard_in) 1689: illegal character: L
(standard_in) 1689: syntax error
(standard_in) 1689: illegal character: K
(standard_in) 1689: syntax error
(standard_in) 1689: syntax error
(standard_in) 1690: illegal character: S
(standard_in) 1690: syntax error
(standard_in) 1691: syntax error
(standard_in) 1692: illegal character: K
(standard_in) 1692: syntax error
(standard_in) 1693: syntax error
(standard_in) 1694: syntax error
(standard_in) 1694: syntax error
(standard_in) 1695: syntax error
(standard_in) 1696: syntax error
(standard_in) 1696: syntax error
(standard_in) 1696: syntax error
(standard_in) 1697: illegal character: L
(standard_in) 1697: illegal character: \370
(standard_in) 1697: syntax error
(standard_in) 1697: syntax error
(standard_in) 1698: syntax error
(standard_in) 1698: syntax error
(standard_in) 1699: illegal character: H
... and the list goes on for ages

Leave off the | bc and see what you're actually feeding into bc. If your script prints anything, you'll have to redirect that to /dev/null so that it doesn't get fed into bc.

shit got messy without the | bc ... x)

~/oblig1$ bash oblig2.sh
(
( ( /home/oblig1/oblig1a * 60) * sh ) +
( ( --2011-10-12 20 * 60) * 03 ) +
( ( L�ser osyda50 * 60) * hive ) +
( ( Ansluter till osyda50 * 60) * hive ) +
( ( HTTP foresprrsel sendt, mottar topptekster * 60) *  ) +
( ( �teranv�nder befintlig anslutning till osyda50 * 60) * hive ) +
( ( HTTP foresprrsel sendt, mottar topptekster * 60) *  ) +
( ( Lengde * 60) *  20732 (20K) [text/html] ) +
( ( Sparar till "01 * 60) * html" ) +
( (  * 60) *  ) +
( (      0K  * 60) *  ) +
( (  * 60) *  ) +
( ( 2011-10-12 20 * 60) * 03 ) +
( (  * 60) *  ) +
( ( --2011-10-12 20 * 60) * 03 ) +
( ( L�ser osyda50 * 60) * hive ) +
( ( Ansluter till osyda50 * 60) * hive ) +
( ( HTTP foresprrsel sendt, mottar topptekster * 60) *  ) +
( ( �teranv�nder befintlig anslutning till osyda50 * 60) * hive ) +
( ( HTTP foresprrsel sendt, mottar topptekster * 60) *  ) +
( ( Lengde * 60) *  36367 (36K) [text/html] ) +
( ( Sparar till "02 * 60) * html" ) +
( (  * 60) *  ) +
( (      0K  * 60) *  ) +
( (  * 60) *  ) +
( ( 2011-10-12 20 * 60) * 03 ) +
( (  * 60) *  ) +
( ( --2011-10-12 20 * 60) * 03 ) +
( ( L�ser osyda50 * 60) * hive ) +
( ( Ansluter till osyda50 * 60) * hive ) +
( ( HTTP foresprrsel sendt, mottar topptekster * 60) *  ) +
( ( �teranv�nder befintlig anslutning till osyda50 * 60) * hive ) +
( ( HTTP foresprrsel sendt, mottar topptekster * 60) *  ) +
( ( Lengde * 60) *  14439 (14K) [text/html] ) +
( ( Sparar till "03 * 60) * html" ) +
( (  * 60) *  ) +

Yeah; anything that prints to stdout in that loop gets sent to bc, that stuff included. You have to redirect your script to /dev/null with > /dev/null after the program.

1 Like
~/oblig1$ bash oblig2.sh
(standard_in) 2: syntax error
(standard_in) 2: syntax error
(standard_in) 3: syntax error
/usr/bin/time -f "%E"  ~/oblig1/oblig1a.sh >/dev/null 2>&1

Try reversing the order, 2>&1 > /dev/null

that will bring me back to the loooong list of standard_in errors I had before. it pretty much puts me back where I was without >/dev/null by the looks of it when I remove | bc to check again.

it seems the problem is

:~/oblig1$ bash oblig2.sh
(
0 )
/ 100

okay, apparently bc doesn't like newlines.

You're also not getting any of the output you wanted. :frowning: You might have to make a version of your script which prints no output.

Instead of echo "stuff" do

printf " %s " "stuff" so you'll get one long output string separated by spaces instead of newlines.

 
#!/bin/bash
(
for I in {1}
do
/usr/bin/time -f "%E"  ~/oblig1/oblig1a.sh >/dev/null  2>&1 
done | while IFS=":." read MIN SEC TENTH
do
printf " %s " "( ( $MIN * 60) * $SEC ) +"
done
) 

doesnt return anything atm tho.

Probably because you're still redirecting the output you need...

How about this: Save it in a temp file then use that file.

#!/bin/bash
for I in {1}
do
        /usr/bin/time -f "%E"  ~/oblig1/oblig1a.sh
done 2> timefile

(       while IFS=":." read MIN SEC TENTH
        do
                printf " %s " "( ( $MIN * 60) * $SEC ) +"
        done

        echo "0" # Print the last number so it's ...+0.  Also, a newline.
) < timefile | bc

Also: "for I in {...}" is inefficient and prone to breaking when you run out of arguments. how about:

for ((I=0; I<10; I++))
do
...
done
 
(standard_in) 1: syntax error
(standard_in) 1: illegal character: :

without | bc

~/oblig1$ bash oblig2.sh
 ( ( oblig2.sh: * 60) * line ) +  ( ( oblig2.sh: * 60) * line ) +  ( ( oblig2.sh: * 60) * line ) +  ( ( oblig2.sh: * 60) * line ) +  ( ( oblig2.sh: * 60) * line ) + 0

---------- Post updated at 03:16 PM ---------- Previous update was at 02:24 PM ----------

ok Corona688, thanks for all help, can lock this down now. didnt solve it, but running out of time before I gotta turn it in anyhow. your help has ben mighty appreciated, thanks for wasting your time on a newbie like me :slight_smile:
if you're interested I can get you the solution as to how to get it running with bc next week if you want. ;D

Your script is chattering junk into stderr, you need to fix the errors in your script before you'll have clean output to put into bc.

It was a silly question anyway. :confused: You solved it hours ago and more easily, just with the 'wrong' tool.