How can i speed this script up?

Shell Programming and Scripting
1

Hi,

Im quite new to scripting and would like a bit of assistance with trying to speed up the following script. At the moment it is quite slow....

Any way to improve it?

total=111120
while [ $total -lt 111130 ]
do
total=`expr $total + 1`
INCREMENT=$total
firstline = "blablabla"
secondline = "blablabla" $INCREMENT
Thirdline = "blablabla"
fourthline = "blablabla"
fifthline = "blablabla"
sixthline = "blablabla"
seventhline = "blablabla"

echo $firstline >> newfile.xml
echo $secondline >> newfile.xml
echo $Thirdline >> newfile.xml
echo $fourthline >> newfile.xml
echo $fifthline >> newfile.xml
echo $sixthline >> newfile.xml
echo $seventhline >> newfile.xml

database query which uses file

echo file done

done

Thanks!

2

Please clarify your final goal : what are you trying to do ?

Did you consider

cat -n yourfile

?

There are several posts around that talk about xml files generation and/or parsing
You can use the Search tool from menu to look for those posts.

---------- Post updated at 06:57 PM ---------- Previous update was at 06:38 PM ----------

INCREMENT=111120
while [ $INCREMENT -lt 111130 ]
do
let INCREMENT+=1

echo "blablabla
blablabla $INCREMENT
blablabla
blablabla
blablabla
blablabla
blablabla" >> newfile.xml

# ...

echo "file $INCREMENT done"

done

?

3

Do you realize you're never clearing that XML file? It's just growing and growing every loop!

Anyway, you can put all that text in a here-document to avoid 99 echo calls writing to the same file:

cat <<EOF >file.xml
line1
lne2 $variable
line3
EOF

Instead of running the same thing 9 times to do 9 queries, you could probably run the same thing once to handle all nine queries, tying them directly in with a pipe.

total=111120
while [ $total -lt 111130 ]
do
        total=`expr $total + 1`

# Note the ending EOF below the last 'blablabla' MUST be at the very
# beginning of the line or the here-document will eat the entire rest of the
# script!
        cat <<EOF
blablabla
blablabla $total
blablabla
blablabla
blablabla
blablabla
blablabla
EOF

        # Print this to stderr so it doesn't end up in your query or what have you
        echo file done >&2
done | database_query_using_stream

What is this 'database query which uses file'? I suspect that's going to be the limiting step here and we can't tell you how to rewrite it without seeing it.

4

Please post what Operating System and version you have and what Shell you are using.
Please quantify a performance problem with real numbers and mention the hardware specification and other contention for resources which gives a feeling for scale. There's a whole difference between performance problems on a home PC and those on a 10,000 user system.

5

Using AIX 5.2. Scripting in #!\bin\sh.

Ok - Just to clarify what i am trying to do. I am trying to get the creation of the file to be as efficient as possible. Or there may be another way of achieving my objective without creating a file. Explained below....

The first part of the file variable written to the file should read:

echo $firstline > newfile.xml

The $firstline, $secondline variable are actually XML tags with information in.
The $thirdline contains an Insert statement to a database
<Insertquery>Insert into newtable<Insertquery>
The $forthline contains the values to enter into the table
<values>'one', '2012-03-23 $INCREMENT', 'message'<values>

I am trying to update a database table which can only be updated through the use of an xml file with all the various tags in and the correct format.

The newfile.xml is the file that is used as the template which updates the database. However, the second value to be entered in the database need to be a unique value. This is why i have added the $INCREMENT in the values.

A java call is made and it used the XML file to insert into a database the values. I need to insert 1000 records however, i think the creation of the file is making it slow at the moment as it is taking 1 second per transaction.

6

There is not need to create a file. I would use a here document like Corona suggested into a stream. Did you try his suggestion? If you need to use variables for some reason you can also use them inside the here document, if the need to be separate queries then you can also try and replace the cat statement with your database query statement and not use a pipe:

database query << EOF
....
EOF

The second EOF needs to be at the beginning of the line and you cannot use indentation..