How to convert this script so that it runs the dbv at most 5x at a time instead of serially?

Hi,

Attached is a generated script of the database files if/when we want to run dbv = dbverify.

The script runs fine by itself but it does take awhile to finish. In a worse case scenario, it can take up to 7 hours to finish as the dbv runs serially.

I need help and guidance to somehow have some parallelism in terms of running the dbv. For instance, I want to at least have 5xdbv running at some point in time. So for a start, run 5 of these dbv's, and then 'monitor' in the background if some has finished, then start more dbv until there's 5xdbv running again and so on and so forth.

As there are several databases on the same server, I cannot simply ps -ef | grep dbv to check for the number of dbv commands running. That is, I may have two of these scripts that I want to run for two different databases. So the 5xdbv run is for each database. If there is two of these scripts then there should be 10xdbv running at one time, 5 for each database.

Not sure where or how to start :o Any advice will be much appreciated. Thanks in advance. BTW, prefers something that I can use on both Solaris and Linux

P.S.:
Belated Merry Christmas, Happy New Year to All.

You should know the answer to the following questions before you start.
What effect does increasing the block size from 8192 to 32768 have?
What percentage of disk i/o capacity does a step take? You can use sar to determine this. and from this you should be able to calculate the number of megabytes per hour that you can process at a given disk usage threshold.

Pick two or three of the largest tables and manually run them in series. Record the wall clock time. Run them again in parallel and record the wall clock time. If you have other users on the system while you are running this experiment. record the response times when running serial and parallel.

2 Likes

Hi jgt,

I don't want to change the block size. I just want to be able to run several dbv's instead of one at a time.

Anyway, I think Run multiple sessions of DBV has what I am after.

Thanks.

I think we (that includes jgt) have understood that. The problem is: verifing the DB creates a certain amount of usage on your disk subsystem because it causes (quite some amount of) disk I/O. if you run several instances of it in parallel it depends on the amount of disk I/O a single instance creates if these several instances run faster linearly with the number of instances.

i.e. if a single instance of the script taxes the disk 10% then running 5 simultaneous instances will drive that up to 50% and run perhaps somewhere in the ballpark of 1/5 of the time. If a single instance taxes the disk already 100% then running 5 instances in parallel will perhaps run about as fast a single instance because you can't tax any resource more than 100%. The disk I/O will be queued (thus serialised again) and this will put a cap at runtime. Perhaps reality is somewhere in between those extremes but you may want to find out beforehand where you stand.

I hope this helps.

bakunin

1 Like

Hi,

Yeah we have tested with running 5 dbv's at a time and the load is acceptable to the server.

--- Post updated at 04:25 PM ---

Hi jgt

Finally got to understand what you are wanting me to check first. Got confused when you mentioned about the block size thingy.
Thanks.

FYI, had decided to keep it 'simple' use wait instead and run the dbv in the background using &. It's kinda 'parallel' but running it in group of 5 'serially' :slight_smile: