Controlling time stamps in a bash script

Hi,
I have a bash script that generates CSV (.txt) files at fairly regular time intervals. I'm currently time stamping each batch of measurements at the time I write the rows into a MySQL database. As the result, one set of data might get the time 12:01:32 and the next set of data gets the time 12:03:14. The average time between measurements is about 1-3 minutes.

This works fine, but I'm working on a frontend where I want to display measurements from various events in the same graphs, and the resolution should be 5 minutes.

So my script should ideally only kick off a new measurement every 5th minute and when it stores the times it should also be at even 5-minute intervals.

Some pseudo code showing the current behavior if it explains better:

#!/bin/bash

while true
do

   # Code to generate csvFile.txt. Should be initialized every 5th min.

   # Below is how I currently time stamp the rows in the database
   # Here I want the time given at fixed 5 minute intervals
   awk -v A="$(date "+%y%m%d  %T")"  '{print A,$0}' csvFile.txt | tr -s ' ' 
   ';' > csvFileToDB.txt

   # Code to inser the resulting files to MySQL not shown here
 fi
done

To summarize the wanted behavior with an example:

The script starts a new measurement at a fixed 5-minute interval. For example, one interval should start at 12:00. It then generates the CSV file (it normally takes one to two minutes) and stores the values to the database with the start time for the interval (in this case 12:00). Then the script waits until 12:05 before it starts over with the same procedure again.

Anyone who knows how to do this?

Thanks
//Z

Use cron to run things you're scheduling regularly, put it in your crontab(crontab -e) like this:

0-59/5 * * * * /path/to/script.sh

Edit your cron table with cron -e. If you get errors, try setting a more complete PATH in your script.

Can you insert a sleep command into the loop? If it kicked off a background job every five minutes, sleep 300 would do the job. If every five minutes was a definite goal, code to create a time five minutes from now could be put into a script variable, then you would start your procedure. After that procedure, another loop that would only sleep for 1 second would wait for the target time to catch up.
If exactly five minutes is not that critical, just sleep for 3.5 minutes (tweak as needed) before starting over.

Thanks for suggestions. I need this to happen at exactly 5 min intervals as I have other databases with values also from each 5 min and I want to be able to print data from different tables in the same graphs. So the times have to be aligned (means 'sleep' is out of question). I will read up on crontab and see if it does the trick.

That's pretty much what cron already does for you -- sleeps an exact interval, corrects, and runs your program.

Ok, tested and that was EXACTLY what I wanted to achieve. Thanks a lot Corona688!

Back to this one again. Tried to automate a script called getStats.sh that looks like this:

#!/bin/bash
./startMeasuring.sh
./collectStats.sh

I go to crontab -e and add this line:

0-59/5 * * * * /usr/Measurements/getStats.sh

The problem is that nothing happens. I have tried to reboot the computer, but the scheduled task never starts.

Does someone know what the problem is? Or do you know how to figure it out through logs etc?

Cheers/Z

Any error messages? I guess it starts but doesn't find the two scripts called. cron only provides a minimalistic environment, so you need to either define your own or use absolute pathnames for scripts.

Hello Zooma,

If you want your script to run each 5th minute then could you please try following into your crontab by doing crontab -e and let us know if this helps you.

0,5,10,15,20,25,30,35,40,45,50,55 * * * * /path/to/script.sh

Thanks,
R. Singh

cron uses a very minimal path. I suggest setting your own PATH, or sourcing /etc/profile, in your scripts or it may not find all the commands they need.

0-59/5 * * * * . /etc/profile ; /usr/Measurements/getStats.sh

Just using /etc/profile to set the environment for a user's cron job probably isn't going to help much. There is a good chance you'll also need the user $HOME/.profile , $HOME/.bashrc , or some other user specific initialization script that the script being executed (and any commands it invokes) depend upon. The problem is that you are running a shell script that uses relative pathnames to find scripts to run:

./startMeasuring.sh
./collectStats.sh

and there is nothing in your script to move into the directory that would make those relative pathnames valid. And, we don't know whether or not those scripts depend on being invoked from a certain directory so they can find other files.

The directory in which cron runs a job is implementation specific, but the likelihood that it will be the current working directory at the time that the user invoked crontab to add that job to the list of jobs cron will execute is extremely small.

Any script being executed by cron needs to be capable of setting up whatever environment variables and it needs to run and needs to be capable of making its current working directory appropriate for it to access any files it needs to access and to put any files it may create in a place where the user will be able to find them when the script completes.

Thanks for all replies. I tried calling for a more simple file (echo "hello") and it works fine, so it seems that the problem is related to that I try to run scripts that are calling for other scripts (as some of you suggested).

I have close to 20 bash/awk scripts that are located in the /usr/Measurement/ folder. The file that I call every 5th minute with Crontab is supposed to coordinate and run these scripts and then populate a database with the output.

Questions:

1. Is this doable with Crontab or are there too many pitfalls?
2. Is there some other better way of scheduling these tasks? The measurements need to happen exactly every fifth minute, and as the time to run through the ~20 scripts may vary I can't use sleep() or anything else that is relative to the excecution time. Has to be a timer.

Cheers and thanks for all answers!
Z

That's what cron is made for, given its "features" are dealt with, e.g. sourcing various .profile scripts in the beginning of your scripts, as proposed above.
How exact does the start of the various script need to be? Although cron closely interacts with the system clock, there are situations (e.g. heavy system load) wherein starts can be delayed.

Thanks Rudi. I will try to fix it and post the "solution" here once done. It doesn't have to be exact, it can be a bit late but not drift.

Cheers,
Z

Thanks to your comments above, I managed to solve the problem. I made two changes:

1. In crontab, I added an '&':

0-59/5 * * * * /usr/myFolder/start.sh &

2. I added the working directory for all files (both scripts that I execute and temporary txt files I write to). Like this:

Before:

./fileName.sh

After:

/usr/myFolder/fileName.sh

This did the trick and the start.sh file is executed every 5th minute.

I'm glad that you got it to work. With your updated scripts, what happens if you just use:

0-59/5 * * * * /usr/myFolder/start.sh

(leaving off the & )? If you're trying to avoid having cron send you mail showing output produced by your script, you can do that with redirections instead of creating extra processes:

0-59/5 * * * * /usr/myFolder/start.sh >/dev/null 2>&1

Hi Don,
Seems you have a valid point. I removed the "&" and it still works fine. Thanks.

Cheers
//Z

Did you ever check the output of your cron : by default , stdout is send to the user's mailbox.
So check :

#mailx 

@blastit.fr
No mails received. No log output. But now it works.