Hi,
I'm looking at doing a cron job for a script that could take a very short time to complete or a very long time to complete based on variable activity on a server. I don't want to do a weird schedule, but I don't want to be so aggressive that the process attempts to start before another instance has started.
How do I put in a check to see if a process (in this case, "overviewer.py") is still running, and if so, end the script without starting another instance of overviewer.py?
Thank you!
This is what PID files are generally used for.
When you start the first process, save its PID into a file in /var/run/processname.pid to check later.
Next time you want to start another, check
if [ -f /var/run/processname.pid ] && read PID < /var/run/processname.pid && ps "$PID" >/dev/null 2>/dev/null
then
echo "Already running as PID $PID"
exit 1
fi
OK... So I've never done anything like that before...
Here's the script I have that I want to modify.
#!/bin/bash
echo Overviewer Starting... >>/home/mcmaps/scripts/overviewer.log
date >>/home/mcmaps/scripts/overviewer.log
echo ------------------------- >>/home/mcmaps/scripts/overviewer.log
overviewer.py --verbose --quiet --processes 1 --config=/home/mcmaps/servers/bukkitpvt/overviewer.conf >>/home/mcmaps/scripts/overviewer.log
echo ------------------------- >>/home/mcmaps/scripts/overviewer.log
echo Overviewer Complete >>/home/mcmaps/scripts/overviewer.log
echo ============================================ >>/home/mcmaps/scripts/overviewer.log
An example of the log it creates:
============================================
Overviewer Starting...
Mon Sep 16 13:57:52 PDT 2013
-------------------------
overviewer.py:290 12313 2013-09-16 13:57:52 INFO Welcome to Minecraft Overviewer!
observer.py:105 12313 2013-09-16 13:59:37 INFO Rendered 0 of 0. 100% complete
observer.py:97 12313 2013-09-16 13:59:37 INFO Rendered 0 of 0. 100% complete
-------------------------
Overviewer Complete
============================================
Overviewer Starting...
Mon Sep 16 14:10:45 PDT 2013
-------------------------
overviewer.py:290 13681 2013-09-16 14:10:45 INFO Welcome to Minecraft Overviewer!
observer.py:105 13681 2013-09-16 14:12:31 INFO Rendered 0 of 1808. 0% complete
observer.py:105 13681 2013-09-16 14:12:33 INFO Rendered 11 of 1808. 0% complete
observer.py:105 13681 2013-09-16 14:12:35 INFO Rendered 22 of 1808. 1% complete
observer.py:105 13681 2013-09-16 14:12:36 INFO Rendered 33 of 1808. 1% complete
observer.py:105 13681 2013-09-16 14:12:37 INFO Rendered 44 of 1808. 2% complete
observer.py:105 13681 2013-09-16 14:12:38 INFO Rendered 55 of 1808. 3% complete
observer.py:105 13681 2013-09-16 14:12:39 INFO Rendered 66 of 1808. 3% complete
observer.py:105 13681 2013-09-16 14:12:40 INFO Rendered 77 of 1808. 4% complete
observer.py:105 13681 2013-09-16 14:12:42 INFO Rendered 88 of 1808. 4% complete
observer.py:105 13681 2013-09-16 14:12:43 INFO Rendered 99 of 1808. 5% complete
observer.py:105 13681 2013-09-16 14:12:48 INFO Rendered 150 of 1808. 8% complete
observer.py:105 13681 2013-09-16 14:12:57 INFO Rendered 201 of 1808. 11% complete
observer.py:105 13681 2013-09-16 14:13:08 INFO Rendered 252 of 1808. 13% complete
observer.py:105 13681 2013-09-16 14:13:17 INFO Rendered 303 of 1808. 16% complete
observer.py:105 13681 2013-09-16 14:13:27 INFO Rendered 354 of 1808. 19% complete
observer.py:105 13681 2013-09-16 14:13:40 INFO Rendered 405 of 1808. 22% complete
observer.py:105 13681 2013-09-16 14:13:49 INFO Rendered 456 of 1808. 25% complete
observer.py:105 13681 2013-09-16 14:14:07 INFO Rendered 557 of 1808. 30% complete
observer.py:105 13681 2013-09-16 14:14:26 INFO Rendered 658 of 1808. 36% complete
observer.py:105 13681 2013-09-16 14:14:45 INFO Rendered 759 of 1808. 41% complete
observer.py:105 13681 2013-09-16 14:15:07 INFO Rendered 860 of 1808. 47% complete
observer.py:105 13681 2013-09-16 14:15:24 INFO Rendered 961 of 1808. 53% complete
observer.py:105 13681 2013-09-16 14:15:43 INFO Rendered 1062 of 1808. 58% complete
observer.py:105 13681 2013-09-16 14:16:05 INFO Rendered 1163 of 1808. 64% complete
observer.py:105 13681 2013-09-16 14:16:24 INFO Rendered 1264 of 1808. 69% complete
observer.py:105 13681 2013-09-16 14:16:43 INFO Rendered 1365 of 1808. 75% complete
observer.py:105 13681 2013-09-16 14:17:04 INFO Rendered 1466 of 1808. 81% complete
observer.py:105 13681 2013-09-16 14:17:33 INFO Rendered 1567 of 1808. 86% complete
observer.py:105 13681 2013-09-16 14:17:59 INFO Rendered 1668 of 1808. 92% complete
observer.py:105 13681 2013-09-16 14:18:21 INFO Rendered 1769 of 1808. 97% complete
observer.py:97 13681 2013-09-16 14:18:28 INFO Rendered 1808 of 1808. 100% complete
-------------------------
Overviewer Complete
============================================
What I see if I run it again and then view "top" in another terminal session.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
15228 mcmaps 20 0 115m 33m 5080 R 100.1 0.9 0:06.94 overviewer.py
But the PID changes each time I run the script. So how would I know what to store (PID wise) in "/var/run/processname.pid" and how would I know what to look for?
Totally lost. Sorry. 
That's why you save it in a PID file, so you know what to check.
#!/bin/bash
if [ -f /tmp/overviewer.pid ] && read PID < /tmp/overviewer.pid && ps "$PID" >/dev/null 2>/dev/null
then
echo "Overviewer already running as PID $PID" >&2
exit 1
fi
# Automatically delete PID file on exit
trap "rm /tmp/overviewer.pid" EXIT
# Automatically append stdout to this file
exec 1>>/home/mcmaps/scripts/overviewer.log
echo Overviewer Starting...
date
echo -------------------------
# Run this in the background with &
overviewer.py --verbose --quiet --processes 1 --config=/home/mcmaps/servers/bukkitpvt/overviewer.conf &
echo "$!" > /tmp/overviewer.pid
wait
echo -------------------------
echo Overviewer Complete
echo ============================================
You improved my Linux life immensely today with one character: &
Thank you for that lesson! Awesome.
So, in the interest of trying to learn from you and not just steal the work you've done...
I'm confused by the sequence of things here. The "trap" command (which I've never used) is early in the script. That doesn't delete stuff before we can use it? And does this end the script if overviewer is already in progress or wait for one instance to end and then fires off a new one immediately. I'm confused by the "wait" command.
Sorry for being such a newbie.
---------- Post updated at 02:58 PM ---------- Previous update was at 02:57 PM ----------
Oh the wait probably has to do with a background process... something I've not done before in my scripts. I think I'm catching up now.
I put it in a trap so that, if you ctrl-C this or it gets KILL-ed, it still gets rid of /tmp/overviewer.pid
The trap command is meant to respond to software interrupts ( giving it a chance to respond to ctrl-C or a KILL command, for instance), and also has the special EXIT condition for "code to run when the program quits for whatever reason".
Yes, see the exit 1.
It waits for all background process to finish before continuing.
OK... Starting to understand more. And now I have to go read MAN pages for each of these. I try to do that each time I learn from others on this forum.
So, can I still pipe (>>) after a process run in background?
i.e.
overviewer.py --verbose --quiet --processes 1 --config=/home/mcmaps/servers/bukkitpvt/overviewer.conf & >>/home/mcmaps/scripts/overviewer.log
Or wait... you did something with:
# Automatically append stdout to this file
exec 1>>/home/mcmaps/scripts/overviewer.log
So that captures everything?
That is not a pipe, that's a file redirection. | is a pipe.
And yes,
# Automatically append stdout to this file
exec 1>>/home/mcmaps/scripts/overviewer.log
That opens the file once in advance and attaches it to stdout, so you don't need to reopen-append the same file 17 times in a row later.
Anything you run inherits the same set of open files as the shell you're running. This is why they all print to the same screen. Change the shell's own "screen" (i.e. standard output, stdout, or file descriptor #1) into a file, and they'll all write to that file.
Wow... Thank you so much. I have a ton of reading to do now to benefit from all that you have taught me today.
Two comments
1.
In order to recover from a system crash the pidfile should reside in a file system that is emty or emptied at system boot.
2.
Collecting output from a background process can become garbled, especially if mixed with another stream like the foreground echo at the end.
The program logic doesn't just blindly check if the file exists, it checks if it exists and is valid.
It will not become garbled here due to printing before the process exists and after it quits and nowhere inbetween.
a different process can run with that PID...
Maybe the ps $PID is safe enough on earth but not on a mars mission.
Hi everyone,
So the thread so far has left me confused. So I kept digging. If found some code and modified it to this:
if ps -u mcmaps | grep 'overviewer.py'
then
echo Running
else
echo Not Running
fi
So far, this works. The "overviewer.py" is only run from the mcmaps user. And when it is currently running, the script above reports "Running." When it's not running, it accurately states, "Not Running" (so far).
So, would this work? What are the dangers of this?
Here is what I would do for the final script (if people say it would be safe enough):
if ps -u mcmaps | grep 'overviewer.py'
then
echo overviewer.py is already running.
else
overviewer.py --verbose --quiet --processes 1 --config=/home/mcmaps/servers/bukkitpvt/overviewer.conf
fi
The idea here is that I run the script as a cron job every 30 minutes. And most of the time, it completes in 10 minutes or less. But if there were a lot of changes for the overviewer.py to work on, the script could take 30+ minutes. In which case, I want the script to exit out and wait for the next scheduled cron job event to run rather than trying to run two instances over overviewer.py.
Am I on the right track? Is this way of doing it flawed?
That's another way to do it.
The ps command lists many processes, but with -u mcmaps the overhead is small.
There might be the problem that ps cuts the overviewer.py to say overview . Run the ps command on the command line to find out!
The dangers of not using a proper PID file are those of accidentally creating more than one instance by running one check shortly after another.