Building software in most languages is a pain. Remember ant build.xml, maven2 pom files, and multi-level makefiles?
Python has a simple solution for building modules, applications, and extensions called distutils. Disutils comes as part of the Python distribution so there are no other packages required.
Pull down just about any python source code and you�re more than likely going to find a setup.py script that helps make building and installing a snap. Most engineers don�t add functionality when using distutils, instead opting to use the default commands.
In some cases, developers might provide secondary scripts to do other tasks for building and testing outside of the setup script, but I believe that can lead to unnecessary complication of common tasks.
For those who are not familiar with setup scripts, Figure 1 shows a simple example.
#!/usr/bin/env python"""Setup script."""from distutils.core import setupsetup(name = "myapp", version = "1.0.0", description = "My simple application", long_description = "My simple application that doesn't do anything.", author = "Me", author_email = 'me@example.dom', url = "http://example.dom/myapp/", download_url = "http://example.dom/myapp/download/", platforms = ['any'], license = "GPLv3+", package_dir = {'myapp': 'src/myapp'}, packages = ['myapp'], classifiers = [ 'License :: OSI Approved :: GNU General Public License (GPL)', 'Development Status :: 5 - Production/Stable', 'Topic :: Software Development :: Libraries :: Python Modules', 'Programming Language :: Python'],)Figure 1.
This setup script lists the metadata, maps to a package directory, and provides the general commands expected from distutils setup. This is great, but as I�ve said, it may not do everything you need it to.
What if, for example, we want to be able to see all the TODO tags in the codebase on any platform? In other words, grep won�t cut it.
Let�s start by writing a function that searches for TODO tags in files and prints them back to the screen in a nice format like:
src/myapp/__init__.py (11): TODO: remove me
def report_todo(): """ Prints out TODO's in the code. """ import os import re # The format of the string to print: file_path (line_no): %s line_str format_str = "%s (%i): %s" # regex to remove whitespace in front of TODO's remove_front_whitespace = re.compile("^[ ]*(.*)$") # Look at all non pyc files from current directory down for rootdir in ['src/', 'bin/']: # walk down each root directory for root, dirs, files in os.walk(rootdir): # for each single file in the files for afile in files: # if the file doesn't end with .pyc if not afile.endswith('.pyc'): full_path = os.path.join(root, afile) fobj = open(full_path, 'r') line_no = 0 # look at each line for TODO's for line in fobj.readlines(): if 'todo' in line.lower(): nice_line = remove_front_whitespace.match( line).group(1) # print the info if we have a TODO print(format_str % ( full_path, line_no, nice_line)) line_no += 1Figure 2.
The report_todo function is self-contained and quite simple, if a bit clunky by itself. Who wants to install and use a �report-todo� command on machines just to look for TODO tags?
We want to turn this into a setup command so that we can ship it with our application �myapp� to be used by developers via setup.py. We will probably add more setup commands in the future, so let�s create a class to subclass our commands from. We do this because most of our commands will probably be simple and won�t need to override the *_option methods (though they can if they need to).
import osfrom distutils.core import setup, Commandclass SetupBuildCommand(Command): """ Master setup build command to subclass from. """ user_options = [] def initialize_options(self): """ Setup the current dir. """ self._dir = os.getcwd() def finalize_options(self): """ Set final values for all the options that this command supports. """ passFigure 3.
The SetupBuildCommand defines a few methods that are required for all setup commands. As I stated before, most of our commands will probably not deviate from these defaults, so defining them higher in the object hierarchy means simpler code in the commands themselves. If we do want to add arguments to to any of our commands, we can override initialize_options() and finalize_options() to handle the arguments properly.
Now that we have the SetupBuildCommand to subclass from we can merge our report_todo function into a SetupBuildCommand class. To do this, we create a class that subclasses SetupBuildCommand, and then add a description variable and a run method (which is what is executed when the command runs).
class TODOCommand(SetupBuildCommand): """ Quick command to show code TODO's. """ description = "prints out TODO's in the code" def run(self): """ Prints out TODO's in the code. """ import re # The format of the string to print: file_path (line_no): %s line_str format_str = "%s (%i): %s" # regex to remove whitespace in front of TODO's remove_front_whitespace = re.compile("^[ ]*(.*)$") # Look at all non pyc files in src/ and bin/ for rootdir in ['src/', 'bin/']: # walk down each root directory for root, dirs, files in os.walk(rootdir): # for each single file in the files for afile in files: # if the file doesn't end with .pyc if not afile.endswith('.pyc'): full_path = os.path.join(root, afile) fobj = open(full_path, 'r') line_no = 0 # look at each line for TODO's for line in fobj.readlines(): if 'todo' in line.lower(): nice_line = remove_front_whitespace.match( line).group(1) # print the info if we have a TODO print(format_str % ( full_path, line_no, nice_line)) line_no += 1Figure 4.
The run method in this example is the same as report_todo but in a method format (with self) and renamed to run.
Now it�s time to bring all this together. If we add Figure 1 and Figure 2 to the setup script (Figure 1), then all that is left is to map the setup command with a command name. Do this via the cmdclass argument, and in the form:
cmdclass = {'name': CommandClass}
Figure 5 shows it all together in an abbreviated form.
#!/usr/bin/env python"""Setup script."""import osfrom distutils.core import setup, Commandclass SetupBuildCommand(Command): """ Master setup build command to subclass from. """ # See Figure 3class TODOCommand(SetupBuildCommand): """ Quick command to show code TODO's. """ # See Figure 4setup(name = "myapp", version = "1.0.0", description = "My simple application", long_description = "My simple application that doesn't do anything.", author = "Me", author_email = 'me@example.dom', url = "http://example.dom/myapp/", download_url = "http://example.dom/myapp/download/", platforms = ['any'], license = "GPLv3+", package_dir = {'myapp': 'src/myapp'}, packages = ['myapp'], classifiers = [ 'License :: OSI Approved :: GNU General Public License (GPL)', 'Development Status :: 5 - Production/Stable', 'Topic :: Software Development :: Libraries :: Python Modules', 'Programming Language :: Python'], cmdclass = {'todo': TODOCommand},)Figure 5.
You�ll notice that `python setup.py �help-commands` now shows �Extra commands� with our TODO command listed.
Give it a go and see what you have left to do in your code!
