Encode configuration files in python

Python is a powerful language for encoding configuration information for a software program. That’s true especially in regard to the dictionary contruct and the ability to nest data structures, which allows to encode complex configuration parameters. Also, storing the configuration as a text file allows for easier debugging and manual editing of the configuration.

For example, the unix passed file can be encoded as a dictionary with the user name as the key. Each entry in the dictionary would be another dictionary with name, uid, etc. as keys; or it could be a tuple with fixed positions for name, uid, etc.

Of course, since python is not designed to be used for encoding configuration, it does not directly provide routines to load and save configuration files written as python scripts. Saving the configuration file is fairly simple as the str method cleanly converts any python data structure to a string that can be directly written to a file.

Note that for a string type configuration parameter, you need to explicitely add quotes while writing to the file. The python code to save a configuration file myconfig.cfg would look as follows:

f = open("/path/to/myconfig.cfg", "w") f.write("some_config_param = ") f.write(str(some_config_param)) f.write("n")

Loading a configuration file written as python data structures and then accessing the configuration information seamlessly is slightly more complex. Using the imp module is one possible way to load the configuration file. The imp module provides two functions: find_module to search for a module using the standard heuristics; and load_module to load the file found by find_module and return a module object. The find_module return values are passed to load_module as parameters. Once the module object is available, one can access the configurtion information through its attributes.

There are couple of issues with using the imp module:

• The file needs to have a .py extension since findmodule searches files with only certain extensions and it guesses the type of the file from the extension. This can be worked around by opening the config file instead of using find_module and passing the open file to load_module.
• The load_module method compiles the file as a .pyc file before importing it. That leaves a unrequired file behind. The compiled file is used by load_module if it is newer than the config file. In case of a race between two threads, one saving the config file and the other one loading it, the compiled file could get a timestamp the same as the config file but with the old contents. Any further load_module calls load from the compiled file and hence, they load the old config data.

The execfile function is a better way to load a configuration file. Again, the execfile method is not intended for loading python data structures. Its primary use is to run an independent piece of python code. But the function allows us to specify the global and local dictionaries as parameters. Also, the effects of the executed code are reflected in the parameters.

The python code to load a configuration file myconfig.cfg would look as follows:

configuration_globals = {} configuration_locals = {} execfile("/path/to/myconfig.cfg", configuration_globals, configuration_locals) some_config_param = configuration_locals["some_config_param"]

Happy programming!