Override

A simple, pure Python configuration tool.

TL;DR: It inserts a templated Python shim file between your application and your config, at 'build time'.

• Usage
  • Pure Python Configs
  • Overriding
  • Template
  • Tips, Tricks & Pitfalls
• API

Summary

Instead of juggling lots of configurations and copying between them, you can:

• Use Python to obtain and modify configurations as normal.
• Inherit configurations using standard Python import behaviour.
• Change configuration parameters at run time using command line or environment variables.
• Leverage import caching - once the import hierarchy is resolved, other code from the process will be using Python's import cache. There are no repetitive function calls involved in 'getting' your config.
• Keep your local/development config .gitignore'd, and it will inherit new parameters from upstream, throughout code changes.
• When you eventually resign yourself to having a different config for testing, it is very clear which parameters have been changed.

The simplest use case looks something like this:

  config_A - config_B
               /
              /
           shim
             |
       application

Usage

Pure Python Configs

Often you need variants on a theme:

# common configuration
a_setting = 1
another = 2
watch_for_file_changes = True

# production configuration
from common import *
watch_for_file_changes = False

Now you simply build neat, ordered hierarchies of configurations using plain old Python.

By using the oft-maligned * syntax we have a transparent cascade of parameters.

By inheriting from, and developing against, a common configuration, you can be (reasonably) sure that your production config isn't missing values provided elsewhere. It's still up to you to set the correct values, of course!

Here's an extreme example of hierarchy, in which each file imports the one above it:

• database, logs, taskrunner # might define some configs in different files to keep them tidy
• common # we define all the things we consider to be configurable
• live # most production environments have different logging settings
• production_A # maybe this environment is special
• built_config # this file is generated by override and selects the production_A config
• config.py # you might want some common interface post-import
• some_code.py # and, at last, you use your config variable

Overriding

Normally you would just run your project, and use the config in your code:

python my_project.py

But you can override your config on the command line:

python my_project.py --override=key.subkey=value

Or using an environment variable:

export override=key.subkey=value
python my_project.py

Template

The shim looks like this (yes, pep8 compliant):

# -*- coding: utf-8 -*-
# This file is auto-generated from settings in:
# C:\Users\david\Documents\coding\override\example\example2.py

from my_configs.one import *
post_import(locals())

from override import RuntimeUpdates as _RTU
_RTU('set').apply_all(locals())

Tips, Tricks & Pitfalls

You may want to consider that:

• Configs are Python and expected to be loaded from a trusted source. You could always, say, load commonly adjusted json/yaml/ini user settings as part of one of your trusted configs, but this isn't the intended use case.
• Diamond inheritance can be painful or impossible. Such a case might be having different test parameters for different environments. You could, however run two configuration shims to allow complete multiplexing across those dimensions.
• 'calculated parameters' should be deferred until the full config tree has been imported. For example, defining fully qualified urls at import time based on a single hostname. To do this, you can either define a post-import function or include another layer in your import hierarchy.
• As a general rule, constants are not configurable and are best kept in a separate file structure. They can then be attached to the base config, imported after the config, or just kept entirely independent.
• The system is built to handle nested configuration structures (e.g. Django or Celery objects). However your config code will also need to handle this correctly: e.g. dictionary key sets and pops instead of re-assignment of references.
• Often you will want to generate other output as part of the configuration write. For example, writing out version information, parameters for dependent builds, or files used by a deployment environment. This can be done directly with the callback provided.

API

Until autodocs happen, you'll just have to read this interface: override/project.py:Project