I’ve been working quite a bit lately on streamlining Lincoln Loop’s standard deployment systems. One thorn we’ve always had is how to handle application configuration.
In the past, we would have our configuration management system write the configuration out to a JSON file at a known location on the filesystem. The application would read the JSON and set the necessary variables accordingly. This accomplished a few goals:
- Deployments didn’t require any sort of modification to the code from the upstream repository.
- Production secrets could be encrypted and stored safely away from the code.
- Unlike environment variables, the data could have proper types (booleans, lists, etc.)
- It avoided using environment variables altogether, which can be problematic from a security perspective in many scenarios.
That being said, the setup had some downfalls as well.
- The configuration variables needed were not well documented. You had to read through the code to understand how and where they were used.
- We were maintaining a separate Django settings module for local and deployed environments since the file wasn’t used locally.
- The configuration file was not friendly for other services like Heroku or Docker where environment variables are typically used.
Basically, I want to use our Python projects like I would expect to use any other software. Install the software, create a configuration file, and run.
Photo by Jonathan Farber on Unsplash
Introducing Goodconf
I wrote Goodconf (with lots of help from Chris Beaven) to solve some of our issues in a reusable library.
Goodconf is inspired by derpconf and logan which are spun out of Thumbor and Sentry respectively.
With Goodconf, you create a class which defines all the configuration values your application expects to receive. They can have default values and help text which can be used to generate documentation.
from goodconf import GoodConf, Value
class MyConf(GoodConf):
"Configuration for My App"
DEBUG = Value(default=False, help="Toggle debugging.")
DATABASE_URL = Value(
default='postgres://localhost:5432/mydb',
help="Database connection.")
SECRET_KEY = Value(
initial=lambda: base64.b64encode(os.urandom(60)).decode(),
help="Used for cryptographic signing. "
"https://docs.djangoproject.com/en/2.0/ref/settings/#secret-key")
You can then instantiate the class like so:
config = MyConf(
file_env_var="MYAPP_CONF",
default_files=["/etc/myapp/myapp.yml", "myapp.yml"])
This lets you define the default location for configuration files (/etc/myapp/myapp.yml or $(pwd)/myapp.yml if that doesn’t exist) and also an environment variable that can be used to load a file from a different location.
There is also a Django helper which lets you do manage.py --config=/path/to/config.yml ....
With the configuration defined, you simply need to load it and start using it. Loading the config will try to read from the configuration files and fallback on environment variables (cast to the proper Python type) if none are found.
config.load()
SECRET_KEY = config.SECRET_KEY
...
My favorite feature of Goodconf is the output you can generate from a config class:
config.generate_yaml()boilerplate YAML config with help text as commentsconfig.generate_json()boilerplate JSON configconfig.generate_markdown()Documentation in Markdown format
This lets you quickly generate a config file for local development and documentation to drop into a README.md.
Using Goodconf
Goodconf’s README has some examples of how to use the library, but if you’re like me, it’s easier to see an example. Check out our saltdash repo for an example of using Goodconf in a Django project.
saltdash/__init__.pydefines the configurationsaltdash/settings/__init__.pyshows how to use it to define Django settingssetup.cfg(under[options.entry_points]) shows how to install scripts for the management command and configuration generator
Try it Out
We’re using Goodconf in production now and happy with the results. We hope you’ll try it out on your own projects. If you do, please give us feedback here or in GitHub.
Photo by Jonathan Farber on Unsplash
