How to Write an Installable Django App

In the Django framework, a project refers to the collection of configuration files and code for a particular website. Django groups business logic into what it calls apps, which are the modules of the Django framework. There’s plenty of documentation on how to structure your projects and the apps within them, but when it comes time to package an installable Django app, information is harder to find.

In this tutorial, you’ll learn how to take an app out of a Django project and package it so that it’s installable. Once you’ve packaged your app, you can share it on PyPI so that others can fetch it through pip install.

In this tutorial, you’ll learn:

What the differences are between writing stand-alone apps and writing apps inside of projects
How to create a setup.cfg file for publishing your Django app
How to bootstrap Django outside of a Django project so you can test your app
How to test across multiple versions of Python and Django using tox
How to publish your installable Django app to PyPI using Twine

Be sure to follow along with the examples by downloading the source code at the link below:

Download the sample code: Click here to get the code you’ll use to learn how to write an installable Django app in this tutorial.

Prerequisites

This tutorial requires some familiarity with Django, pip, PyPI, pyenv (or an equivalent virtual environment tool), and tox. To learn more about these topics, check out:

Remove ads

Starting a Sample Django App in a Project

This tutorial includes a working package to help guide you through the process of making an installable Django app. You can download the source code at the link below:

Download the sample code: Click here to get the code you’ll use to learn how to write an installable Django app in this tutorial.

Even if you originally intend to make your Django app available as a package, you’re likely to start inside a project. To demonstrate the process of moving from Django project to installable Django app, I’ve made two branches available in the repo. The project branch is the starting state of an app inside of a Django project. The master branch is the finished installable app.

You can also download the finished app at the PyPI realpython-django-receipts package page. You can install the package by running pip install realpython-django-receipts.

The sample app is a short representation of the line items on a receipt. In the project branch, you’ll find a directory named sample_project that contains a working Django project. The directory looks like this:

sample_project/
│
├── receipts/
│   ├── fixtures/
│   │   └── receipts.json
│   │
│   ├── migrations/
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   │
│   ├── __init__.py
│   ├── admin.py
│   ├── apps.py
│   ├── models.py
│   ├── tests.py
│   ├── urls.py
│   └── views.py
│
├── sample_project/
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
│
├── db.sqlite3
├── manage.py
├── resetdb.sh
└── runserver.sh

The most current version of Django at the time this tutorial was written was 3.0.4, and all testing was done with Python 3.7. None of the steps outlined in this tutorial should be incompatible with earlier versions of Django—I’ve used these techniques since Django 1.8. However, some changes are necessary if you’re using Python 2. To keep the examples simple, I’ve assumed Python 3.7 across the code base.

Creating the Django Project From Scratch

The sample project and receipts app were created using the Django admin command and some small edits. To start, run the following code inside of a clean virtual environment:

Shell
      
$ python -m pip install Django
$ django-admin startproject sample_project
$ cd sample_project
$ ./manage.py startapp receipts

This creates the sample_project project directory structure and a receipts app subdirectory with template files that you’ll use to create your installable Django app.

Next, the sample_project/settings.py file needs a few modifications:

Add '127.0.0.1' to the ALLOWED_HOSTS setting so you can test locally.
Add 'receipts' to the INSTALLED_APPS list.

You’ll also need to register the receipts app’s URLs in the sample_project/urls.py file. To do so, add path('receipts/', include('receipts.urls')) to the url_patterns list.

Exploring the Receipts Sample App

The app consists of two ORM model classes: Item and Receipt. The Item class contains database field declarations for a description and a cost. The cost is contained in a DecimalField. Using floating-point numbers to represent money is dangerous—you should always use fixed-point numbers when dealing with currencies.

The Receipt class is a collection point for Item objects. This is achieved with a ForeignKey on Item that points to Receipt. Receipt also includes total() for getting the total cost of Item objects contained in the Receipt:

Python
      
    
# receipts/models.py
from decimal import Decimal
from django.db import models

class Receipt(models.Model):
    created = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return f"Receipt(id={self.id})"

    def total(self) -> Decimal:
        return sum(item.cost for item in self.item_set.all())

class Item(models.Model):
    created = models.DateTimeField(auto_now_add=True)

    description = models.TextField()
    cost = models.DecimalField(max_digits=7, decimal_places=2)
    receipt = models.ForeignKey(Receipt, on_delete=models.CASCADE)

    def __str__(self):
        return f"Item(id={self.id}, description={self.description}, " \
            f"cost={self.cost})"

The model objects give you content for the database. A short Django view returns a JSON dictionary with all the Receipt objects and their Item objects in the database:

Python
      
    
# receipts/views.py
from django.http import JsonResponse
from receipts.models import Receipt

def receipt_json(request):
    results = {
        "receipts":[],
    }

    for receipt in Receipt.objects.all():
        line = [str(receipt), []]
        for item in receipt.item_set.all():
            line[1].append(str(item))

        results["receipts"].append(line)

    return JsonResponse(results)

The receipt_json() view iterates over all the Receipt objects, creating a pair of the Receipt objects and a list of the Item objects contained within. All of this is put in a dictionary and returned through Django’s JsonResponse().

To make the models available in the Django admin interface, you use an admin.py file to register the models:

Python
      
    
# receipts/admin.py
from django.contrib import admin

from receipts.models import Receipt, Item

@admin.register(Receipt)
class ReceiptAdmin(admin.ModelAdmin):
    pass

@admin.register(Item)
class ItemAdmin(admin.ModelAdmin):
    pass

This code creates a Django ModelAdmin for each of the Receipt and Item classes and registers them with the Django admin.

Finally, a urls.py file registers a single view in the app against a URL:

Python
      
    
# receipts/urls.py
from django.urls import path

from receipts import views

urlpatterns = [
    path("receipt_json/", views.receipt_json),
]

You can now include receipts/urls.py in your project’s url.py file to make the receipt view available on your website.

With everything in place, you can run ./manage.py makemigrations receipts, use the Django admin to add data, and then visit /receipts/receipt_json/ to view the results:

Shell
      
        
      
    
$ curl -sS http://127.0.0.1:8000/receipts/receipt_json/ | python3.8 -m json.tool
{
    "receipts": [
        [
            "Receipt(id=1)",
            [
                "Item(id=1, description=wine, cost=15.25)",
                "Item(id=2, description=pasta, cost=22.30)"
            ]
        ],
        [
            "Receipt(id=2)",
            [
                "Item(id=3, description=beer, cost=8.50)",
                "Item(id=4, description=pizza, cost=12.80)"
            ]
        ]
    ]
}

In the block above, you use curl to visit the receipt_json view, resulting in a JSON response containing the Receipt objects and their corresponding Item objects.

Remove ads

Testing the App in the Project

Django augments the Python unittest package with its own testing capabilities, enabling you to preload fixtures into the database and run your tests. The receipts app defines a tests.py file and a fixture to test with. This test is by no means comprehensive, but it’s a good enough proof of concept:

Python
      
    
# receipts/tests.py
from decimal import Decimal
from django.test import TestCase
from receipts.models import Receipt

class ReceiptTest(TestCase):
    fixtures = ["receipts.json", ]

    def test_receipt(self):
        receipt = Receipt.objects.get(id=1)
        total = receipt.total()

        expected = Decimal("37.55")
        self.assertEqual(expected, total)

The fixture creates two Receipt objects and four corresponding Item objects. Click on the collapsible section below for a closer look at the code for the fixture.

A Django test fixture is a serialization of the objects in the database. The following JSON code creates Receipt and Item objects for testing:

JSON
      
    
[
{
    "model": "receipts.receipt",
    "pk": 1,
    "fields": {
        "created": "2020-03-24T18:16:39.102Z"
    }
},
{
    "model": "receipts.receipt",
    "pk": 2,
    "fields": {
        "created": "2020-03-24T18:16:41.005Z"
    }
},
{
    "model": "receipts.item",
    "pk": 1,
    "fields": {
        "created": "2020-03-24T18:16:59.357Z",
        "description": "wine",
        "cost": "15.25",
        "receipt": 1
    }
},
{
    "model": "receipts.item",
    "pk": 2,
    "fields": {
        "created": "2020-03-24T18:17:25.548Z",
        "description": "pasta",
        "cost": "22.30",
        "receipt": 1
    }
},
{
    "model": "receipts.item",
    "pk": 3,
    "fields": {
        "created": "2020-03-24T18:19:37.359Z",
        "description": "beer",
        "cost": "8.50",
        "receipt": 2
    }
},
{
    "model": "receipts.item",
    "pk": 4,
    "fields": {
        "created": "2020-03-24T18:19:51.475Z",
        "description": "pizza",
        "cost": "12.80",
        "receipt": 2
    }
}
]

The above fixture is referenced in the ReceiptTestCase class and is loaded automatically by the Django test harness.

You can test the receipts app with the Django manage.py command:

Shell
      
        
      
    
$ ./manage.py test
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
.
----------------------------------------------------------------------
Ran 1 test in 0.013s

OK
Destroying test database for alias 'default'...

Running manage.py test runs the single test defined in receipts/tests.py and displays the results.

Making Your Installable Django App

Your goal is to share the receipts app without a project and to make it reusable by others. You could zip up the receipts/ directory and hand it out, but that’s somewhat limiting. Instead, you want to separate the app into a package so it’s installable.

The biggest challenge in creating an installable Django app is that Django expects a project. An app without a project is just a directory containing code. Without a project, Django doesn’t know how to do anything with your code, including running tests.

Moving Your Django App Out of the Project

It’s a good idea to keep a sample project around so you can run the Django dev server and play with a live version of your app. You won’t include this sample project in the app package, but it can still live in your repository. Following this idea, you can get started with packaging your installable Django app by moving it up a directory:

Shell
      
$ mv receipts ..

The directory structure now looks something like this:

django-receipts/
│
├── receipts/
│   ├── fixtures/
│   │   └── receipts.json
│   │
│   ├── migrations/
│   │   ├── 0001_initial.py
│   │   └── __init__.py
│   │
│   ├── __init__.py
│   ├── models.py
│   ├── tests.py
│   ├── urls.py
│   ├── views.py
│   ├── admin.py
│   └── apps.py
│
├── sample_project/
│   ├── sample_project/
│   │   ├── __init__.py
│   │   ├── asgi.py
│   │   ├── settings.py
│   │   ├── urls.py
│   │   └── wsgi.py
│   │
│   ├── db.sqlite3
│   ├── manage.py
│   ├── resetdb.sh
│   └── runserver.sh
│
├── LICENSE
└── README.rst

To package your app, you need to pull it out of the project. Moving it is the first step. I typically keep the original project around to test with, but I don’t include it in the resulting package.

Bootstrapping Django Outside of a Project

Now that your app is outside of a Django project, you need to tell Django how to find it. If you want to test your app, then run a Django shell that can find your app or run your migrations. You’ll need to configure Django and make it available.

Django’s settings.configure() and django.setup() are key to interacting with your app outside of a project. More information on these calls is available in the Django documentation.

You’re likely to need this configuration of Django in several places, so it makes sense to define it in a function. Create a file called boot_django.py containing the following code:

Python
      
    
# boot_django.py
#
# This file sets up and configures Django. It's used by scripts that need to
# execute as if running in a Django server.
import os
import django
from django.conf import settings

BASE_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), "receipts"))

def boot_django():
    settings.configure(
        BASE_DIR=BASE_DIR,
        DEBUG=True,
        DATABASES={
            "default":{
                "ENGINE":"django.db.backends.sqlite3",
                "NAME": os.path.join(BASE_DIR, "db.sqlite3"),
            }
        },
        INSTALLED_APPS=(
            "receipts",
        ),
        TIME_ZONE="UTC",
        USE_TZ=True,
    )
    django.setup()

Lines 12 and 27 set up the Django environment. The settings.configure() call takes a list of arguments that are equivalent to the variables defined in a settings.py file. Anything you would need in your settings.py to make your app run gets passed into settings.configure().

The above code is a fairly stripped-down configuration. The receipts app doesn’t do anything with sessions or templates, so INSTALLED_APPS only needs "receipts", and you can skip over any middleware definitions. The USE_TZ=True value is necessary because the Receipt model contains a created timestamp. Otherwise, you would run into problems loading the test fixture.

Remove ads

Running Management Commands With Your Installable Django App

Now that you have boot_django.py, you can run any Django management command with a very short script:

Python
      
    
#!/usr/bin/env python
# makemigrations.py

from django.core.management import call_command
from boot_django import boot_django

boot_django()
call_command("makemigrations", "receipts")

Django allows you to programmatically call management commands through call_command(). You can now run any management command by importing and calling boot_django() followed by call_command().

Your app is now outside the project, allowing you to do all sorts of Django-y things to it. I often define four utility scripts:

load_tests.py to test your app
makemigrations.py to create migration files
migrate.py to perform table migrations
djangoshell.py to spawn a Django shell that’s aware of your app

Testing Your Installable Django App

The load_test.py file could be as simple as the makemigrations.py script, but then it would only be able to run all the tests at once. With a few additional lines, you can pass command-line arguments to the test runner, allowing you to run selective tests:

Python
      
    
#!/usr/bin/env python
# load_tests.py
import sys
from unittest import TestSuite
from boot_django import boot_django

boot_django()

default_labels = ["receipts.tests", ]

def get_suite(labels=default_labels):
    from django.test.runner import DiscoverRunner
    runner = DiscoverRunner(verbosity=1)
    failures = runner.run_tests(labels)
    if failures:
        sys.exit(failures)

    # In case this is called from setuptools, return a test suite
    return TestSuite()

if __name__ == "__main__":
    labels = default_labels
    if len(sys.argv[1:]) > 0:
        labels = sys.argv[1:]

    get_suite(labels)

Django’s DiscoverRunner is a test discovery class compatible with Python’s unittest. It’s responsible for setting up the test environment, building the suite of tests, setting up the databases, running the tests, and then tearing it all down. Starting on line 11, get_suite() takes a list of test labels and directly calls the DiscoverRunner on them.

This script is similar to what the Django management command test does. The __main__ block passes any command-line arguments to get_suite(), and if there are none, then it passes in the test suite for the app, receipts.tests. You can now call load_tests.py with a test label argument and run a single test.

Line 19 is a special case to help when testing with tox. You’ll learn more about tox in a later section. You can also check out a potential substitute for DiscoverRunner in the collapsible section below.

One of the installable Django apps that I’ve written is django-awl. It’s a loose collection of utilities that I’ve accumulated over my years writing Django projects. Included in the package is an alternative to DiscoverRunner called WRunner.

The key advantage to using WRunner is that it supports wildcard matching of test labels. Passing in a label that begins with an equals sign (=) will match any test suite or method name that contains that label as a substring. For example, the label =rec would match and run the test ReceiptTest.test_receipt() in receipt/tests.py.

Defining Your Installable Package With `setup.cfg`

To put your installable Django app on PyPI, you need to first put it in a package. PyPI expects an egg, wheel, or source distribution. These are built using setuptools. To do this, you need to create a setup.cfg file and a setup.py file at the same directory level as your receipts directory.

Before digging into that, though, you want to make sure you have some documentation. You can include a project description in setup.cfg, which is automatically displayed on the PyPI project page. Make sure to write a README.rst or something similar with information about your package.

PyPI supports the reStructuredText format by default, but it can also handle Markdown with extra parameters:

Config File
      
    
# setup.cfg
[metadata]
name = realpython-django-receipts
version = 1.0.3
description = Sample installable django app
long_description = file:README.rst
url = https://github.com/realpython/django-receipts
license = MIT
classifiers =
    Development Status :: 4 - Beta
    Environment :: Web Environment
    Intended Audience :: Developers
    License :: OSI Approved :: MIT License
    Operating System :: OS Independent
    Programming Language :: Python :: 3 :: Only
    Programming Language :: Python :: 3.7
    Programming Language :: Python :: Implementation :: CPython
    Topic :: Software Development :: Libraries :: Application Frameworks
    Topic :: Software Development :: Libraries :: Python Modules

[options]
include_package_data = true
python_requires = >=3.6
setup_requires =
    setuptools >= 38.3.0
install_requires =
    Django>=2.2

This setup.cfg file describes the package that you’ll build. Line 6 uses the file: directive to read in your README.rst file. This saves you from having to write a long description in two places.

The install_requires entry on line 26 tells any installers, such as pip install, about the dependencies your app has. You will always want to tie your installable Django app to its minimum supported version of Django.

If your code has any dependencies that are only required to run the tests, then you can add a tests_require = entry. For example, before mock became part of the standard Python library, it was common to see tests_require = mock>=2.0.0 in setup.cfg.

It’s considered best practice to include a pyproject.toml file with your package. Brett Cannon’s excellent article on the subject can run you through the details. A pyproject.toml file is also included with the sample code.

You’re almost ready to build the package for your installable Django app. The easiest way to test it is with your sample project— another good reason to keep a sample project around. The pip install command supports locally defined packages. This can be used to make sure that your app still works with a project. However, one caveat is that setup.cfg won’t work on its own in this case. You’ll also have to create a shim version of setup.py:

Python
      
    
#!/usr/bin/env python

if __name__ == "__main__":
    import setuptools
    setuptools.setup()

This script will automatically use your setup.cfg file. You can now install a local editable version of the package to test it from within sample_project. To be doubly sure, it’s best to start with a brand-new virtual environment. Add the following requirements.txt file inside the sample_project directory:

Text

# requirements.txt
-e ../../django-receipts

The -e tells pip that this is a local editable installation. You’re now ready to install:

Shell
      
        
      
    
$ pip install -r requirements.txt
Obtaining django-receipts (from -r requirements.txt (line 1))
Collecting Django>=3.0
  Using cached Django-3.0.4-py3-none-any.whl (7.5 MB)
Collecting asgiref~=3.2
  Using cached asgiref-3.2.7-py2.py3-none-any.whl (19 kB)
Collecting pytz
  Using cached pytz-2019.3-py2.py3-none-any.whl (509 kB)
Collecting sqlparse>=0.2.2
  Using cached sqlparse-0.3.1-py2.py3-none-any.whl (40 kB)
Installing collected packages: asgiref, pytz, sqlparse, Django, realpython-django-receipts
  Running setup.py develop for realpython-django-receipts
Successfully installed Django-3.0.4 asgiref-3.2.7 pytz-2019.3 realpython-django-receipts sqlparse-0.3.1

The install_requires list in setup.cfg tells pip install that it needs Django. Django needs asgiref, pytz, and sqlparse. All the dependencies are taken care of and you should now be able to run your sample_project Django dev server. Congratulations—your app is now packaged and referenced from within the sample project!

Remove ads

Testing Multiple Versions With `tox`

Django and Python are both constantly advancing. If you’re going to share your installable Django app with the world, then you’re probably going to need to test in multiple environments. The tox tool needs a little help to be able to test your Django app. Go ahead and make the following change inside of setup.cfg:

Config File
      
    
# setup.cfg
[metadata]
name = realpython-django-receipts
version = 1.0.3
description = Sample installable django app
long_description = file:README.rst
url = https://github.com/realpython/django-receipts
license = MIT
classifiers =
    Development Status :: 4 - Beta
    Environment :: Web Environment
    Intended Audience :: Developers
    License :: OSI Approved :: MIT License
    Operating System :: OS Independent
    Programming Language :: Python :: 3 :: Only
    Programming Language :: Python :: 3.7
    Programming Language :: Python :: Implementation :: CPython
    Topic :: Software Development :: Libraries :: Application Frameworks
    Topic :: Software Development :: Libraries :: Python Modules

[options]
include_package_data = true
python_requires = >=3.6
setup_requires =
    setuptools >= 38.3.0
install_requires =
    Django>=2.2
test_suite = load_tests.get_suite

Line 28 tells the package manager to use the load_tests.py script to get its test suite. The tox utility uses this to run its tests. Recall get_suite() in load_tests.py:

Python
      
    
# Defined inside load_tests.py
def get_suite(labels=default_labels):
    from django.test.runner import DiscoverRunner
    runner = DiscoverRunner(verbosity=1)
    failures = runner.run_tests(labels)
    if failures:
        sys.exit(failures)

    # If this is called from setuptools, then return a test suite
    return TestSuite()

What is happening here is admittedly a little bit weird. Normally, the test_suite field in setup.cfg points to a method that returns a suite of tests. When tox calls setup.py, it reads the test_suite parameter and runs load_tests.get_suite().

If this call didn’t return a TestSuite object, then tox would complain. The weird part is that you don’t actually want tox to get a suite of tests because tox is not aware of the Django test environment. Instead, get_suite() creates a DiscoverRunner and returns an empty TestSuite object on line 10.

You can’t simply have DiscoverRunner return a suite of tests because you have to call DiscoverRunner.run_tests() for the setup and teardown of the Django test environment to execute correctly. Merely passing the correct tests to tox wouldn’t work because the database wouldn’t be created. get_suite() runs all the tests, but as a side effect of the function call rather than as the normal case of returning a test suite for tox to execute.

The tox tool allows you to test multiple combinations. A tox.ini file determines which combinations of environments to test. Here’s an example:

Config File
      
    
[tox]
envlist = py{36,37}-django220, py{36,37}-django300

[testenv]
deps =
    django220: Django>=2.2,<3
    django300: Django>=3
commands=
    python setup.py test

This file states that tests should be run for Python 3.6 and 3.7 combined with Django 2.2 and 3.0. That’s a total of four test environments. The commands= section is where you tell tox to call the test through setup.py. This is how you invoke the test_suite = load_tests.get_suite hook in setup.cfg.

Note: The test subcommand of setup.py has been deprecated. Packaging in Python is changing quickly at the moment. Although calling python setup.py test isn’t generally recommended, it is what works in this specific situation.

Publishing to PyPI

Finally, it’s time to share your installable Django app on PyPI. There are multiple tools for uploading a package, but you’ll focus on Twine in this tutorial. The following code builds the packages and invokes Twine:

Shell
      
$ python -m pip install -U wheel twine setuptools
$ python setup.py sdist
$ python setup.py bdist_wheel
$ twine upload dist/*

The first two commands build the source and binary distributions of your package. The call to twine uploads to PyPI. If you have a .pypirc file in your home directory, then you can preset your username so the only thing you’re prompted for is your password:

Config File
      
    
[disutils]
index-servers =
    pypi

[pypi]
username: <YOUR_USERNAME>

I often use a small shell script to grep the version number from the code. Then I call git tag to tag the repo with the version number, remove the old build/ and dist/ directories, and call the above three commands.

For more details on using Twine, see How to Publish an Open-Source Python Package to PyPI. Two popular alternatives to Twine are Poetry and Flit. Package management in Python is changing rapidly. PEP 517 and PEP 518 are redefining how to describe Python packages and dependencies.

Conclusion

Django apps rely on the Django project structure, so packaging them separately requires extra steps. You’ve seen how to make an installable Django app by extracting it from a project, packaging it, and sharing it on PyPI. Be sure to download the sample code at the link below:

Download the sample code: Click here to get the code you’ll use to learn how to write an installable Django app in this tutorial.

In this tutorial, you’ve learned how to:

Use the Django framework outside of a project
Call Django management commands on an app that is independent of a project
Write a script that invokes Django tests, optionally using a single test label
Build a setup.py file to define your package
Modify the setup.py script to accommodate tox
Use Twine to upload your installable Django app