Avoid Hardcoding IDs in Your Tests

2020-06-11 Between a rock and a hardcoded place

This is a test anti-pattern I’ve seen creep in on many Django test suites. I know of several large projects where it became a major undertaking to undo it. The good news is it’s easy to avoid adding it when you first write your tests. 🙂

Take this test:

from http import HTTPStatus

from django.test import TestCase

from example.core.models import Book


class BookViewTests(TestCase):
    def test_view(self):
        book = Book.objects.create(title="Meditations")

        response = self.client.get("/books/1/")

        self.assertEqual(response.status_code, HTTPStatus.OK)
        self.assertContains(response.content, book.title)

It creates a book object in the database, retrieves the associated page, and checks the response looks correct.

In all likelihood this test will pass, at least initially. The test database should be empty at the start, the database will assign the new book ID 1, and when the test fetches the URL “/books/1/” the view finds the correct book.

Unfortunately, many potential changes to the project will break the ID = 1 assumption. For example, if you create some “default books” in a post_migrate signal handler (as I covered previously), the book in the test will be given a different ID. Also, whilst databases tend to be predictable, they often don’t guarantee that future versions or configuration changes won’t ever affect ID generation.

The Fix

The fix is relatively straightforward: stop hardcoding the ID in the URL, and instead template it. Python 3.6’s f-strings make this fluent. The above test would only need changing where it fetches the response:

response = self.client.get(f"/books/{book.id}/")

This change is straightforward, and it clarifies where the number in the URL comes from.

Other Forms

This problem is also not specific to Django, Models, or AutoFields. It can appear anywhere that you use a data generator in your tests, when you predict its output unnecessarily. For example:

other fields’ defaults.
other database libraries such as SQL Alchemy.
other kinds of data store.

Keep an eye out for it.

Preventative Measures

You can guard against this problem by making your AutoField IDs less predictable, each starting at a higher number than 1. Using a different, large offset for each table can also prevent bugs where you mix up IDs between models, or mix up list indices and IDs.

I first saw this recommendation in the Phabricator team’s blog post Things You Should Do Now. Anders Hovmöller has also covered it Django in Intermittent tests: aligned primary keys.

You can change a table’s ID offset with an ALTER query. The SQL is database specific:

On MariaDB/MySQL, it’s ALTER TABLE ... AUTO_INCREMENT=n.
On PostgreSQL, it’s ALTER SEQUENCE ... RESTART WITH n. You’ll need to figure out the sequence names for the tables though.
On SQLite, I’m not sure - it looks a bit hairy. The best source I found was this Stack Overflow post which covers changing some internal tables.
On Oracle, you’re on your own. I haven’t even researched it :)

If you want to do this for tests only, you can do it in a custom test runner. There you can override Django’s default test database creation and add extra behaviour. For example, for MariaDB/MySQL, and presuming you only have one database:

# example/test/runner.py
from random import randint

from django.db import connection
from django.test.runner import DiscoverRunner


class ExampleTestRunner(DiscoverRunner):
    def setup_databases(self, **kwargs):
        databases = super().setup_databases(**kwargs)
        with connection.cursor() as cursor:
            table_names = connection.introspection.django_table_names(
                only_existing=True, include_views=False
            )
            for table_name in table_names:
                quoted_name = connection.ops.quote_name(table_name)
                start = randint(1_000, 10_000)
                cursor.execute(f"ALTER TABLE {quoted_name} AUTO_INCREMENT={start}")
        return databases

You can use this runner by setting the TEST_RUNNER setting:

TEST_RUNNER = "example.test.runner.ExampleTestRunner"

Notes on the implementation:

It uses the undocumented introspection.django_table_names() method to get the list of tables. Although a simple method, this might change between Django versions (tested on version 3.0).
For each table, a random offset is generated with randint(). This is then used in an ALTER TABLE ... AUTO_INCREMENT= query. The query doesn’t use the normal SQL escaping because the table name is already escaped with quote_name(), and start will only be an integer.

Fin

I hope this helps you avoid and prevent this common test issue,

—Adam

Read my book Boost Your Git DX to Git better.

One summary email a week, no spam, I pinky promise.

Related posts:

Tags: django, python