Django: Defer a model field by default
Some models have one or a few large fields that dominate their per-instance size. For example, take a minimal blog post model:
from django.db import models
class Post(models.Model):
blog = models.ForeignKey("Blog", on_delete=models.CASCADE)
title = models.TextField()
body = models.TextField()
body
is typically many times larger than the rest of the Post
. It can be a good optimization to defer()
such fields when not required:
def index(request):
posts = Post.objects.defer("body")
...
Deferred fields are not fetched in the main query, but will be lazily loaded upon access. Deferring large fields can noticeably reduce data transfer, and thus query time, memory usage, and total page load time. This comes with the risk that you accidentally lazy-load the deferred field, which done in a loop leads to N+1 queries.
When most usage of a model does not require the field, you might want to defer a field by default. Then you don’t need to sprinkle .defer(...)
calls everywhere, and can instead use .defer(None)
in the few sites where the field is used.
Defer by default with a custom base manager
To defer fields by default, follow these steps:
- Create a manager class that makes the appropriate
defer()
call in itsget_queryset()
method. - Attach the manager to the model, ideally as
objects
. - Make the manager the Model’s base manager by naming it in
Meta.base_manager_name
.
(This manager class should not apply any filtering, as noted in the base manager documentation).
For example, to adapt the above Post
model to defer body
:
from django.db import models
class PostManager(models.Manager):
def get_queryset(self):
return super().get_queryset().defer("body")
class Post(models.Model):
blog = models.ForeignKey("Blog", on_delete=models.CASCADE)
title = models.TextField()
body = models.TextField()
objects = PostManager()
class Meta:
base_manager_name = "objects"
Situations where the defer()
applies
The field will then be deferred in most situations. We can check this by testing whether the field is in an instance’s __dict__
attribute.
Let’s look at some examples, which also use these referring models:
class Blog(models.Model):
title = models.TextField()
class Banner(models.Model):
post = models.ForeignKey(Post, on_delete=models.CASCADE)
text = models.TextField()
The examples:
Fetching a
Post
directly:In [1]: from example.models import * In [2]: post = Post.objects.earliest("id") In [3]: "body" in post.__dict__ Out[3]: False
Fetching a post through a related manager:
In [4]: blog = Blog.objects.earliest("id") In [5]: post = blog.post_set.earliest("id") In [6]: "body" in post.__dict__ Out[6]: False
Accessing a post through a prefetched related manager:
In [7]: blog = Blog.objects.prefetch_related("post_set").earliest("id") In [8]: post = blog.post_set.all()[0] In [9]: "body" in post.__dict__ Out[9]: False
Lazy-loading a post through a foreign key:
In [10]: banner = Banner.objects.earliest("id") In [11]: post = banner.post In [12]: "body" in post.__dict__ Out[12]: False
Accessing a post through a prefetched foreign key:
In [13]: banner = Banner.objects.prefetch_related("post").earliest("id") In [14]: post = banner.post In [15]: "body" in post.__dict__ Out[15]: False
Fin
Thanks to Pascal Fouque at my client Silvr for asking me to look into doing this.
May you defer fields but not your dishes,
—Adam
Newly updated: my book Boost Your Django DX now covers Django 5.0 and Python 3.12.
One summary email a week, no spam, I pinky promise.
Related posts:
- Django and the N+1 Queries Problem
- Django: Flush out test flakiness by randomly ordering
QuerySet
s - Django: Avoid database queries in template context processors
Tags: django