Weeknotes 2021 Week 11

Releasing Django 3.2rc1

As part of Django 3.2 autocomplete_fields now supports custom to_field and limit_choices_to when searching a related model.

This week we had one release blocker left to fix on this, which came up for multi-table inheritance cases. It all looked nice and smooth, but more complex cases kept throwing up errors.

The question was, for all the weird and wonderful ways you can construct model relationships, how do we resolve the correct lookup attribute on the target model?

Don’t quite have the energy to write this up in all its gory-detail but, if you’re following a reverse relationship you get a related descriptor. From there you can access the remote field, and the related model.

What should work (i.e., roughly what we did have) is getting the attribute name of the pk on the related model:

remote_model._meta.pk.name

But for more complex cases this isn’t correct. For MTI, or where the pk is itself a foreign key, this result isn’t correct.

For example, you end up resolve the related object itself, rather than the lookup value, so you end up with JSON like {"id": "<Band object: 1>"} which isn't quite what we’re after. Alternatively, you try with pk.attname, and you get field resolution errors in the MTI cases.

Ultimately, in some ORM Dark Magic, you have to get the attribute name from the field on the remote model itself, in a little two-step:

remote_model_opts = self.field.remote_field.model._meta
to_field_name = getattr(
    self.field.remote_field, 
    'field_name', 
    remote_model_opts.pk.attname
)
to_field_name = remote_model_opts.get_field(to_field_name).attname

Having got the failing tests in place, we needed Mariusz’ expertise to pin this down nicely in the end. (If you look closely in the PR history, the mid-way point wasn’t as nice.)

I said the fix was Dark Magic. Mariusz said that after six years of looking at ORM tickets it wasn’t. On any reasonable standard, I think that means it is. 😀

This ticket took far longer than I’d imagined. That’s OK, I learnt a lot. Apart from the ORM trickery here, the Admin itself is monstrously complex at points. You can spend a lot of time looking at code without really getting much clearer on whether or why it should work. Ultimately write the tests, see what happens, put breakpoints in telling places, repeat.

In the end it all passed, and we released 3.2rc1 on Thursday, on schedule.

Do give it a run, especially if you’re using autocomplete_fields with tricky model structures.

We don’t really, but let’s pretend, or maybe we do… — Mariusz and I keep a mental list of folks reporting issues on the X.Y.0 straight after release. We’ll know if you didn’t test it. 😀