Why do we need apps?

Isn’t it possible to achieve the same with regular Python modules? We could create a models directory and create a subdirectory for each component, same goes for views, etc. However, this approach allows for code structure to grow organically rather than forcing to split everything into apps from the beginning.

It seems that it’s more about the development process rather than something inherent to apps. Multiple people could still work on a single app and they would still run into migration conflicts if the process is not organized properly.

1 Like

This is the use case for people who are going to distribute their apps in future and using apps in such scenarios makes perfect sense.

Nothing stops you from separating the models or templates by creating subfolders inside models or templates folders. This is essentially the same thing except we’re not introducing any new concepts or force users to adopt a certain approach in the beginning.

I’m not sure if it’s completely true for user-created apps: if I split my code between multiple apps, the code would still import stuff from each other and removing the app from INSTALLED_APPS wouldn’t be enough. My idea is simple: apps concept makes perfect sense as a tool for code you need to distribute, but I can’t see any value from applying this approach for a regular project.

In sort, I think it comes down to - yes, you could use Python modules, but then what do you use as the namespace for templates, models, migrations, and so on?

The full module path is ugly and not stable enough to use in a database table. The “last” path of the path is ill-defined if you put some things in a submodule. And to add on top of that, apps can be installed twice with different aliases and paths, or have their python module moved around as part of a code cleanup but have their alias kept the same.

Apps are basically just a discovery plus alias mechanism for Python modules - a necessary one, given Django’s design. I am unconvinced that using plain Python modules in their place, and the increase in code and increase in mysterious errors that result, is a user-friendly action. Maybe it’s cleaner, in some sense, but I feel that it is less pragmatic.

1 Like

What do you mean? If we use a single app for the whole project then you have this namespace, no?

Could you give a simple example please? I’m not sure I understand.

Indeed, but at some point, you are going to want to use something from someone else (unless you truly want to write everything in house), and at that point, you need the namespacing. Imagine if you used a third-party app for social login but it collided with your existing models or templates because there was no way to namespace them.

Let’s say I have a Python module called rfid_inventory.directory. What should we use as the prefix for models created from this module - say, an Item model? (See above why you need prefixing as a single developer, and of course if you have multiple developers, it’s even more important to save your developers from having to check every existing model name before they make a new one)

  • If we use the full path, rfid_inventory.directory, we’re going to get a pretty ugly table name like rfid_inventory__directory__item. That’s maybe doable, but…
  • Your model is probably actually in the module in rfid_inventory.directory.models.item, making the table name rfid_inventory__directory__models__item__item. That’s not a good schema at that point.
  • If I rename my app’s source code directory from directory to items, now I have to rename all my tables in production in order to ship my app. This can’t be done without downtime.

The reasonable solution is to say “let’s put a prefix attribute on the Meta of each model to stop this”, and lo and behold, that’s what apps do for you by default.

I still think you’re correct that a small, single-developer project probably doesn’t need apps, but Django is generally about trying to give you a good foundation that’ll scale to more developers and more traffic. If we took out everything that small, single-developer sites didn’t need, it would be a framework that basically never got any traction.

Besides, Python already has a great app-free, small, single-developer-friendly framework - Flask. I think Django’s App abstraction is a very reasonable tradeoff to make your project much more expandable in future - without that small amount of upfront effort, which we almost entirely template away, you would run into a wall as soon as you wanted to start using third-party code.

1 Like

Thanks for the example! I think you probably misunderstood my suggestion. I’m not saying to ditch all apps infrastructure. I’m saying that we should work with a single user-created app per project. It would still provide the namespacing, would keep backwards compatibility with other parts. Most importantly, it would be less confusing for developers who create new projects, it would allow the code structure to grow organically.

1 Like

I mean, again, that is great for single-developer projects, and you should feel free to do that! But projects with tens of engineers benefit massively from splitting into apps to deconflict all their changes and source code, so I don’t think we should make Django pop out a “default” app, nor do I think it’s good advice for most people who have aspirations of a site that grows even moderately larger.

1 Like

That’s what I’m trying to understand. What benefits do apps introduce?

You said:

  • deconflicting changes and source code: apps intrinsically don’t solve code conflicts, this is about development process. Even if you split your project into multiple apps, nothing stops multiple people to work simultaneously on a single app. Expecting each team member to work on their “own app” won’t solve conflicts unless you create a robust development process which documents how changes should be introduced. This is a “people problem” not a software problem.
  • namespaces: this is only relevant when you use 3rd party apps, and single app solves this problem. Locally, everything except models can be namespaced naturally with directories or Python modules. Finally, you wouldn’t use two models with the same name in a single project. This sounds like a data model problem or like a situation when you do need to use a new app. But how often do you experience such problem? If it’s a niche use case then it definitely shouldn’t be a default approach.

Have I missed something here? I’d be happy to learn more usecases of apps. I’m not suggesting anything radically new here. I’m not suggesting to break backwards compatibility or anything. I’m trying to suggest an improvement that would simplify the mental model for the majority of developers, while retaining the ability to create more than one app per project for people who really need this (those who distribute apps or teams that work on huge projects). As I suggested earlier we could approach it from either documentation or project scaffolding perspective. All major frameworks work that way (Rails, Laravel, Yii etc) and developers build massive projects using these technologies. I think it’s valuable to developers to use technology that introduces the least amount of surprise.

Namespacing is super important when it specifically comes to the ORM, and specifically migrations. Allowing teams to develop apps one-per-team and coordinate the migrations among themselves is a lot easier than trying to keep a single global migration history.

Also:

I’ve seen this at least three times, so it absolutely happens. You’re right that you can solve a lot of these problems with coordination, but most large development teams are bad at that, and so having a natural way to separate apps from each other and reduce the communication complexity that teams need is welcomed.

That makes sense, thank you! I see how teams could benefit from the apps namespacing. Would you agree that developers who work as a team on huge project are still in minority? What I’m suggesting is to leave everything as is for teams, but improve things a bit for solo developers or people who are just starting out with Django because they are the majority. Like pre-generating a “core” app when starting a project, which they could split into separate apps as they grow or something like that. You get the best of both worlds. What do you think?

I know exactly what you mean, but this is probably where you and I differ - I think instructing people to design things with multiple apps from the start is the right approach, honestly. I don’t mind if you’ve decided one app is for you, but for those who are unsure, I like suggesting that you design your code as a set of loosely-coupled apps. In my view, it adds a tiny bit of friction up front for a lot of benefit in terms of separation, future scaling, refactor-ability, and so on.

Given that, you’ll be hard pressed to convince me we should change the new-project experience!

3 Likes

I agree with Andrew here.

1 Like

A slightly different take: imagine for a moment that we’re not talking about Django, but about web development in general. We could – and some frameworks encourage doing this, at least at the start – just write a single file and call it the_website.py and put absolutely all the code for our web site in that file.

But after a while that would get unwieldy. If, say, someone needed to go edit a particular thing, or add a new feature, there’d be a lot of code for them to scroll/search through to do it, and even with good code-indexing and searching tools the cost, in developer time, of finding the right things in that file would grow and grow.

So at some point we stop using the single-file approach and instead start maintaining separate files. And probably we settle on a way of doing this that’s based on some kind of logical structure. MVC is one option, but not the only one. And things get better for a while.

But inevitably the code will keep growing. Say we went with MVC; at some point we might have dozens or hundreds or even thousands of data-model classes in the models file, and the same for the other components. The problem is back: how can developers effectively engage with and work on this?

And again the answer is usually going to be separating things based on some kind of logical structure. One possible structure (again, not the only one) is to examine the functionality, and group together the bits of code that implement each general piece of the site’s functionality. So maybe all the auth and user code in one place, all the blog code in another place, all the marketing-lead-form code in another place, and so on.

And at this point we’ve arrived at something resembling Django: our site is composed of multiple pieces of functionality, each of which is implemented in a relatively self-contained way, and each of which uses a standard organizational pattern for its own components (models, views, and so on).

Now, not everybody necessarily needs all this – there will be plenty of projects that never grow all that large – and not necessarily everybody who needs this will do it (I’ve worked on my share of codebases that used the “only a few files, all of them huge” approach). But as a general pattern, it’s useful enough that it might be worth showing it to people up-front and encouraging them to adopt it.

And especially for Django, which wants to provide at least a certain amount of functionality pre-implemented for you, this is a useful approach. Imagine if, in order to write your first Django app, you had to scroll around in files that also contained the complete implementations of the contrib apps, for example. Even for experienced developers that would probably be annoying and difficult; for newer developers it would be much worse. So rather than starting with the “everything all in one place” approach, it makes sense for Django to introduce you to the idea of separating functionality into specific apps right from the start.

5 Likes

Apps always felt too much ceremony for me too so a couple of years ago we went to one app by default and never regretted about that. This is a great article on that topic and it has other useful practices too.

Basically from my experience it’s not clear how apps helps until you want to reuse code between projects. You can easily avoid namespace conflict with some extra levels eg. static/<module>/image.jpg or template/<module>/template.html. To make apps loosely coupled you need to make an extra effort and plan beforehand and this is very hard to accomplish in agile world. I often even don’t know in what module should I place my model and it’s quite hard to move models between apps. In one app setup I don’t fear to make mistake by placing model in wrong module because it takes a couple of minutes to move it into another one when things got clear (and you know the fear of refactoring is the worst thing).

When I see that some module is really can be reused between projects or it feels like it better to be separated from the rest of the project to prevent developers to place project specific logic into it then it can be moved into separate app quite easily because it’s already loosely coupled with the rest of the application that’s why we are moving it from the main app. Even if it contains some project specific entities they can be quickly refactored out of it.

If you’re going to try one app setup then one thing that I would recommend is to get rid of app prefix in table’s names, follow article’s advise and always set db_table explicitly, with that change writing and reading sql become a lot better. In future you may use a checker that warns developers if they added model without db_table so naming tables won’t require extra mental energy.

Overall apps is a useful concept but I would recommend everyone to give a try one app setup and see if you move faster with it and do you face any problems that wouldn’t be possible in multi app setup. Would be good if Django docs had some high level overview of both setups so people knew that app separation is not required thing in Django.

UPDATE:

One thing I want to clear up is that we still use apps like module structure, so if we have one app named mystore inside it we place modules like this:

mystore/
  models.py
  users/
    models.py
    views.py
  products/
    models.py
    views.py

this way we have many good properties that @andrewgodwin said apps have, eg. easy to remove some feature if it become obsolete and separation of concern.

4 Likes

This is a topic I’ve been thinking a lot about lately. I’d really like to agree with @andrewgodwin and @ubernostrum here, but I see a problem with that approach: It works as long as the separate apps are, as James and Andrew mentioned, self-contained and loosely-coupled. But as the project grows, things have a tendency to entangle themselves, like a set of cables left in a backpack. I’ve seen this happen especially in teams where there are relatively junior devs and lax discipline. You start with a division to apps that seems logical, but as the requirements change, as in-project utilities are developed, etc, suddenly you start to see dependencies in all directions. App A models depend on app B models, while app B views depend on app A utilities. Foreign keys and M2Ms are added in both directions. And Django supports dynamic and rapid development, this can all be made to work without too much effort.

At that point, refactoring is nearly impossible. Moving models between apps is not supported by Django; it requires custom migration operations, and even with them it’s not trivial. If you’re using the Admin, moving models between apps will change your UI. Even moving views between apps without affecting front-end code is difficult, if you did things the recommended way with separate urls.py files. each included with a prefix. To keep the URLs as they were, urls.py from one app needs to import views from another app, and there goes the loose coupling you’ve been trying to regain.

I am all for the vision that Andrew and James are promoting. We should be teaching people to break things up from the start. But currently, Django makes it very hard to correct mistakes in that division, or adapt it to changing requirements. In my opinion, the set of tools for this kind of refactoring is one of the greatest gaps Django still needs to fill, and we may come to find that we need to change some of our tenets in order to support them.

9 Likes

So am I right to assume that when you placed a model in an app, the models stays there forever (even if it no longer makes sense), unless you’re willing to do manual database work which doesn’t make any sense (that is why we use Django in the first place)? A problem like that would never happen with using a single app per project (you can shuffle models between modules at any point in development).

How is this a great development practice? This approach requires to spec out the project in perfect detail upfront to model the apps and models, in other words a waterfall approach (or live with the fact that your code structure doesn’t make sense as your codebase grows). The Django website says that the framework “encourages rapid development”, but encouraging the use of apps basically prevents to use agile practices effectively.

3 Likes

I think you’re overstating the case a little for how hard it is to move models around :slight_smile: It’s totally possible to develop iteratively while splitting your code into multiple apps! It’s not particularly pleasant but you can do it using just the migrations framework. I think it would be great if Django was better at handling this (but I’m not volunteering to do the work so I accept the compromise). I’ve done this kind of surgery on a work project a few times and it’s not been a big obstacle.

Sure, nothing will save you from bad process. But having your models divided up into different apps can avoid unnecessary process when different people are working on different bits of code.

Could you please tell me more about the process of moving a model from one app to another (considering this model has foreign keys with models from other apps)? I assume that this is why Django docs recommends to start a project with a custom user model since it’s a nightmare to migrate the model later

Could you please tell me more about the process of moving a model from one app to another (considering this model has foreign keys with models from other apps)?

Can’t you just set db_table to keep the old table name ? Of course you will need to change the app name in your foreignkey definitions but then I think it should work.

Exactly what I did in this situation, plus for deployment I had to delete some lines from django_migrations and fake apply them. Not really sure of the top of my head, but let us know if you would like more detailed a tutorial.

It’s more complicated than that.

Literally the first line in the answer to the question “how to move a model between two apps” is “*don’t do it!!*”.

1 Like