Publishing Django Apps

Publishing Django Apps

A Django application is a collection of files in a structure that exports a Web application. It includes things like views, models, migrations, commands, and everything I need to implement a Web site.

A Django project is one level higher that allows me to combine multiple Django applications and expose them from a one Web site.

Typically, a large site will be partitioned into multiple Django apps. There is one top-level urls.py in the project that then routes to different apps’ own urls.py. Most small companies have this setup.

One interesting aspect of this arrangement is that reusable logic and components can be packaged up as a Django application that can then be imported and incorporated into different Django Web sites. This reusable Django application doesn’t have to be authored by the same people implementing the Django website or even be cognizant of the Web projects in which it is brought into.

I have had success with publishing a couple of such Django apps:

  • vmigration-helper — an app that exports only custom commands with no UI or models. This project uses Poetry.
  • vcelery-task-runner — an app that exports UI, models, and logic to invoke Celery tasks. This project uses the setuptools, twine, and wheel.

Project Directory Planning

Planning: naming and subdirectory structure

There are some things to think about before starting work on a reusable Django app that will be published. Doing this early will avoid having to rename and rework.

Name of the package (e.g. as it appears on PyPI)

Think of a unique and representative name for the package as it appears on PyPI. The name should follow the constraints of the index and also not collide or be confused with other packages published there. There is no “namespace” feature of packages (e.g. as that supported by mvn repository), so the package name by itself will need to some namespacing properties. I’ve done this by prefixing my packages with “v” (e.g. “vmigration-helper”). Another way is simply having entire prefixes. For example, mycompany-projectname.

Supporting project and apps

src-layout

Python packaging guides (especially that from setuptools) in general prefers a “src-layout” structure of code to be exported from packages:

project/
  pyproject.toml
  ...
  src/
    mypackage/
      __init__.py
      < exported files here >
      ...

The key aspect here is a top-level “src” subdirectory underwhich is the main package to be exported.

single-package

One alternative is to skip the src subdir. This is described as the “flat-layout” from setuptools:

project/
  pyproject.toml
  mypackage/
    __init__.py
    < exported files here >
    ...

There are additional variances described in https://setuptools.pypa.io/en/latest/userguide/package_discovery.html. Since this post is not about setuptools, I will not repeat what’s there.

What’s important is to match the structure that Django projects used so that I can publish a Django application.

Mapping to Django

Constraint:

  • Only ONE Django app can be exported at a time. This is probably just as well since it will be an incentive for me to author focused and small packages.
  • The Django app name should match the package name that will be published to PyPI (not necessary but useful).

Here’s a typical Django project:

myproject/
  ...
  pyproject.toml
  manage.py
  myproj/
    settings.py
    urls.py
    ...
  app1/
    __init__.py
    urls.py
    apps.py
    management/
      __init__.py
      commands/
        __init__.py
        ...
    migrations/
      __init__.py
      ...
    templates/
      ...
    tests/
      ...
    ...

Typically, I would like to export everything from app1 except tests/.

Since there are multiple 1st level subdirectories under myproject/ (myproj/ and app1/), setuptools won’t use the automatic discovery, even when it tries the flat-layout strategy.

There are various tweaks to be done in pyproject.toml with mixed success. The strategies I found that works is:

Explicit Package List
  • List the package names in pyproject.toml under tools.setuptools.packages:
[project]
name = "app1"
version = "0.0.1.dev0"
...

[tool.setuptools]
packages = [
  "app1",
  "app1.management",
  "app1.management.commands",
  "app1.migrations",
  "app1.templates",
]

NOTE that I skipped the tests package. Therefore, it will not be exported, and people importing my app won’t see the tests files. This is just a subjective decision; I suppose I could just include everything.

The downside of this approach is that, if I ever add an package or subpackage I want to export, I will need to add it to the pyproject.toml in that section.

Use Custom Discovery with MANIFEST.in:

Use the Custom Discovery by using where, include, exclude, and use MANIFEST.in.

Notes on MANIFEST.in: https://setuptools.pypa.io/en/latest/userguide/miscellaneous.html#

pyproject.toml

[project]
name = "app1"
version = "0.0.1.dev0"
...

[tool.setuptools.packages.find]
where = ["."]
include = ["vsample_app"]
namespaces = false

MANIFEST.in

global-exclude tests.py

Project Name

The project name in this example, “app1,” matches the Python package (subdirectory) name. This is also not required, but it’s clearer to users since there is only one name for them to remember:

  • pip install app1
    • My project is named “app1” in pyproject.toml
  • from app1.management.commands import ( ... )
    • My subdirectory (aka Python package) is named “app1”

Versioning

The convention is to use the Semantic Versioning strategy, although there are some packages whose authors take a lot of liberties with the major vs minor bump decision (e.g. not bumping the major version when they should, leading to breaking changes that are not detected until too late).

In addition, once a package is published to PyPI, it cannot be updated. (It can be deleted or “yank”ed, none of which allows re-uploading a new copy).

To avoid having to bump versions unnecessary between iterations, use the development version (such as 0.0.1.dev0, 0.0.1.dev1, 0.0.1.dev2, and so on, while working on 0.0.1)

Use TestPyPI

Until the package is well tested in end-to-end scenarios, publish it to TestPyPI instead of the main PyPI.

Additional Topic

Tags: , ,