Packaging

Packaging#

The discipline of taking a working src/ tree and turning it into a distributable artefact that the operator (and other operators) can pip install. This page covers the build side: pyproject.toml, build backends, wheel and sdist artefacts, version management, and publishing to PyPI or an internal index. For the consumer side (installing, locking, mirroring) see Tooling.

Anatomy#

A modern Python package is a src-layout repository plus a pyproject.toml that declares the build backend, the project metadata, and the dependency graph. PEP 517 / 518 / 621 settled the file structure; every modern tool reads it.

project/
├── pyproject.toml
├── README.md
├── LICENSE
├── src/
│   └── mypkg/
│       ├── __init__.py
│       └── core.py
└── tests/
    └── test_core.py

The src/ layout keeps the package out of the import path when the test runner is in the project root, forcing an installed-mode import. The alternative (no src/) lets test imports succeed against the working tree even when the package is broken on install.

Artefact	Role
`pyproject.toml`	Project metadata, dependencies, build-system declaration.
`src/<pkg>/__init__.py`	The package’s import entrypoint. Holds `__version__` if not sourced from elsewhere.
`LICENSE`	Required for redistribution; PyPI surfaces the classifier.
`README.md` / `README.rst`	The long description PyPI renders on the project page.
`dist/`	Build output. Holds the `.whl` (wheel) and `.tar.gz` (sdist) artefacts. Never commit.

pyproject.toml#

The single source of truth. The [build-system] table tells pip / uv how to build; [project] carries the metadata; tool tables ([tool.ruff], [tool.pytest.ini_options]) configure the rest of the toolchain.

[build-system]
requires      = ["hatchling>=1.24"]
build-backend = "hatchling.build"

[project]
name            = "mypkg"
version         = "0.1.0"
description     = "One-line summary."
readme          = "README.md"
license         = { file = "LICENSE" }
requires-python = ">=3.12"
authors         = [{ name = "Operator", email = "op@example.com" }]
keywords        = ["cybint", "tooling"]
classifiers     = [
  "Programming Language :: Python :: 3",
  "License :: OSI Approved :: MIT License",
  "Operating System :: OS Independent",
]
dependencies = [
  "httpx>=0.27",
  "click>=8.1",
]

[project.optional-dependencies]
dev = ["pytest>=8", "ruff>=0.4", "mypy>=1.10"]

[project.urls]
Homepage  = "https://example.com/mypkg"
Source    = "https://github.com/operator/mypkg"
Issues    = "https://github.com/operator/mypkg/issues"

[project.scripts]
mypkg = "mypkg.cli:main"           # console-script entry point

[tool.hatch.build.targets.wheel]
packages = ["src/mypkg"]

The [project.scripts] table is what creates a mypkg command on the user’s PATH after install; the value is a module:callable reference.

Build backends#

The build backend turns source into a wheel. The operator picks one and stops thinking about it.

Backend	When
`hatchling`	Modern default. Fast, declarative, version source from file or VCS tag. Recommended for new projects.
`setuptools`	Universal. Required when the package compiles C / Cython extensions or uses legacy `setup.py` hooks.
`flit-core`	Minimalist; pure-Python single-module packages.
`pdm-backend`	Used when the project is managed end-to-end by `pdm`.
`maturin`	The bridge to Rust extensions via PyO3.
`scikit-build-core`	The bridge to CMake-driven native extensions.

Versioning#

PEP 440 governs the version string. Stick to semantic versioning (MAJOR.MINOR.PATCH) for release artefacts; PEP 440 allows suffixes for pre-release (1.2.0a1, 1.2.0rc1) and development (1.2.0.dev3) builds.

Three places to store the version, pick one.

Source	How
Hard-coded in `pyproject.toml`	`version = "1.2.0"` under `[project]`. Simple; manual.
Hard-coded in `__init__.py`	`__version__ = "1.2.0"` plus `dynamic = ["version"]` and a backend hook. One file edit per release; runtime introspection works.
From a VCS tag	`hatch-vcs` or `setuptools-scm` reads `git describe`; no commit needed to bump.

Stamping from the VCS tag is the lowest-friction option for any project that publishes from CI.

Build#

The build package (or uv build / hatch build) invokes the backend and produces both a wheel and an sdist in dist/.

$ python -m pip install --upgrade build
$ python -m build

Successfully built mypkg-0.1.0.tar.gz and mypkg-0.1.0-py3-none-any.whl

uv does the same with no separate install step.

$ uv build

Two artefacts come out of dist/.

Artefact	Contents
`mypkg-1.2.0-py3-none-any.whl`	Built distribution. The fast path. `pip install` unzips it directly. Filename encodes Python tag, ABI tag, platform tag.
`mypkg-1.2.0.tar.gz`	Source distribution. `pip` falls back to it when no compatible wheel exists; runs the backend on the user’s machine.

Always ship both; PyPI accepts both, and pip prefers the wheel.

Publish#

twine is the conventional uploader; uv publish and hatch publish do the same. Authenticate with an API token (pypi-...) generated on the PyPI account page, scoped to a single project once the project exists.

$ python -m pip install --upgrade twine
$ python -m twine upload dist/*

Or with uv.

$ uv publish

Test with TestPyPI first.

$ python -m twine upload --repository testpypi dist/*
$ python -m pip install --index-url https://test.pypi.org/simple/ mypkg

The token lives in ~/.pypirc or in CI secrets. Treat it like an SSH key; rotate on compromise.

# ~/.pypirc
[pypi]
  username = __token__
  password = pypi-AgEIcHlwaS5vcmcCJ...

[testpypi]
  repository = https://test.pypi.org/legacy/
  username   = __token__
  password   = pypi-AgENdGVzdC5weXBp...

The username for token auth is the literal string __token__; the token itself is the password.

Internal indexes#

For private packages (offensive tooling, customer-specific adapters, anything that should not hit the open registry), publish to an internal index instead. The structure mirrors PyPI. Upload over HTTP, install with an --index-url override.

Server	When
Artifactory / Nexus / GitLab Package Registry	Enterprise default; SSO and audit; mirrors PyPI under the same URL.
`devpi-server`	Self-hosted; cheap; staging plus release indexes; the operator’s home lab.
`pypiserver`	Single-file HTTP server backed by a directory of wheels. Minimum-viable internal index.

Always pin --index-url (not --extra-index-url) when serving internal packages of the same name as a public one; pip and uv will silently pick the higher version across indexes, which is the dependency-confusion attack vector.

CI#

Build and publish from CI; never publish from the operator’s laptop on a regular cadence. The minimum-viable pipeline.

Trigger on a tag matching v*.*.*.
Check out the repository at the tag.
Run tests, lint, and type-check.
python -m build.
twine check dist/* to catch metadata mistakes.
twine upload dist/* with the API token.

GitHub Actions, GitLab CI, and CircleCI all have first-class PyPI publish actions. PyPI’s trusted publishing removes the token entirely; the OIDC token from the CI provider is exchanged for a short-lived PyPI credential at upload time. Use it when the CI provider supports OIDC.

Files#

The on-disk surface around a package.

Path	Purpose
`pyproject.toml`	Project metadata and build configuration.
`src/<pkg>/`	The importable package.
`src/<pkg>/__init__.py`	Package entrypoint; often holds `__version__`.
`dist/`	Build artefacts (wheel, sdist). Gitignored.
`build/`	Backend scratch space. Gitignored.
`*.egg-info/`	Legacy setuptools metadata. Gitignored.
`~/.pypirc`	Per-user upload credentials.
`MANIFEST.in`	Legacy include / exclude rules for the sdist; rarely needed with modern backends.

Common Tasks#

Initialise a new package.

$ uv init --package mypkg
$ cd mypkg

Build wheel and sdist locally.

$ uv build

Inspect the wheel’s contents.

$ unzip -l dist/mypkg-0.1.0-py3-none-any.whl

Sanity-check metadata before upload.

$ python -m twine check dist/*

Upload to TestPyPI.

$ python -m twine upload --repository testpypi dist/*

Upload to PyPI.

$ python -m twine upload dist/*

Install your own wheel into a fresh venv to verify.

$ uv venv .venv-test
$ .venv-test/bin/pip install dist/mypkg-0.1.0-py3-none-any.whl
$ .venv-test/bin/mypkg --help

Bump version from a git tag (with hatch-vcs).

$ git tag v0.2.0
$ uv build                          # version derived from tag

References#

Setup for project bootstrap and venv management.
Tooling for installing, locking, and the toolchain.
Testing for the test suite that should pass before publish.
PEP 517 — build-system interface.
PEP 518 — pyproject.toml.
PEP 621 — [project] metadata.
PEP 440 — version specifiers.
Packaging Python Projects.
Trusted Publishers — OIDC-based PyPI uploads.