Frameworks

A framework is an opinionated runtime the operator hands control to, instead of a library the operator calls. Python’s framework ecosystem is broad, pick the framework that matches the kind of job, web service, data pipeline, CLI, ML model, infrastructure orchestration, or a security toolkit.

For installing frameworks and managing their deps, see Setup and Tooling. For the libraries underneath, see Libraries. For the networking surface most frameworks ride on, see Networking.

Web

The web framework picks the structure of every HTTP service the operator ships.

Framework	Style	When to reach for it
Django	Batteries-included	Anything with a database, admin, auth, sessions, and templates. The default for full-stack web apps.
Flask	Microframework	Small services where you want to pick every piece (ORM, auth, templating) themselves.
FastAPI	Async, type-driven	Modern HTTP/JSON APIs. Type hints drive validation, docs, and OpenAPI. Default for new JSON services.
Starlette	ASGI toolkit	The substrate under FastAPI. Use directly when the operator needs less framework than FastAPI gives.
Litestar	Async alternative to FastAPI	Class-based routing, plugin system, multipart-first.
Quart	Async Flask	When the operator has a Flask codebase that needs async.

ASGI / WSGI servers

The framework defines the application; the server hosts it.

• gunicorn, WSGI; the long-time default for Django and Flask.

• uvicorn, ASGI; for FastAPI, Starlette, Litestar, Quart.

• granian, Rust-based ASGI / WSGI server.

• hypercorn, ASGI with HTTP/2 and HTTP/3.

GraphQL and APIs

• strawberry-graphql, type-driven GraphQL.

• graphene, the older GraphQL library.

• django-rest-framework, REST APIs on Django.

Middleware

Middleware is a chain-of-responsibility pattern: a request flows through a stack of handlers, each of which can inspect, modify, short-circuit, or pass through to the next. Used heavily in Django, Flask, FastAPI, and Starlette, and in operator pipelines (request signing, retry, rate-limit, auth, logging).

Plain middleware is a callable that takes the request and the next handler and returns a response.

from typing import Callable

Handler    = Callable[[dict], dict]
Middleware = Callable[[dict, Handler], dict]

A logging middleware tags the request and response.

def logging_middleware(request, next_handler):
    print(f">>> {request['method']} {request['path']}")
    response = next_handler(request)
    print(f"<<< {response['status']}")
    return response

An auth middleware short-circuits unauthorised requests.

def auth_middleware(request, next_handler):
    if not request.get("token"):
        return {"status": 401, "body": "unauthorized"}
    return next_handler(request)

Compose the stack with a builder.

def build_stack(handler, *middlewares):
    for mw in reversed(middlewares):
        prev = handler
        handler = lambda r, m=mw, h=prev: m(r, h)
    return handler

app = build_stack(
    core_handler,
    logging_middleware,
    auth_middleware,
)

WSGI middleware wraps the WSGI application(environ, start_response) callable. ASGI middleware does the same for the async application(scope, receive, send) form. Django, Flask, FastAPI, and Starlette all expose this hook.

class HeaderTaggingMiddleware:
    def __init__(self, app):
        self.app = app

    async def __call__(self, scope, receive, send):
        async def send_wrapper(message):
            if message["type"] == "http.response.start":
                message["headers"].append(
                    (b"x-operator", b"cybint")
                )
            await send(message)
        await self.app(scope, receive, send_wrapper)

Mount the wrapper around the ASGI app.

app = HeaderTaggingMiddleware(fastapi_app)

Middleware near the outside wraps everything; middleware near the inside wraps only the handler. Typical order, outside in, logging, tracing, error handling, CORS, compression, auth, rate limit, request validation, the handler.

Data

The operator’s stack for analysis, ETL, and ML model training.

• numpy, ndarray and vectorised numeric operations. The foundation everything else builds on.

• pandas, dataframes and series. The operator’s tool for tabular OSINT data, log analysis, and ad-hoc analytics.

• polars, Rust-backed dataframe with lazy execution and multi-core performance. Pandas alternative for larger data.

• dask, parallel pandas / numpy with out-of-core execution.

• pyarrow, Arrow columnar format. Backs parquet IO for pandas and polars.

• duckdb, in-process analytical SQL on Parquet or in-memory frames.

• matplotlib, plotting.

• seaborn, statistical plots on matplotlib.

• plotly, interactive plots.

CLI

The frameworks for shipping command-line tools.

• click, declarative CLI with decorators. The classic.

• typer, click driven by type hints. Modern default for new CLIs.

• argparse`, stdlib option-parser. Fine for small scripts.

• rich-click, click output via rich.

• textual, modern TUI framework for full-screen terminal apps.

• prompt-toolkit, interactive shells with completion and syntax highlighting.

ML

Used briefly here; full coverage is out of scope for the operator handbook.

• scikit-learn, classical ML (classification, regression, clustering, decomposition).

• pytorch, deep learning, the research-default framework.

• tensorflow, deep learning, Google’s framework.

• transformers, pretrained LLMs and ASR / vision models.

• langchain, llamaindex, LLM-app frameworks.

Infrastructure: orchestration

Workflow orchestrators schedule and run DAGs of tasks with retries, dependencies, and a UI.

• apache-airflow, the standard. DAGs defined in Python, web UI, scheduler, executors. Heavy.

• prefect, modern alternative. Python-native, easier to develop and test.

• dagster, data-engineering-first orchestrator with strong type / asset model.

• luigi, the original Spotify framework. Lighter, file-based.

Infrastructure: task queues

Distributed task execution against a message broker.

• celery, the heavyweight default. Redis or RabbitMQ broker; celery beat for periodic. Rich, mature, complex.

• dramatiq, simpler alternative to Celery. Same brokers; smaller API surface.

• rq, Redis-backed, very small. For one-shot background jobs.

• huey, another lightweight Redis/SQLite alternative.

• arq, asyncio-native Redis task queue.

Infrastructure: config management

Agentless or agent-based, configure remote hosts at scale from a declarative spec.

• ansible, agentless, SSH-based, YAML playbooks. The default for configuring remote hosts. Python under the hood; modules and plugins are Python.

• saltstack, agent-based, ZeroMQ-driven, YAML state files. Faster fan-out, heavier setup.

• fabric, Python-native SSH task runner. Lighter than Ansible; write Python tasks instead of YAML.

Infrastructure: IaC

Infrastructure as code, declare cloud resources in Python.

• pulumi, multi-cloud IaC in Python (instead of Terraform HCL). The same state model as Terraform.

• aws-cdk, AWS-only IaC in Python.

• terraform-cdktf, CDK for Terraform, Python target.

Infrastructure: serverless

Python-native frameworks for AWS Lambda and equivalents.

• chalice, microframework for AWS Lambda + API Gateway. AWS-only.

• zappa, deploy a WSGI app (Flask, Django) to Lambda.

• serverless-framework, Node-based CLI with Python support.

Infrastructure: kubernetes

• kubernetes, the official Python client.

• pykube-ng, simpler alternative client.

• kopf, framework for writing Kubernetes operators (controllers) in Python.

Infrastructure: observability

• opentelemetry-python, traces, metrics, logs. Vendor-neutral.

• prometheus-client, expose Prometheus metrics from a Python process.

• sentry-sdk, error reporting and performance monitoring.

• structlog, structured logging (also in Libraries).

OSINT

Operator-facing OSINT collection and analysis frameworks.

• recon-ng, modular reconnaissance framework. Modules in Python; the operator extends with new collectors and exporters.

• maltego-trx, build Maltego transforms in Python.

• theHarvester, email, subdomain, and host harvester. CLI primarily, but a Python library too.

• spiderfoot, Python-based OSINT automation with hundreds of modules.

Red team

Frameworks for offensive operations against authorized targets.

• pwntools, CTF and exploit development framework. ROP, shellcode, format-string bugs, remote / local target abstractions.

• impacket, the reference Python implementation of network protocols (SMB, MSRPC, Kerberos, LDAP, WMI). The operator’s toolkit for Active Directory recon and post-exploitation.

• mitmproxy, interactive HTTPS proxy with a Python scripting API. The operator’s tool for inspecting and modifying traffic from a target app or device.

• responder, LLMNR / NBT-NS / mDNS poisoner.

• bloodhound.py, Python BloodHound ingestor for Active Directory enumeration.

Forensics

Frameworks for memory, disk, and artifact analysis.

• volatility3, memory forensics. Parse memory dumps; extensible with Python plugins.

• yara-python, Python bindings for YARA rule matching against files, memory, or strings.

• pefile, parse PE (Windows executable) headers.

• dfvfs, Digital Forensics Virtual File System. Mount and walk disk images.

• plaso / log2timeline, super-timeline extraction from forensic artifacts.

• pyshark, tshark-driven Python interface for PCAP analysis.

Reverse engineering

Frameworks for static and dynamic binary analysis.

• angr, binary analysis platform. Symbolic execution, control-flow recovery, exploit-primitive hunting.

• r2pipe, Python bindings for radare2. The operator scripts r2 sessions in Python.

• frida-python, dynamic instrumentation. Inject JavaScript into a running process (Android, iOS, desktop) and drive it from Python.

• capstone, multi-platform disassembler.

• keystone, multi-platform assembler (companion to capstone).

• unicorn, CPU emulator for multi-architecture binaries.

• ghidra-bridge, drive Ghidra from a Python interpreter outside the Ghidra JVM.

Scraping / automation

The frameworks for crawling sites and driving real browsers.

• scrapy, full-stack web scraping framework. The headline tool for News pipeline.

• playwright-python, modern browser automation. Headless Chrome, Firefox, WebKit.

• selenium, the classic browser automation framework.

• pyppeteer, Puppeteer port for Chrome control.

References

• Libraries for libraries that are not frameworks.

• Networking for networking-specific stacks (HTTP, sockets, DNS, SSH, scapy).

• Projects for projects that consolidate the frameworks above.

• Operations for the offensive / defensive context the red-team and forensics frameworks operate in.

• Awesome Python, a curated list by topic.

• Awesome Python Security, security-oriented Python libraries and frameworks.