Scalability

A campaign rarely stays the size it started at. Collection volume climbs, redirectors get added, analyst queries get heavier, the defended estate doubles in sensors. Scalability is the property that adding resources adds capacity, and the operator’s infrastructure either supports the campaign’s growth or becomes the thing that limits it. Most operator-built systems start with plenty of room to grow vertically; horizontal becomes necessary later.

Vertical vs. Horizontal

The first scaling axis to pick. Vertical scaling buys headroom by upgrading the box; horizontal scaling buys it by adding boxes. The common mistake is jumping to horizontal too early; vertical is simpler, cheaper, and goes much further than most engineers expect.

• Vertical, bigger machines. Fewer moving parts. Hard cap at the largest instance.

• Horizontal, more machines. Effectively unbounded. Requires the workload to be partitionable and the state to be shared via a coordination point (DB, cache, queue).

Vertical scaling is undervalued. A single big Postgres instance handles more than most applications will ever throw at it.

Stateless Compute

If the application servers carry no per-request state, they scale horizontally trivially: add instances, balance traffic across them. Achieving statelessness means evicting the state to some shared place that the next instance can read from – session stores, object storage, queues.

• Push session state to a shared store (Redis, signed cookies, JWT).

• Push uploaded files to object storage.

• Push background work to queues.

The hard scaling problems usually live in state, not in stateless request handlers.

Caching

Caches turn expensive operations into cheap ones, at the cost of staleness and invalidation complexity. Phil Karlton’s quip is right: the two hard problems in computer science are cache invalidation and naming things. The cache hierarchy below covers where each layer fits.

• Browser cache, HTTP headers (Cache-Control, ETag).

• CDN edge cache, static assets and increasingly dynamic content.

• Reverse-proxy cache, nginx, Varnish in front of the app.

• Application cache, in-process LRU for the very hottest items.

• Distributed cache, Redis / Memcached across instances.

• Database cache, buffer pool, query cache, materialized views.

Strategies: cache-aside, read-through, write-through, write-behind. Pick based on consistency requirements and tolerable miss cost.

Sharding / Partitioning

When a single database can’t keep up. Sharding is one of the last scaling levers operators reach for, not because it doesn’t work, but because the partition key choice locks in the data layout for years and is much harder to change than to pick. Three axes for the split.

• Vertical partitioning, different tables on different boxes.

• Horizontal partitioning, the same table split by key range, hash, or list.

• Functional partitioning, different services own different domains.

Choose the partition key carefully; it’s hard to change later. Pick something that distributes hot keys evenly and keeps related data co-located.

Patterns.

• Tenant-per-DB / per-schema for SaaS isolation.

• Hash by user_id for evenly distributed user-scoped workloads.

• Time-based partitioning for append-heavy time series.

Read Replicas

Most workloads are read-heavy. Async streaming replicas absorb reads while the primary handles writes; a 10:1 read:write ratio is common, so each replica added carries roughly its weight in read capacity. The trade is replication lag, which the application has to learn to live with.

• Tolerate replication lag in the application: read-your-writes patterns, session pinning to the primary for critical reads.

• Don’t use replicas as a backup; they replicate deletes faithfully.

Async / Queueing

Move expensive work off the request path. The user gets a fast 202 Accepted; workers grind through the queue without holding up the response. The pattern is one of the highest- leverage moves in scaling a web app. Below are the typical candidates for offloading.

• Email/SMS sending, image processing, analytics, exports.

• Webhooks to third parties.

• Anything > ~100 ms that doesn’t need to be in the response.

The user gets a fast 202 Accepted; workers grind through the queue.

Load Shedding

When you can’t keep up, drop requests early instead of slowing everything down. A fast 429 or 503 is a kindness compared to serving traffic that piles up behind a saturated downstream and times out anyway. The patterns below shed load at the edge, where it’s cheap.

• Token-bucket rate limits at the edge.

• Per-tenant quotas.

• Reject low-priority work first when overloaded.

• Fast-fail at the load balancer beats unbounded queueing inside.

Capacity Planning

The discipline that separates “we kept up” from “we got lucky.” Most scaling decisions are made on intuition and look fine in calm weather; the planning practices below build the data the team will need when traffic actually spikes.

• Measure before you scale, profile, trace, identify the bottleneck.

• Model growth, linear in users? quadratic in graph degree?

• Stress test to known load levels; find the knee in the latency curve.

• Headroom, run at 50-70% of capacity so you have room to absorb spikes.

• Auto-scale on the right signal: CPU is fine for many web workloads; queue depth or active connections often beats it.

Common Anti-Patterns

The four scaling moves that look responsible from the outside and cost more than the problem they solve. Each one has a legitimate use case; the anti-pattern is reaching for them preemptively, before the simpler approach has actually run out of room.

• Sharding too early, introduces complexity that vertical scaling would have avoided.

• Caching everything, staleness bugs and invalidation pain.

• Microservices for “scale”, network and observability costs usually exceed the savings.

• Over-tuning the ORM, the actual query is the bottleneck; learn the query plan.

Cost

Scalability without cost discipline becomes “the expensive system that might also be slow”. Measure cost per request, per user, per tenant. Optimize the top line items first.