Service Mesh

Service Mesh#

A service mesh runs a small proxy alongside every workload. The proxy handles inter-service traffic: mTLS, retries, timeouts, circuit breakers, traffic shifting, observability. The application code stops caring how it talks to other services.

Cloud-native’s answer to “every microservice ends up reimplementing the same network plumbing”.

        flowchart LR
  subgraph Pod1["Pod A"]
    AppA[App A]
    SidecarA[Sidecar Proxy]
    AppA -->|"localhost"| SidecarA
  end

  subgraph Pod2["Pod B"]
    AppB[App B]
    SidecarB[Sidecar Proxy]
    AppB -->|"localhost"| SidecarB
  end

  SidecarA -->|"mTLS<br/>retries<br/>circuit-break<br/>traffic shift"| SidecarB

  ControlPlane[Mesh Control Plane] -.->|policy + config| SidecarA
  ControlPlane -.->|policy + config| SidecarB
  SidecarA -.->|telemetry| Obs[Observability]
  SidecarB -.->|telemetry| Obs

Where It Fits#

A service mesh is a class of cloud-native infrastructure, distinct from L4 load balancers, edge L7 proxies, API gateways, ingress controllers, the CNI underneath, and the service-discovery layer above. The clean taxonomy:

Category	What it does	Examples
L4 load balancer	TCP / UDP forwarding; no application awareness	HAProxy (TCP), AWS NLB, MetalLB, IPVS
L7 load balancer / rev proxy	HTTP routing, TLS termination at the edge	nginx, HAProxy (HTTP), Envoy standalone, Traefik
API gateway	Public-API edge: auth, rate-limit, transforms	Kong, AWS API Gateway, Apigee, Tyk, Ambassador
Ingress controller	Maps L7 LB onto K8s `Ingress` / `Gateway` CRDs	nginx-ingress, Traefik, Contour, Istio Gateway
Service mesh (this page)	East-west L7 between every pair of services	Istio, Linkerd, Consul Connect, Cilium Mesh, Kuma
Service proxy	The data-plane primitive a mesh is built from	Envoy, linkerd2-proxy, ztunnel
Service discovery	Resolve “where is service X?”	CoreDNS, Consul, K8s built-in

Two axes worth holding clearly:

North-south vs east-west. Ingress / API gateway = north-south (clients → cluster). Service mesh = east-west (service → service inside the cluster). They coexist; many environments run both.
L3/L4 vs L7. The CNI (Calico, Cilium, Flannel) gives pods their basic IP connectivity at L3/L4. The mesh runs on top and adds L7 semantics. Cilium blurs this with eBPF mesh features in the kernel.

What a Mesh Provides#

The features a mesh adds without changing application code. mTLS with automatic rotation, traffic management with weighted and header-based routing, resilience primitives like retries and circuit breakers, authorization policies, uniform observability, and multi-cluster federation.

mTLS between services, automatic certificate issuance and rotation; service-to-service identity.
Traffic management, weighted routing, header-based routing, fault injection.
Resilience, timeouts, retries with budget, circuit breakers, bulkheads.
Authorization, which services may call which, with which methods, on which paths.
Observability, per-service-pair latency, error rate, request volume, with no application-side instrumentation.
Multi-cluster, federate meshes across clusters / regions.

The application code does HTTP / gRPC; the mesh does the rest.

The Major Implementations#

The mesh products an operator picks among. Istio is the feature leader; Linkerd wins on simplicity and footprint; Cilium puts mesh logic in the kernel via eBPF; Consul Connect bridges containers and VMs; AWS App Mesh, OSM, and Kuma cover specific niches.

Istio, the most feature-rich; Envoy-based; deep ecosystem. Heavier operational footprint; getting lighter with the Ambient data plane (no sidecars on every pod).
Linkerd, focused on simplicity, fast, lightweight. Rust-based data plane (linkerd2-proxy). Limited feature set vs. Istio, but covers the 90% case.
Cilium Service Mesh, eBPF-based; can run without sidecars by pushing logic into the kernel. Pairs naturally with Cilium’s CNI.
Consul Connect, HashiCorp’s mesh; multi-platform (VMs + containers); pairs with Consul service discovery.
AWS App Mesh, managed; tied to AWS.
Open Service Mesh (OSM), archived; mention only because docs refer to it.
Kuma, multi-zone; backed by Kong.

Sidecar vs. Sidecar-less#

The sidecar pattern (a proxy container in every pod) is the classic shape. Costs:

Memory + CPU per pod, 100s of MB across thousands of pods adds up.
Extra startup time.
Pod restart needed for proxy upgrades.

Modern alternatives:

Ambient mode (Istio), ztunnel runs per node; per-namespace L7 proxies (waypoint) appear only when needed. Drops the per-pod cost.
Cilium Service Mesh, eBPF in the kernel handles most of the data plane; no per-pod proxy.
Proxyless gRPC, xDS-aware gRPC libraries pull mesh config directly. Only works for gRPC.

The trend in 2026 is away from per-pod sidecars for steady-state cost.

What a Mesh Is Not#

The boundary an honest evaluation should respect. A mesh isn’t an API gateway, a CNI replacement, or an application framework, and it isn’t required for every microservice estate. Many teams ship microservices fine with DNS, ingress mTLS, and per-language libraries.

Not an API gateway, those handle north-south (client → cluster) traffic. A mesh handles east-west (service → service).
Not a CNI, a CNI provides pod networking; a mesh layers on top. Cilium blurs this.
Not an application framework, the mesh doesn’t replace SDKs for application-level concerns (caching, business logic, complex retries with state).
Not required, many teams operate microservices fine with in-cluster DNS, mTLS at the ingress, and per-language libraries.

When to Adopt#

The signals that justify the operational complexity. Many services, mTLS-by-default requirements, traffic-shaping needs, uniform observability, runtime policy, and a funded platform team, adopt when several apply, not as a forward-looking architectural bet.

A service mesh earns its keep when several of these are true:

You have many services (dozens or more).
You need mTLS by default and don’t want each team implementing it.
You need traffic shaping, canaries, blue/green, weighted rollouts.
You need uniform observability, per-service-pair golden signals across the fleet.
You need policy, who can call whom, enforced at runtime.
Your platform team is funded for the operational complexity.

If you have 5 services and no team to operate the mesh, the answer is usually no.

Operational Costs#

The costs that don’t disappear after install. A control plane to operate, per-pod overhead in classic sidecar mode, hop latency, debugging the proxy-versus-app boundary, mesh upgrades on their own schedule, and multi-cluster federation complexity all compound.

Real, persistent costs:

Control plane to operate, a critical dependency.
Per-pod overhead in classic sidecar mode.
Latency added per hop, usually <1 ms but real.
Debugging, “is it the app or the proxy?” is a new question.
Upgrades, mesh versions need their own change-management story.
Multi-cluster federation is significantly harder than single-cluster.

These are why “platform engineering” exists; meshes are platform engineering’s flagship product.

Common Pitfalls#

The traps that surface during real adoptions. A mesh won’t fix chatty service design; enabling all features day one produces unreviewable change; some workloads shouldn’t be meshed at all; and the mesh’s data volume needs dashboards and alerts that scale with it.

Adopting a mesh to fix bad service design, the mesh doesn’t fix chatty services or sync chains.
Enabling all features day one, start with mTLS + observability; add policy and traffic shaping when you have a use case.
Ignoring escape hatches, some workloads (databases, legacy apps) shouldn’t be meshed.
Underinvesting in tooling, the mesh produces a lot of data; dashboards and alerts must keep up.