Configuration

Configuration is the layer between code and behavior. The operator changes it without rebuilding, audits it without reading source, and watches it react in production faster than a deploy. It comes in two flavours, plain configuration (feature flags, service endpoints, runtime parameters) and secrets (credentials, keys, tokens). Both want the same properties, versioned, auditable, distributable, revocable; the secrets variant adds encryption at rest, fine-grained access control, and short-lived issuance.

This section covers the services the operator stands up or consumes for both. The orchestrator-native primitives (Kubernetes ConfigMaps, Docker configs) live with their runtimes; here are the stand-alone systems that back them.

Two Layers

        flowchart LR
  subgraph plain[Plain configuration]
    direction TB
    ETCD[(etcd)]
    CKV[(Consul KV)]
    ZK[(ZooKeeper)]
  end
  subgraph sec[Secret managers]
    direction TB
    VAULT[(HashiCorp Vault)]
    SM[AWS Secrets Manager]
    KV[Azure Key Vault]
    GSM[GCP Secret Manager]
    OSS[Doppler / Infisical]
  end
  subgraph cluster[In-cluster patterns]
    direction TB
    ESO[External Secrets Operator]
    SS[Sealed Secrets]
    SOPS[SOPS-encrypted git]
  end
  APP[Workload] --> plain
  APP --> sec
  sec --> cluster
  cluster --> APP
    

• Plain configuration stores hold non-sensitive runtime parameters and act as service discovery / coordination back-ends.

• Secret managers hold credentials, signing keys, API tokens; gate access through ACLs and short-lived leases; audit every read.

• In-cluster patterns bridge git-stored declarations to in-cluster secrets without leaking the cleartext through git.

Plain Configuration

etcd

The CNCF-graduated distributed key-value store. Raft consensus, mTLS for client and peer traffic, watch streams, leases. The control-plane datastore behind Kubernetes.

Property	Detail
Ports	2379/tcp (client), 2380/tcp (peer).
Quorum	Odd member count (3 or 5). 3 tolerates 1 failure; 5 tolerates 2.
Auth	mTLS plus user / role / role-binding RBAC.
Backup	etcdctl snapshot save. Daily plus pre-upgrade; off-cluster.
Encryption at rest	Off by default; enable via Kubernetes EncryptionConfiguration or systemd --experimental-encryption-key.

$ etcdctl --endpoints=https://etcd:2379 \
    --cacert=/etc/etcd/ca.pem --cert=/etc/etcd/client.pem --key=/etc/etcd/client.key \
    put /config/feature/new-routing true
$ etcdctl get /config/feature/new-routing
$ etcdctl watch --prefix /config/

Standalone etcd is rare outside Kubernetes; mostly used as a custom-built coordination service or a distributed-lock back-end.

Consul KV

HashiCorp’s KV store, bundled with Consul’s service catalog, health checks, and (since Consul Connect) service mesh. The Vault of plain config plus a service-discovery DNS surface.

$ consul kv put feature/new-routing true
$ consul kv get feature/new-routing
$ consul kv export feature/

Hierarchical paths and watch streams make it the standard pick when the operator wants config plus discovery plus health checks in one daemon.

ZooKeeper

Apache’s older coordination service. Sequential ZNodes, ephemeral nodes, watches. Kafka, HBase, Hadoop, and Solr ran on it before each grew its own equivalent. Mostly legacy in 2026; operator sees it under long-lived data platforms.

Redis as config store

Common but lossy. No durability guarantees by default, no Raft, no built-in mTLS until 6+. Workable for feature-flag fanout when losing config briefly is acceptable; the operator should not treat it as authoritative state.

Secret Managers

The systems built specifically for credential storage. All of them ship audit logs, encryption at rest, fine-grained ACLs, versioning, and short-lived issuance.

HashiCorp Vault

The standard self-hosted secret manager. KV v2 for static secrets, plus secret engines that mint short-lived credentials on demand (PKI for x.509 certs, Database for ephemeral DB users, AWS / GCP / Azure for cloud IAM, SSH for one-time SSH keys).

$ vault kv put secret/web/db url=postgres://...
$ vault kv get secret/web/db
$ vault read database/creds/readonly         # generates ephemeral DB user
$ vault write pki/issue/web common_name=web.example.com ttl=72h

Auth methods include AppRole (service accounts), Kubernetes (token-reviewer), AWS / GCP / Azure (cloud IAM), OIDC, and TLS. Operator-built capability typically authenticates via the workload identity already in place.

AWS Secrets Manager

AWS-managed equivalent. Rotation built in for RDS, Redshift, DocumentDB credentials. Integrates with IAM for access control, CloudTrail for audit, and KMS for encryption.

$ aws secretsmanager create-secret --name prod/db --secret-string '{"url":"..."}'
$ aws secretsmanager get-secret-value --secret-id prod/db
$ aws secretsmanager rotate-secret --secret-id prod/db --rotation-lambda-arn ...

Pricing is per secret per month plus per API call; small accounts should use SSM Parameter Store (cheaper, less feature-rich) for non-rotating secrets.

Azure Key Vault

Azure’s managed equivalent. Three flavours: Secrets (strings), Keys (cryptographic operations against an HSM-backed key), and Certificates (issuance + renewal pipeline). Integrates with Entra ID for auth, Azure Monitor for audit.

$ az keyvault secret set --vault-name kv-prod --name db-url --value postgres://...
$ az keyvault secret show --vault-name kv-prod --name db-url

GCP Secret Manager

GCP’s managed equivalent. Versioned secret payloads, IAM-driven access, Cloud Audit Logs for every read.

$ echo -n "$value" | gcloud secrets create db-url --data-file=-
$ gcloud secrets versions access latest --secret=db-url

Doppler / Infisical / 1Password Secrets Automation

SaaS secret platforms aimed at developer workflows. CLI fetches environment, web UI for editing, references in CI / hosting providers. Right when the team needs a UI and is OK with a vendor in the secret path.

In-Cluster Patterns

The three ways secrets reach a Kubernetes pod without ending up in git in cleartext.

External Secrets Operator

A CRD-driven controller that reconciles ExternalSecret objects into native Secret objects, pulling cleartext from one of the secret managers above. The cleanest pattern for cloud-managed estates.

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata: { name: db, namespace: prod }
spec:
  refreshInterval: 1h
  secretStoreRef: { name: aws, kind: SecretStore }
  target: { name: db }
  data:
    - secretKey: url
      remoteRef: { key: prod/db, property: url }

Sealed Secrets

Bitnami’s controller encrypts a secret on the operator’s box and stores the encrypted blob in git. The cluster’s controller is the only thing that can decrypt it.

$ echo -n "password123" | kubectl create secret generic db --dry-run=client \
    --from-file=password=/dev/stdin -o yaml \
    | kubeseal --controller-name sealed-secrets-controller \
               --format yaml > db-sealed.yaml
$ git add db-sealed.yaml && git commit

The cluster needs a stable controller key; back it up like an HSM root.

SOPS-encrypted git

Mozilla’s SOPS encrypts YAML / JSON / env values in place with KMS, age, or PGP keys. Pairs with helm-secrets, Flux’s decryption.provider: sops, or Argo CD via the argocd-vault-plugin.

$ sops -e -i secrets.yaml
$ git add secrets.yaml
$ sops -d secrets.yaml          # local view

Picking One

Need	Pick
Service discovery + config + health	Consul.
Coordination behind a control plane	etcd.
Self-hosted secret manager with dynamic secrets	Vault.
Managed secret manager inside one cloud	AWS Secrets Manager / Azure Key Vault / GCP Secret Manager.
Secrets-from-git GitOps on Kubernetes	Sealed Secrets, SOPS, or External Secrets Operator pulling from a real manager.
Secrets across cloud + on-prem	Vault with the cloud-IAM auth methods.

Operator Practice

• Never commit cleartext secrets. The simplest rule, the hardest to enforce after the fact. git-secrets, trufflehog, and CI scanners catch most accidents.

• Short-lived over long-lived. Vault’s dynamic engines, AWS STS, GCP service-account impersonation. Static credentials are the worst kind to rotate.

• Audit every read. Every manager above logs reads; ship the log off-system and alert on anomalies.

• Break-glass account separate. A standing-by, sealed account for recovery when the standard auth path fails. Documented procedure to unseal it; rotated immediately after use.

• Test the recovery path. Backup is a guess; restore is the proof.

References

• Monitoring for audit-log destinations.

• Instrumentation for emitting secret-access events.

• Objects for the in-cluster Secret and ConfigMap objects.

• HashiCorp Vault

• etcd docs

• Consul docs

• External Secrets Operator

• Bitnami Sealed Secrets

• SOPS