Traditional Ops#
Before “DevOps” was a word, infrastructure ran under a separate team of system administrators with its own queue, its own tooling, and its own culture. Developers finished code and threw the tarball over the wall; ops caught it, deployed it during a maintenance window, and lived with whatever it broke. Every practice the operator now treats as normal (CI/CD, IaC, observability, immutable images, blameless postmortems) is a direct reaction to a failure mode of this era. The operator who understands the pain understands why the tooling is shaped the way it is.
Infrastructure#
The estate the sysadmin inherited.
Bare-metal servers bought, racked, cabled, and named by hand.
Snowflakes, each box configured slightly differently over the years through patches, manual fixes, and undocumented tweaks.
Manual deploys, SSH in, drop files, restart services, usually during a scheduled maintenance window.
Pets, servers had names (
apollo,zeus,mercury); when one was sick the team nursed it back to health rather than replace it.Tickets, developers requested resources from ops through a queue with lead times measured in days or weeks.
Word-doc runbooks, documentation scattered across shared drives, inconsistently updated, often stale by the time the on-call needed them.
Tools#
Many of these are still in production today. What changed is the workflow around them, not the tools themselves.
Tool |
Role |
|---|---|
|
The fundamental remote-access primitive. |
Bash |
Imperative automation scripts. |
Perl |
Imperative automation scripts. |
Python |
Imperative automation scripts. |
|
File distribution between hosts. |
|
File distribution between hosts. |
|
Scheduling. |
|
Centralised logging, when anyone bothered to set it up. |
Nagios |
Uptime and threshold monitoring. |
Zabbix |
Uptime and threshold monitoring. |
VLAN |
Network segmentation in the switch fabric. |
Physical firewall |
Network segmentation at the perimeter. |
F5 |
Hardware load balancer. |
Citrix NetScaler |
Hardware load balancer. |
Tape |
Backups, with the retrieval drill nobody ever practised. |
Pain#
The failure modes that produced everything that followed.
Pain |
Shape |
|---|---|
“Works on my machine” |
Dev and prod environments diverged. |
Long lead times |
New server in 2 weeks; new firewall rule in 3 days. |
Outages from drift |
A “fix” applied to one server but not another caused mysterious bugs. |
Heroic culture |
The team kept the systems running through effort, not design. |
Bus factor |
The one engineer who knew how the mail server was configured ran the company. |
Coordination overhead |
Changes touched many teams; release windows compressed risk. |
Burnout |
On-call was night-and-day pages with little automation. |
The DevOps movement (c.2009) crystallised as the reaction to this list.
Survivors#
The era was not wrong about everything. Practices the operator still uses, in updated form.
Practice |
Status |
|---|---|
Postmortems and runbooks |
The form is more rigorous now, but the practice predates DevOps. |
Capacity planning |
Still required; the math just runs against cloud instance types instead of physical hardware. |
On-call rotations |
Still the model; better tooling makes them more sustainable. |
Changes in maintenance windows |
Still appropriate for high-risk changes (database migrations, network re-cabling). |
Replacements#
The before/after pairs that define the shift.
Then |
Now |
|---|---|
Snowflake servers |
Immutable images plus reconciliation. |
Manual deploys |
CI/CD pipelines with automated rollback. |
Tickets for resources |
IaC plus self-service. |
Hand-tuned monitoring |
Observability (metrics, logs, traces). |
Pets |
Cattle. Replace, do not repair. |
Word-doc runbooks |
Versioned runbooks in the same repo as the code. |
Tape and the retrieval drill |
Object-store backups with restore tests in CI. |
Holdouts#
Plenty of estates still run this way, on purpose.
Banks on mainframes and AIX where the migration math never closes.
Hospitals with vendor-supplied appliance servers that ship as black boxes.
Manufacturing floors with industrial control systems on hardened Windows.
Government environments where change-control gates exist by policy, not preference.
Telecom carriers running custom hardware with carrier-grade uptime requirements.
Long-lived internal apps nobody has the budget to migrate.
These environments are not broken. They carry different failure modes and different risk tolerances, and the operator who walks in expecting cloud-native idioms will misread the room. Treat the estate on its own terms.
Culture#
The technical changes get the press; the cultural shift was the larger move.
From |
To |
|---|---|
“Dev throws code, Ops catches it” |
“You build it, you run it” (Werner Vogels, AWS). |
Silos |
Shared on-call. |
Change-aversion |
Change-velocity (with safety built into automation). |
“Infrastructure is somebody else’s problem” |
“Infrastructure is part of the application”. |
The shift in ownership, not the tools, is what made everything that follows possible.
References#
man 1 ssh,man 1 rsync,man 1 cron,man 5 syslog.conf.Virtualization for the abstraction that ended the bare-metal era.
Containers for the unit of deployment that replaced the snowflake.
DevOps for the practices that grew out of this era’s pain.
The DevOps Handbook (Gene Kim, Jez Humble, Patrick Debois, John Willis)