sed

sed#

sed, the stream editor, applies one or more editing commands to a stream of text, line by line. Designed by Lee McMahon at Bell Labs in 1973. POSIX-standardized; available everywhere.

In daily use, sed is mostly the substitute-and-print one-liner. It’s also a fully programmable Turing-complete editor; nobody should do that.

The Substitute Command#

The 95% case.

$ sed 's/old/new/' file

$ sed 's/old/new/g' file

$ sed -i 's/old/new/g' file

$ sed -i '' 's/old/new/g' file

$ sed 's|/old/path|/new/path|g' file

$ sed -E 's/^([^,]+),(.*)$/\2,\1/' file

Flags#

The trailing modifiers on a substitute command. g is the near-universal one, replace all occurrences on the line, not just the first. The rest tune case sensitivity, narrow the match to a specific occurrence, or print and write only on successful substitution.

Flag

Effect

g

replace all (not just first)

i / I

case-insensitive

N

replace only the Nth match (number)

p

print the line if a substitution happened

w file

write to file if substitution happened

Address Ranges#

Most sed commands take an address that scopes the operation to a subset of input. Line numbers, regex matches, ranges between two patterns, and the $ for last line all work – plus a leading ! to invert the range. The address is what turns sed from substitute-only into a slicing tool.

Apply a command only to certain lines.

$ sed -n '5,10p' file
$ sed '5,10d' file
$ sed '/^#/d' file
$ sed '/BEGIN/,/END/p' file
$ sed -n '$p' file
$ sed '1!d' file

Common One-Liners#

The handful of sed invocations that operators reach for over and over, stripping carriage returns, trimming trailing whitespace, deleting blank lines, expanding tabs, inserting or appending content, slicing N lines after a match. Worth recognizing on sight from a colleague’s terminal.

$ sed 's/\r$//' file

$ sed 's/[[:space:]]*$//' file

$ sed = file | sed 'N; s/\n/ /'

$ sed -n '/pattern/p' file

$ sed '/pattern/ s/old/new/' file

$ sed '/pattern/i\
prepended line' file
$ sed '/pattern/a\
appended line'  file

$ sed '/^$/d' file

$ sed '/^$/N;/^\n$/D' file

$ sed -n '/match/,+5p' file

$ sed 's/\t/    /g' file

Multiple Commands#

$ sed -e 's/foo/bar/g' -e 's/baz/qux/g' file

$ sed 's/foo/bar/g; s/baz/qux/g' file

$ sed -f script.sed file

Captures and Backreferences#

$ sed -E 's/([0-9]+)/[\1]/g' file

$ sed -E 's/^(\S+)\s+(\S+)$/\2 \1/' file

$ sed 's/[0-9]\+/<&>/g' file

GNU sed vs. BSD sed#

The two implementations diverge in real ways.

  • -i, GNU: sed -i 's/x/y/' file. BSD: sed -i '' 's/x/y/' file.

  • -E (extended regex), both support it now; -r is the GNU legacy alias.

  • \d, works in neither in the basic regex; use [0-9].

  • \b (word boundary), GNU yes; BSD no.

  • a, i, c commands, GNU allows on the same line; BSD often needs newlines or backslash continuations.

  • -z (null-delimited records), GNU only.

When portability matters, write a tiny script file rather than fighting the differences inline.

Beyond Substitution#

sed has more commands (d (delete), p (print), y (transliterate, like tr), b (branch), t (test), h / H / g / G (hold space), n / N (advance)) and a separate “hold space” alongside the pattern space.

You can write fizzbuzz, sort numbers, even arithmetic in sed. Don’t. When the script needs hold space, switch to AWK or a real language.

Modern Alternatives#

The tools that solve the same problems sed does, often more ergonomically. sd brings PCRE syntax and a simpler CLI; perl -pe gives the full regex feature set; rg --replace reuses ripgrep’s matching; awk takes over once the logic exceeds a single substitution.

  • sd, sed-like with PCRE syntax; much friendlier; doesn’t try to be Turing-complete.

  • perl -pe 's/.../.../g', when you need real regex.

  • ripgrep --replace, rg can do many sed-like substitutions.

  • awk, when the logic is more than a substitution.

When to Use sed#

The kinds of work where sed is the right pick. Quick substitutions, in-place edits on config files, simple pattern-tied deletions or inserts, anywhere a Python script for one .replace call would be overkill. When the script grows past a few lines, switch tools.

  • Single-line substitutions on a stream.

  • In-place edits on config files in scripts.

  • Simple deletions / insertions tied to patterns.

  • Anything where the alternative is invoking a Python script for one .replace call.

When the sed script grows past 5 lines, you have probably outgrown sed.

Pitfalls#

The traps that catch sed users at least once. Greedy-only matching, shell quoting, newline handling in replacements, the GNU/BSD -i divergence, and locale-dependent character classes all bite at different times. Each has a clean workaround once the cause is recognized.

  • Greedy vs. lazy, BRE / ERE don’t have *?; .* is always greedy.

  • Quoting in shell, single-quote sed scripts to avoid the shell expanding $1 etc.

  • Newlines in replacements, need a literal newline preceded by \; the syntax is unforgiving.

  • GNU vs. BSD ``-i``, the most common cross-platform mistake; pin a portable invocation or use perl -pi.

  • Locale-dependent character classes, [a-z] may include accented characters in some locales; use [[:lower:]] or LC_ALL=C.

Contents