jq

jq#

jq is the standard CLI and language for querying and transforming JSON. It’s also a real programming language – filters compose, you can define functions, and the data model has rich operations.

The 90% You’ll Use#

A starter library of jq filters that covers most everyday work; selecting fields, iterating arrays, filtering by predicate, mapping, sorting, grouping, and counting. Each is short enough to type from memory and powerful enough to replace a one-off Python script.

$ jq '.'
$ jq '.name'
$ jq '.users[0]'
$ jq '.users[]'
$ jq '.users[] | .name'
$ jq '.users[] | select(.active)'
$ jq '.users | length'
$ jq '.users | map(.email)'
$ jq '.users | sort_by(.age)'
$ jq '.users | group_by(.team)'
$ jq -r '.name'

Filter Pipelines#

Filters compose left-to-right with |:

$ curl -fsS https://api.example.com/users \
  | jq '.[] | select(.role == "admin") | {id, name, email}'

Object Construction#

$ jq '{name: .name, count: (.items | length)}'
$ jq '.users | map({id, name: (.first + " " + .last)})'

A bare identifier in object syntax is shorthand for {name: .name}.

Array Construction#

$ jq '[.users[].email]'
$ jq '[.x, .y, .z]'

Square brackets collect a stream of values into an array.

Updating#

jq is not just a query language; the assignment operators let one filter rewrite a JSON document in place. Setting a field, mutating with a function via |=, deleting keys, and merging objects with + cover the everyday update forms operators reach for.

$ jq '.user.name = "operator"'
$ jq '.user.age |= . + 1'
$ jq 'del(.password)'
$ jq '. + {ts: now}'
$ jq '.users |= map(. + {active: true})'

Recursion and Walking#

$ jq '..'
$ jq 'paths'
$ jq 'walk(if type == "string" then ascii_downcase else . end)'

Reduce and Foreach#

$ jq '[.orders[].amount] | add'

$ jq 'reduce .orders[] as $o (0; . + $o.amount)'

$ jq 'group_by(.tenant) | map({tenant: .[0].tenant, total: ([.[].amount] | add)})'

Variables and Functions#

$ jq '[.users[] | . as $u | $u.email]'

$ jq 'def slug: ascii_downcase | gsub(" "; "-"); .name | slug'

Streaming#

For files larger than memory.

$ jq --stream '. | select(length == 2 and .[0][0] == "users")' big.json

Stream mode emits paths and values incrementally so jq doesn’t load the whole document.

Common One-Liners#

A short library of jq invocations worth keeping near the shell. CSV projection, pretty-printing versus compact output, NDJSON enrichment, two-file diffs, and shell-variable parameterization; each is the kind of pattern that makes jq feel like a Swiss-army tool.

$ jq -r '.[] | [.id, .name] | @csv'

$ jq '.'
$ jq -c '.'

$ jq -c '. + {ts: now}' input.ndjson > output.ndjson

$ jq -n --argfile a a.json --argfile b b.json '$a - $b'

$ jq --arg name "$NAME" '.name = $name'

Alternatives#

The other CLIs an operator may meet alongside or instead of jq. gojq and jaq are reimplementations with different performance trade-offs; JMESPath is a separate query language in cloud SDKs; gron, dasel, and yq round out the ecosystem with adjacent and complementary capabilities.

  • gojq, pure-Go reimplementation; drop-in for most uses.

  • jaq, Rust reimplementation; faster on most workloads.

  • JMESPath, a different JSON query language used by AWS CLI --query, Azure CLI, and many SDKs:

    users[?role=='admin'].{name: name, email: email}

  • gron, flatten JSON to greppable . paths; complements jq for ad-hoc exploration.

  • dasel, multi-format query (JSON / YAML / TOML / XML).

  • yq, jq for YAML.

JSONPath#

Another JSON query DSL, originally inspired by XPath.

$.store.book[*].author          # all authors
$..author                       # any author at any depth
$..book[?(@.price < 10)]        # filter
$..book[0,1]                    # first two

Used by Kubernetes (kubectl -o jsonpath=...), some monitoring tools, and Postman tests. Less expressive than jq; more standardized in RFC 9535 (2024).

When to Reach for jq#

The kinds of work where jq is the right tool. Ad-hoc exploration, shell-script glue, JSON-to-CSV projection, and quick reshaping between API contracts are all sweet spots. Application code wants the runtime’s own JSON library; truly large datasets want a real query engine.

  • Ad-hoc JSON exploration on the command line.

  • Shell scripts that consume JSON APIs.

  • Quick transformations between JSON structures.

  • CSV / NDJSON ↔ JSON conversions.

When not to.

  • Application code, use the language’s JSON parser plus normal code.

  • Very large datasets where streaming alternatives (Spark, ClickHouse, DuckDB) make more sense.

Contents