jq Cheatsheet

Common Recipes

Use this jq reference while you build software engineering projects, review code, or refresh the syntax you reach for most.

Reshape an API response

The most common job: take a big response and emit exactly the object you want.

jq -c '{total: (.items | length),
        names: [.items[].name],
        cheap: [.items[] | select(.price < 25) | .id]}' data.json
# {"total":3,"names":["a","b","c"],"cheap":[1,2]}

# rename and drop fields in one pass
jq -c '.items | map({id, label: .name, price})' data.json
# [{"id":1,"label":"a","price":10},...]

# add a derived field to every element
jq -c '.items | map(. + {tax: (.price * 0.1)})' data.json
# [{"id":1,...,"tax":1},{"id":2,...,"tax":2},...]

# group and total
jq -c '.items | group_by(.cat)
       | map({cat: .[0].cat, total: (map(.price) | add)})' data.json
# [{"cat":"x","total":40},{"cat":"y","total":20}]

# group into an object keyed by the grouping value
jq -c '.items | group_by(.cat)
       | map({key: .[0].cat, value: [.[].id]}) | from_entries' data.json
# {"x":[1,3],"y":[2]}

NDJSON and log lines

jq reads a stream of values, so newline-delimited JSON needs no flag at all. Given app.ndjson:

{"ts":"...:00Z","level":"info","msg":"start","dur":5}
{"ts":"...:01Z","level":"error","msg":"boom","dur":50}
{"ts":"...:02Z","level":"error","msg":"boom","dur":70}

# filter lines, emit lines
jq -c 'select(.level == "error") | {ts, msg}' app.ndjson

# tally a field without holding the file in memory
jq -nc 'reduce inputs.level as $l ({}; .[$l] += 1)' app.ndjson
# {"info":1,"error":2}

# slurp only when you need to sort or average across every line
jq -sc '[.[].dur] | add / length' app.ndjson      # 41.666666666666664
jq -sc 'group_by(.msg) | map({msg: .[0].msg, n: length})
        | sort_by(-.n) | .[0]' app.ndjson         # {"msg":"boom","n":2}

# which line was this
jq -c '{n: input_line_number, msg}' app.ndjson    # {"n":1,...} ...

# and back the other way, array to NDJSON
jq -c '.items[]' data.json

-n with inputs processes one value at a time. -s builds the entire array first, so it is the one to avoid on a large log.

Rows out, CSV, TSV, and shell loops

jq -r '.items[] | [.id, .name, .price] | @csv' data.json
# "1","a",10  ... note strings are quoted, numbers are not

# a header row, then the data
jq -r '(.items[0] | keys_unsorted), (.items[] | [.[]]) | @tsv' data.json

# ragged objects, using the union of all keys as the header
jq -r '.items | (map(keys) | add | unique) as $k
       | ($k | @csv), (.[] | [.[$k[]]] | @csv)' data.json

Feed a shell loop with @tsv, never with unquoted word splitting:

jq -r '.items[] | [.id, .name] | @tsv' data.json |
while IFS=$'\t' read -r id name; do
  echo "id=$id name=$name"
done

When a value may itself contain tabs or newlines, pass whole rows as base64:

for row in $(jq -r '.items[] | @base64' data.json); do
  echo "$row" | base64 --decode | jq -r '"\(.id):\(.name)"'
done
# 1:a  2:b  3:c

@sh quotes for the shell, which makes a small config safe to eval:

eval "$(jq -r '.meta | to_entries[] | "\(.key)=\(.value | @sh)"' data.json)"
echo "$count"        # 3

Edit a file in place

jq has no in-place flag. Redirecting to the input file truncates it before jq reads a byte, so jq '.' f.json > f.json leaves you with 0 bytes. Always write elsewhere and move:

tmp=$(mktemp)
jq '.meta.count += 1' data.json > "$tmp" && mv "$tmp" data.json

jq '.version = "9.9.9"' pkg.json > t && mv t pkg.json
jq 'del(.devDependencies)' pkg.json > t && mv t pkg.json

The && matters. If jq fails, the move never runs and the original survives.

Merge and diff two documents

jq -nc '{x:{y:1,z:2}} + {x:{y:9}}'    # {"x":{"y":9}}, shallow, right wins
jq -nc '{x:{y:1,z:2}} * {x:{y:9}}'    # {"x":{"y":9,"z":2}}, recursive

+ replaces a key outright. * descends into objects and merges leaf by leaf, which is what a config override wants.

# base.json {"port":80,"opts":{"a":1,"b":2}}
# over.json {"port":8080,"opts":{"b":9}}
jq -sc '.[0] * .[1]' base.json over.json
# {"port":8080,"opts":{"a":1,"b":9}}

# which top-level keys differ
jq -nc --slurpfile a base.json --slurpfile b over.json \
  '[($a[0] | keys[]) | select($a[0][.] != $b[0][.])]'    # ["opts","port"]

# keys only the first document has
jq -nc --slurpfile a base.json --slurpfile b over.json \
  '($a[0] | keys) - ($b[0] | keys)'

# a readable structural diff, key order normalized
diff <(jq -S . base.json) <(jq -S . over.json)

Survive ragged data

# one default covers missing and null alike
printf '{"a":1}\n{"a":null}\n{}\n' | jq -c '{a: (.a // 0)}'
# {"a":1} {"a":0} {"a":0}

# strip null-valued keys at every depth
echo '{"a":1,"b":null,"c":{"d":null,"e":2}}' |
  jq -c 'walk(if type == "object"
              then with_entries(select(.value != null)) else . end)'
# {"a":1,"c":{"e":2}}

# find a field wherever it hides, skipping every wrong-typed branch
jq -c '[.. | .price? // empty]' data.json          # [10,20,30]

# map over items that may not be an array
jq -c '[.items[]? | .id]' data.json                # [1,2,3], no error if absent

To learn the shape of a document you have never seen, print its leaves:

jq -r 'paths(scalars) as $p | "\($p | join(".")) = \(getpath($p))"' data.json
# name = api
# tags.0 = web
# items.0.id = 1  ...

Several files, and programs in a file

jq -nc '[inputs | input_filename]' data.json pkg.json
# ["data.json","pkg.json"]
jq -sc 'map(.name)' data.json pkg.json               # ["api","app"]
jq -nc 'reduce inputs as $d ({}; . + {($d.name): true})' data.json pkg.json
# {"api":true,"app":true}

Once a filter outgrows a shell line, put it in a file and drop the quoting problem entirely. Comments start with #.

printf '# how many items\n.items | length\n' > count.jq
jq -f count.jq data.json      # 3

Use jq -e when the answer is a decision rather than data. It exits 1 on false or null and 4 when nothing was output, so a shell if reads well:

if jq -e '.items | length > 0' data.json > /dev/null; then echo "has items"; fi