## What it is

jq is a command-line JSON processor — it reads JSON on stdin, runs a query/transformation, and emits JSON (or raw text) on stdout. For deployment workflows that makes it the missing glue between every JSON-emitting tool in a CI pipeline: GitHub and GitLab webhooks, Docker registry manifests, `kubectl -o json`, AWS / GCP CLI outputs, Terraform state, health-check responses.

This sheet covers the filters and patterns you reach for when wiring jq into a deploy script — parsing webhook payloads to extract commit metadata, asserting on health-check responses with `--exit-status` so a bad response fails the build, and building deployment dashboards from log streams without an external service.

## Quick reference

### The identity filter and basic access

```bash
echo '{"name":"app","version":"1.2.3"}' | jq .          # pretty-print
echo '{"name":"app","version":"1.2.3"}' | jq -c .       # compact (one line)
echo '{"name":"app","version":"1.2.3"}' | jq '.name'    # → "app"
echo '{"name":"app","version":"1.2.3"}' | jq -r '.name' # → app  (raw — no quotes)

jq '.foo.bar.baz' file.json                             # nested access
jq '.foo?'                                              # OPTIONAL — null instead of error if missing
jq '."weird key with spaces"'                           # quote keys with special chars
jq '.foo, .bar'                                         # multiple outputs (separated by newline)
```

`-r` (raw output) is the most-used flag in shell scripts — it strips JSON string quotes so `jq -r .version` returns `1.2.3` instead of `"1.2.3"`, ready to interpolate into a shell command.

### Arrays

```bash
echo '[1,2,3,4,5]' | jq '.[0]'                          # → 1 (first element)
echo '[1,2,3,4,5]' | jq '.[-1]'                         # → 5 (last)
echo '[1,2,3,4,5]' | jq '.[1:3]'                        # → [2,3] (slice)
echo '[1,2,3,4,5]' | jq '.[]'                           # → 1\n2\n3\n4\n5 (stream)
echo '[1,2,3,4,5]' | jq 'length'                        # → 5
echo '[1,2,3,4,5]' | jq 'first'                         # → 1
echo '[1,2,3,4,5]' | jq 'last'                          # → 5
echo '[1,2,3,4,5]' | jq 'reverse'                       # → [5,4,3,2,1]
echo '[3,1,4,1,5,9]' | jq 'sort'                        # → [1,1,3,4,5,9]
echo '[3,1,4,1,5,9]' | jq 'unique'                      # → [1,3,4,5,9]
echo '[1,2,3]' | jq 'add'                               # → 6 (sum) or "abc" (concat)
echo '[1,2,3]' | jq 'min, max'                          # → 1, 3
```

### Objects

```bash
echo '{"a":1,"b":2,"c":3}' | jq 'keys'                  # → ["a","b","c"]
echo '{"a":1,"b":2,"c":3}' | jq 'values'                # → [1,2,3]
echo '{"a":1,"b":2,"c":3}' | jq 'length'                # → 3
echo '{"a":1,"b":2,"c":3}' | jq 'has("a")'              # → true
echo '{"a":1,"b":2,"c":3}' | jq 'to_entries'            # → [{"key":"a","value":1},...]
echo '[{"key":"a","value":1}]' | jq 'from_entries'      # → {"a":1}
echo '{"a":1,"b":2}' | jq 'del(.a)'                     # → {"b":2}
echo '{"a":1}' | jq '. + {b:2}'                         # → {"a":1,"b":2}  (merge)
echo '{"a":{"x":1}}' | jq '.a * {y:2}'                  # → {"a":{"x":1,"y":2}}  (recursive merge)
```

### Pipes, parens, and chaining

```bash
# `|` chains filters left-to-right
jq '.commits | length'                                  # count of commits
jq '.commits | .[0] | .sha'                             # first commit's sha
jq '.commits[0].sha'                                    # same — shorter

# Parens group filter expressions
jq '(.commits | length) > 0'                            # → true/false
```

### map, select, group_by

```bash
# map(f) — apply f to each element of an array, return an array
jq '[.users[] | .email]'                                # → ["a@b.com", "c@d.com"]
jq '.users | map(.email)'                               # equivalent, idiomatic

# select(pred) — keep only matching elements
jq '.users[] | select(.role == "admin")'                # stream of admin objects
jq '[.users[] | select(.active)]'                       # array of active users
jq 'map(select(.score > 50))'                           # filter then re-array

# group_by — sort + bucket on a key
jq 'group_by(.team)'                                    # → [[{team:"a",...}], [{team:"b",...}]]
jq 'group_by(.team) | map({team: .[0].team, count: length})'
```

### String operations

```bash
jq -r '.name | ascii_downcase'                          # lowercase
jq -r '.name | ascii_upcase'                            # UPPERCASE
jq -r '.path | split("/")'                              # → ["a","b","c"]  (array)
jq -r '.parts | join("-")'                              # → "a-b-c"
jq -r '.text | gsub(" "; "-")'                          # global substitute (regex)
jq -r '.text | test("error"; "i")'                      # regex test (case-insensitive)
jq -r '.text | match("foo(\\d+)")'                      # regex match — returns captures
jq -r '.text | ltrimstr("v")'                           # strip "v" prefix → 1.2.3
jq -r '.text | rtrimstr(".md")'                         # strip ".md" suffix
jq -r '"\(.tag) released at \(.ts)"'                    # string interpolation
```

`\(...)` is jq's string interpolation operator — much cleaner than concatenation for building log lines or deploy summaries.

### Conditionals

```bash
jq 'if .status == "ok" then "pass" else "fail" end'
jq 'if .count > 10 then "many" elif .count > 0 then "some" else "none" end'
jq '.status // "unknown"'                               # null/false coalescing
jq '.user.email // .user.username // "anon"'            # fallback chain
```

### Recursive descent

```bash
jq '..'                                                 # stream EVERY value (deep)
jq '..|.id?'                                            # every "id" anywhere in the tree
jq '..|select(type == "string") | length' | sort -nu    # length of every string
```

### Variables and `as`

```bash
jq '. as $r | $r.commits | length'                      # bind whole input to $r
jq '.commits[0] as $first | {sha: $first.sha, msg: $first.message}'

# CLI-bound variables (most useful pattern):
jq --arg env "production" '.envs[$env]'                  # string variable
jq --argjson port 8080 '.services[] | select(.port == $port)'   # JSON-typed variable
```

`--arg` always passes the value as a string; `--argjson` parses the value as JSON (so numbers stay numbers, booleans stay booleans).

### Output formatting

```bash
jq -c .                                                  # compact (single-line per top-level value)
jq -r .                                                  # raw — strip outer string quotes
jq --tab .                                               # tabs instead of 2 spaces
jq --indent 4 .                                          # 4-space indent
jq -s .                                                  # SLURP — wrap all inputs in one array
jq -n .                                                  # NULL input — useful with --arg-driven generation
jq -S .                                                  # SORT keys alphabetically
jq --raw-input .                                         # treat input as a string, not JSON
jq -e .                                                  # exit-status — non-zero on false/null (see workflow 4)
```

`--slurp` is the magic flag for concatenating multiple JSON files: `jq -s '.[0] * .[1]' a.json b.json` reads both files into an array and merges them.

### Reading from files and HTTP

```bash
jq .              file.json                              # read file
cat *.json | jq -s 'add'                                 # concat array of arrays via slurp + add
curl -fsS https://api.example.com/status | jq '.'

# Multiple files — process each independently
for f in *.json; do
  jq -r --arg file "$f" '"\($file): \(.status)"' "$f"
done
```

---

## Deployment workflows (the moat)

### 1. Parsing GitHub and GitLab webhook payloads

The shape that comes out of a webhook is sprawling JSON — most of which you don't care about. The bits a deploy script actually needs:

```bash
# GitHub push event — extract ref, sha, author, repo
PAYLOAD=$(cat webhook.json)

REF=$(jq -r '.ref'                              <<< "$PAYLOAD")   # refs/heads/main
SHA=$(jq -r '.after'                            <<< "$PAYLOAD")   # commit SHA
BRANCH=$(jq -r '.ref | sub("refs/heads/"; "")'  <<< "$PAYLOAD")   # main
AUTHOR=$(jq -r '.head_commit.author.username'   <<< "$PAYLOAD")
REPO=$(jq -r '.repository.full_name'            <<< "$PAYLOAD")   # org/repo

# Filter — only deploy main and tags
case "$REF" in
  refs/heads/main)        ENV=production ;;
  refs/heads/staging)     ENV=staging    ;;
  refs/tags/v*)           ENV=production ;;
  *) echo "skipping non-deployable ref $REF"; exit 0 ;;
esac
```

For GitLab — different field names, same shape:

```bash
SHA=$(jq -r '.checkout_sha'                  <<< "$PAYLOAD")
REF=$(jq -r '.ref'                           <<< "$PAYLOAD")
AUTHOR=$(jq -r '.user_username'              <<< "$PAYLOAD")
REPO=$(jq -r '.project.path_with_namespace'  <<< "$PAYLOAD")
```

The two patterns to internalise:

1. **`-r` on every leaf access**, so values land in shell variables ready to use.
2. **`sub(pattern; replacement)`** for parsing prefixed values like `refs/heads/main` into `main` inline.

### 2. Extracting commit SHAs and tags from CI JSON

CI providers all expose JSON state. Common patterns:

```bash
# GitHub Actions — list all jobs in a workflow run, find failed ones
gh run view --json jobs 12345 \
  | jq -r '.jobs[] | select(.conclusion == "failure") | .name'

# Get the latest release tag
gh release list --limit 1 --json tagName \
  | jq -r '.[0].tagName'

# AWS — extract the running task's image tag from ECS
aws ecs describe-tasks --tasks "$TASK_ARN" --cluster "$CLUSTER" \
  | jq -r '.tasks[0].containers[] | select(.name == "app") | .image' \
  | awk -F: '{print $2}'

# Docker registry — find the digest for a specific tag
curl -fsS "https://hub.docker.com/v2/repositories/org/app/tags/v1.2.3" \
  | jq -r '.digest'

# kubectl — get the image of a running deployment
kubectl get deployment app -o json \
  | jq -r '.spec.template.spec.containers[] | select(.name=="app") | .image'
```

### 3. Health-check assertions with `--exit-status`

`-e` (or `--exit-status`) makes jq exit non-zero when the final value is `false` or `null` — turning every JSON assertion into a CI gate:

```bash
#!/usr/bin/env bash
set -euo pipefail

curl -fsS http://127.0.0.1/_internal/health > /tmp/health.json

# Each line FAILS the deploy if the assertion is false
jq -e '.status == "ok"'        /tmp/health.json
jq -e '.db == true'            /tmp/health.json
jq -e '.cache == true'         /tmp/health.json
jq -e '.queue == true'         /tmp/health.json
jq -e '.version | startswith("'"$EXPECTED_VERSION"'")' /tmp/health.json
jq -e '.uptime_seconds < 60'   /tmp/health.json        # confirm we just restarted
```

Each line is a separate assertion with its own exit code, so the failure message identifies *which* check broke. Compare to the alternative of building one giant `and`-chained expression — when it fails you get "exit 1" with no idea which field tripped it.

Combine with retries to handle the cold-start window:

```bash
for attempt in 1 2 3 4 5 6 7 8 9 10; do
  if curl -fsS -m 5 http://127.0.0.1/_internal/health > /tmp/health.json \
       && jq -e '.status == "ok"' /tmp/health.json; then
    echo "healthy after $attempt attempts"
    break
  fi
  sleep 2
done

jq -e '.status == "ok"' /tmp/health.json \
  || { echo "deploy health check failed"; cat /tmp/health.json; exit 1; }
```

### 4. Filtering deploy logs for failure signals

If your app emits structured JSON logs (it should), jq is the right tool to triage a deploy that "feels off":

```bash
# Show only errors from the last hour
journalctl -u myapp --since "1 hour ago" -o json \
  | jq -c 'select(.PRIORITY <= "3")'

# Distribution of HTTP status codes in the last 10 minutes
kubectl logs deploy/app --since=10m \
  | jq -r 'select(.type=="http") | .status' \
  | sort | uniq -c | sort -rn

# Top 10 slowest requests in the last release window
jq -r 'select(.duration_ms > 500) | "\(.duration_ms)\t\(.path)"' app.log \
  | sort -rn | head

# Count of 5xx responses per minute
jq -r 'select(.status >= 500) | .ts | sub(":[0-9]+\\..*$"; "")' app.log \
  | sort | uniq -c
```

The pattern: filter rows with `select(...)`, project the columns you want with `\(...)` interpolation, then let the unix toolbox (`sort`, `uniq -c`, `awk`) handle the aggregation. jq is excellent at *one record at a time*; it's deliberately not a SQL engine.

### 5. Building a deploy summary for chat / status pages

After a successful deploy, post a short structured summary to Slack / Discord / a status page. jq is the formatter:

```bash
#!/usr/bin/env bash
set -euo pipefail

SHA="$1"
ENV="$2"
DEPLOYED_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)

# Pull commit metadata from GitHub API
gh api "repos/${REPO}/commits/${SHA}" > /tmp/commit.json

# Build the Slack message body in jq — no template heredoc, no escaping hell
jq -n \
  --arg env "$ENV" \
  --arg sha "$SHA" \
  --arg at "$DEPLOYED_AT" \
  --slurpfile commit /tmp/commit.json \
  '{
    text: "Deploy to \($env) succeeded",
    blocks: [
      { type: "section",
        text: { type: "mrkdwn",
                text: "*Deploy to `\($env)` ✓*\n`\($sha[0:8])` by *\($commit[0].author.login)*\n_\($commit[0].commit.message | split("\n")[0])_"
              }
      },
      { type: "context",
        elements: [
          { type: "mrkdwn", text: "Deployed at \($at)" }
        ]
      }
    ]
  }' > /tmp/slack-body.json

curl -fsS -X POST "$SLACK_WEBHOOK" \
  -H "Content-Type: application/json" \
  -d @/tmp/slack-body.json
```

`jq -n` (null input) plus `--arg` and `--slurpfile` is the cleanest way to compose JSON in a shell script — no quote-escaping, no `printf` heredocs, every value passes through jq's own JSON encoder.

### 6. Diffing two JSON files in a deploy gate

Useful when you want to assert "this deploy doesn't change the schema in an unexpected way":

```bash
# Diff two API contract files — bail if any endpoint was removed
jq -S . old-openapi.json > /tmp/old.json
jq -S . new-openapi.json > /tmp/new.json

REMOVED=$(jq -r '.paths | keys[]' /tmp/old.json | sort > /tmp/old-paths)
NEW_PATHS=$(jq -r '.paths | keys[]' /tmp/new.json | sort > /tmp/new-paths)

if comm -23 /tmp/old-paths /tmp/new-paths | grep -q .; then
  echo "::error::API endpoints removed:"
  comm -23 /tmp/old-paths /tmp/new-paths
  exit 1
fi
```

`-S` (sort keys) is critical — without it, two equivalent JSON files diff every time because Ruby and Python dump keys in different orders.

---

## Common errors and fixes

| Error / symptom | Cause | Fix |
|---|---|---|
| `jq: error (at <stdin>:1): Cannot index string with string "foo"` | Tried `.foo` on a non-object | Check upstream — usually the response is `null` or an error object |
| `jq: error: syntax error, unexpected '['` | Forgot to quote the filter in shell | Always quote: `jq '.foo[0]'` not `jq .foo[0]` |
| Output has surrounding quotes when piping to next command | Default JSON output preserves type | Add `-r` for raw output |
| `jq -e '.x == "y"'` exits 0 but the value is null | `null == "y"` is `false`, but `false` exits 1 — `null` and the no-data case both exit 1 with `-e` | Use `jq -e '.x // empty | . == "y"'` to make absence vs mismatch distinguishable |
| `Cannot iterate over null` on `.items[]` | The array doesn't exist in the input | Use `.items?[]` — the `?` swallows the error and returns nothing |
| Multiple JSON objects in input crash with parser error | jq parses one value by default; needs `--slurp` for arrays or stream mode | `jq -s '.'` reads all inputs into an array; `jq -c '.'` is fine for streams of objects |
| `--arg name 123` makes `$name` the string `"123"` not the number | `--arg` is always string-typed | Use `--argjson name 123` for non-string values |
| jq 1.6 vs 1.7 behaviour differences | Older Linux distros pin to 1.6; 1.7 added new builtins | Install jq 1.7+ explicitly; document in `composer.json`/`package.json` system requirements |
| Regex doesn't match in `test` / `match` | jq uses Oniguruma regex — backslashes need double-escaping in JSON | Use `'.x | test("foo\\d+")'` (two backslashes in single-quoted shell) |
| `gsub(".*"; "X")` replaces nothing | `gsub` is greedy by default; pattern anchored to start | Quote pattern correctly; use `sub` for first match, `gsub` for global |
| Output piped to bash drops trailing newline | jq always emits trailing newline; some `read` patterns truncate | Use `mapfile -t arr < <(jq -r '.[]')` not `while read` |
| `Cannot iterate over object` on `.[]` | Object iteration is `[]` too — but `.[]` on an object emits values, on an array emits elements; on null/string/number it errors | Wrap with `select(type == "array")` to skip non-arrays |
| jq hangs on large file | Default mode reads input into memory; can be slow on multi-GB | Use `--stream` for event-driven parsing of huge files |

---

## Companion: full DeployHQ deploy workflow

jq is the connective tissue of a modern deploy pipeline — the bit that reads a webhook payload, asserts a health response, formats a Slack notification, and parses a registry manifest. None of those steps individually justifies its own tool; together they're 80% of what makes a CI/CD pipeline observable and gate-able.

Wire these patterns into [DeployHQ's build pipelines](https://www.deployhq.com/features/build-pipelines) — the deploy script (Bash + jq) lives in your repo, runs on the build server, and produces the artifacts that get atomically released to production. See the [deploy from GitHub guide](https://www.deployhq.com/deploy-from-github) for the webhook → deploy flow that this sheet's workflow 1 plugs into.

[Start a free DeployHQ trial](https://www.deployhq.com/signup) to wire jq-based health gates and webhook parsing into your deploy pipeline.

---

## Related cheatsheets

- [curl cheatsheet](https://www.deployhq.com/cheatsheets/curl) — for fetching the JSON payloads that jq then parses.
- [Bash cheatsheet](https://www.deployhq.com/cheatsheets/bash) — for the `set -euo pipefail` scripts that orchestrate the curl-jq pipelines above.
- [Cron and Crontab cheatsheet](https://www.deployhq.com/cheatsheets/cron) — for the scheduled jobs that emit and consume JSON state.
- [Docker cheatsheet](https://www.deployhq.com/cheatsheets/docker) — for the registry manifest JSON that jq extracts image digests from.
- [AWS CLI cheatsheet](https://www.deployhq.com/cheatsheets/aws-cli) — for the cloud-provider JSON outputs (`ecs describe-tasks`, `s3api`, `cloudformation describe-stacks`) that jq filters in the workflows above.
- [Laravel Artisan cheatsheet](https://www.deployhq.com/cheatsheets/laravel-artisan) — for the JSON health endpoint pattern that pairs with the `-e` assertions.
- [Cheatsheets hub](https://www.deployhq.com/cheatsheets) — every DeployHQ cheatsheet in one place.

---

Need help? Email [support@deployhq.com](mailto:support@deployhq.com) or follow [@deployhq on X](https://x.com/deployhq).