## What it is

Linux commands are the core toolkit every deploy runs on — the same `ls`, `grep`, `chmod`, `systemctl`, and `journalctl` that live on the build runner also live on the production host, and the moment a release fails you're driving them by hand over SSH. Most cheatsheets dump these alphabetically; this one groups them by the task you actually have open when something breaks.

The reference below is the 80% surface — files, processes, permissions, disk, networking, logs — organized so you can find a command by what you're trying to do. The section after it reframes the same commands as runnable playbooks for the six situations that come up over and over in deploy ops: a service that won't start, logs you need to read fast, permissions broken after a sync, disk pressure mid-build, a port that's still held, and a rollback.

## Quick reference

### Files and navigation

```bash
pwd                                    # print working directory
ls -lah                                # long list, human sizes, incl. dotfiles
cd -                                   # jump back to previous directory
cp -a src/ dst/                        # copy preserving perms, timestamps, symlinks
mv old new                             # move / rename
rm -rf build/                          # remove recursively (no undo — check twice)
mkdir -p a/b/c                         # create nested dirs, no error if exists
ln -sfn /var/www/releases/42 /var/www/current   # atomic symlink swap (force + no-deref)
find /var/www -name '*.log' -mtime +7  # files matching pattern, older than 7 days
find . -type f -size +100M             # files over 100 MB
readlink -f /var/www/current           # resolve a symlink to its real target
```

`ln -sfn` is the workhorse of atomic releases: `-f` replaces the existing link, `-n` treats the destination as a plain file (not "follow into the directory it points at"). Without `-n` you re-point the wrong target on the second deploy.

### Viewing and searching

```bash
cat app.log                            # dump whole file
less app.log                           # pager (/ to search, F to follow, q to quit)
head -n 50 app.log                     # first 50 lines
tail -n 100 app.log                    # last 100 lines
tail -f app.log                        # follow live (Ctrl-C to stop)
tail -F app.log                        # follow across log rotation (re-opens on rename)
grep -rn "ERROR" /var/log/app/         # recursive, with line numbers
grep -E "5[0-9]{2} " access.log        # extended regex — 5xx status codes
grep -c "Deploy failed" deploy.log     # count matches
sed -n '100,150p' big.log              # print lines 100–150 without loading the whole file
sed 's/old/new/g' file                 # substitute (stdout; add -i to edit in place)
awk '{print $1, $9}' access.log        # print columns 1 and 9 (IP + status)
awk '$9 >= 500' access.log             # rows where column 9 (status) is 5xx
```

`tail -F` (capital) is the one to reach for on a live server: it survives logrotate by re-opening the path when the old file is renamed away. Lowercase `-f` keeps following the now-rotated inode and goes silent.

### Permissions and ownership

```bash
chmod 644 file                         # rw-r--r-- (files)
chmod 755 script.sh                    # rwxr-xr-x (executables, dirs)
chmod -R u+rwX,go+rX dir/              # capital X = execute only on dirs, not files
chown -R deploy:www-data /var/www/app  # recursive owner + group
chgrp -R www-data storage/             # group only
umask 022                              # new files 644, new dirs 755 (default)
sudo -u www-data command               # run as another user
stat -c '%A %U:%G %n' file             # show mode, owner, group in one line
```

`chmod -R` with a capital `X` is the trick for post-deploy fixes: `u+rwX` grants execute on directories (so they stay traversable) without marking every regular file executable. See the permissions playbook below.

### Processes and jobs

```bash
ps aux                                 # every process, BSD-style columns
ps aux --sort=-%mem | head            # top memory consumers
pgrep -a nginx                         # PIDs + command lines matching "nginx"
pkill -TERM -f 'php artisan queue'     # signal by command-line match
kill -TERM 12345                       # graceful stop (SIGTERM, default)
kill -9 12345                          # force kill (SIGKILL — last resort)
top                                    # live process table
htop                                   # nicer top, if installed
nohup ./worker.sh &                    # detach from terminal, survive logout
jobs                                   # background jobs in this shell
fg %1                                  # bring job 1 to foreground
disown -h %1                           # keep job alive after shell exits
```

Reach for `SIGTERM` first — it lets the process flush and shut down cleanly. Only escalate to `kill -9` when a process ignores TERM; a `-9` gives it no chance to release locks or ports, which is how you end up with a stale lockfile or a half-written pidfile.

### Disk and files

```bash
df -h                                  # free space per mount, human units
df -i                                  # inodes free (the "disk full but df says space" case)
du -sh *                               # size of each item in cwd
du -sh * | sort -h | tail              # biggest items, ascending
du -xh / | sort -h | tail -20          # biggest dirs on / only (-x = one filesystem)
lsof /var                              # what has files open under /var
lsof -i :8080                          # what's listening on / connected to port 8080
stat release.tar.gz                    # size, timestamps, inode
tar czf app.tgz app/                   # create gzip'd tarball
tar xzf app.tgz -C /var/www            # extract into a target dir
tar tzf app.tgz | head                 # list contents without extracting
gzip -9 big.log                        # compress (→ big.log.gz)
zcat big.log.gz | grep ERROR           # grep a compressed log without unzipping
```

When `df -h` says there's space but writes still fail with `No space left on device`, check `df -i` — you've run out of inodes, usually from millions of tiny session/cache files. The fix is deleting files, not freeing bytes.

### Networking

```bash
curl -fsSL https://example.com/health  # fail on error, silent, follow redirects
curl -I https://example.com            # headers only
curl -w '%{http_code} %{time_total}s\n' -o /dev/null -s URL   # status + timing
wget -qO- https://example.com          # fetch to stdout
ss -ltnp                               # listening TCP sockets + owning process
ss -tnp state established              # established connections
dig +short example.com                 # resolve A record, terse
dig example.com @1.1.1.1               # query a specific resolver
ping -c 4 example.com                  # 4 packets then stop
ip a                                   # interfaces + addresses
ip route                               # routing table (default gateway)
```

`ss -ltnp` is the modern replacement for `netstat -ltnp`: `-l` listening, `-t` TCP, `-n` numeric (no DNS lag), `-p` show the process. It's the first command in the "stuck port" playbook.

### System info and services

```bash
uname -a                               # kernel, arch, hostname
uptime                                 # load averages + how long up
free -h                                # memory / swap, human units
nproc                                  # CPU count (for -j build parallelism)
systemctl status app.service           # is it running? recent log tail
systemctl restart app.service          # restart a unit
systemctl reload nginx                 # reload config without dropping connections
systemctl is-enabled app.service       # will it start on boot?
journalctl -u app.service -n 100 --no-pager    # last 100 log lines for a unit
journalctl -u app.service --since "10 min ago" # time-bounded
journalctl -u app.service -f           # follow live
journalctl -p err -b                   # error-priority messages, this boot
```

`--no-pager` matters in scripts and over flaky SSH — without it `journalctl` opens `less` and hangs the session waiting for a keypress. `systemctl reload` beats `restart` for web servers: it re-reads config without dropping in-flight connections.

### Users, environment, and transfer

```bash
whoami                                 # current user
id                                     # uid, gid, group memberships
env                                    # all environment variables
printenv PATH                          # one variable
export DEPLOY_ENV=production           # set for this shell + children
echo "$PATH"                           # inspect current PATH
which php                              # resolve a binary via PATH
scp file deploy@host:/var/www/         # copy a file over SSH
scp -r dir/ deploy@host:/var/www/      # recursive copy
tar czf - app/ | ssh host 'tar xzf - -C /var/www'   # stream a dir over SSH, no temp file
```

The `tar … | ssh 'tar …'` stream is faster than `scp -r` for many small files (one compressed stream vs. one round-trip per file) and needs no scratch space on either end.

---

## Deployment workflows (the moat)

These are the exact command sequences you run when a deploy goes sideways. Each is runnable as written — substitute your service name, port, and paths.

### 1. Diagnosing a failed deploy

The service won't come up after a release. Work from the service manager outward: status first, then its logs, then whether something else already holds its port.

```bash
# 1. What does the service manager think happened?
systemctl status app.service                          # look for "failed (Result: exit-code)"

# 2. The actual error — last 100 lines, no pager (won't hang your SSH session)
journalctl -u app.service -n 100 --no-pager

# 3. Only the current boot, error priority — cuts noise fast
journalctl -u app.service -p err -b

# 4. Is the port already taken by the OLD process that never died?
ss -ltnp | grep ':8080'                               # or: lsof -i :8080

# 5. Is a stale process still running from before the deploy?
ps aux | grep '[a]pp-server'                          # [a] trick hides grep itself
pgrep -a app-server
```

The `grep '[a]pp-server'` bracket trick stops `grep` from matching its own command line in `ps` output — you get the real process rows, not a phantom self-match. If step 4 shows the old process still bound to the port, jump to playbook 5.

### 2. Reading logs fast under pressure

You have thousands of lines and thirty seconds. Narrow by time, then by pattern, then follow live.

```bash
# Everything since the deploy started (adjust the timestamp)
journalctl -u app.service --since "2026-07-10 14:30:00" --no-pager

# App log file: last 200 lines, then follow across rotation
tail -n 200 -F /var/log/app/production.log

# 5xx responses only, with the request line and status column
grep -E ' (5[0-9]{2}) ' /var/log/nginx/access.log | tail -50

# Count errors per minute to see when it started spiking
grep ERROR /var/log/app/production.log \
  | awk '{print $1, $2}' | cut -c1-16 | uniq -c

# Follow two logs at once, tagged
tail -F /var/log/app/production.log /var/log/nginx/error.log
```

`cut -c1-16` truncates an ISO timestamp to the minute so `uniq -c` can tally errors per minute — a fast way to pin the exact minute a regression shipped. Use capital `-F` on any live server so the follow survives the next logrotate.

### 3. Fixing permissions after an rsync or deploy

A sync ran as the wrong user, or the release landed `root:root`, and now the app can't read its own files or the web server returns 403. Reset ownership and modes without making every file executable.

```bash
APP=/var/www/app

# 1. Ownership: app user owns files, web-server group can read
sudo chown -R deploy:www-data "$APP"

# 2. Directories 755, files 644 — the capital-X trick does both in one pass
sudo chmod -R u=rwX,g=rX,o= "$APP"

# 3. Writable dirs the app needs (cache, uploads, sessions)
sudo chmod -R ug+rwX "$APP/storage" "$APP/bootstrap/cache"

# 4. Keep the deploy user's key readable only by them
sudo chmod 600 /home/deploy/.ssh/id_ed25519

# Explicit form if you distrust the capital X:
sudo find "$APP" -type d -exec chmod 755 {} +
sudo find "$APP" -type f -exec chmod 644 {} +
```

`chmod -R u=rwX` applies execute (`X`) only to directories and files that already had an execute bit — so scripts stay runnable, plain files become `644`, and directories stay traversable at `755`. The `find … -exec chmod {} +` form (note the `+`, not `\;`) batches many paths into each `chmod` call instead of forking one per file — far faster on a big tree, and it sidesteps `Argument list too long`.

### 4. Disk pressure during a build

The build dies with `No space left on device`. Find the biggest offenders, then clear the safe-to-delete ones.

```bash
# 1. Which mount is full?
df -h

# 2. If df shows space but writes still fail — you're out of inodes
df -i

# 3. Biggest directories on the root filesystem only (-x stays on one FS)
sudo du -xh / 2>/dev/null | sort -h | tail -20

# 4. Biggest items right here (common in a build workspace)
du -sh * | sort -h | tail

# 5. Safe clears — old releases, tmp, package caches
sudo rm -rf /var/www/releases/2026-0[1-5]*           # keep the last few, drop old ones
sudo find /tmp -type f -atime +2 -delete             # tmp files untouched for 2+ days
sudo journalctl --vacuum-size=200M                   # cap the journal

# 6. Docker is the usual culprit on build hosts
docker system df                                     # where the space went
docker system prune -af --volumes                    # dangling images, stopped containers, unused volumes
```

Keep the last two or three releases when clearing `/var/www/releases` — you need them for a fast rollback (playbook 6). `docker system prune -af --volumes` is aggressive: it removes every unused image and volume, which is what you want on a disposable build runner but not on a host that shares a registry cache with running services.

### 5. Finding and killing a process holding a port

The new release can't bind `:8080` because the old one never let go. Identify the holder, ask it to stop, then force it only if it refuses.

```bash
# 1. Who's on the port? (both work; ss is faster, lsof is more detailed)
ss -ltnp | grep ':8080'
sudo lsof -i :8080 -sTCP:LISTEN

# 2. Graceful stop first — SIGTERM lets it release the socket cleanly
sudo kill -TERM "$(ss -ltnp 'sport = :8080' | grep -oP 'pid=\K[0-9]+' | head -1)"

# 3. Confirm it actually died
ss -ltnp | grep ':8080' || echo "port is free"

# 4. Only if it's still there after a few seconds — force it
sudo kill -9 <PID>

# 5. If it's a managed service, don't fight it by hand — restart the unit
sudo systemctl restart app.service
```

Always try `SIGTERM` before `SIGKILL`: a `-9` gives the process no chance to release the socket or clean up, which can leave the port in `TIME_WAIT` and make the next bind fail anyway. If the process is under systemd, `systemctl restart` is the right move — killing it by PID just triggers the unit to respawn it, and you'll chase your own tail.

### 6. Restoring from a backup

A release is bad and you need the previous good state back — code, and optionally data. With directory-per-release layout, the rollback is a symlink swap.

```bash
# --- Code rollback (symlinked releases) ---
ls -1dt /var/www/releases/*/ | head           # newest releases, newest first
ln -sfn /var/www/releases/2026-07-09-1830 /var/www/current   # point back to last good
sudo systemctl reload app.service             # pick up the swapped symlink

# --- Data / files from a tarball backup ---
tar tzf backup-2026-07-09.tgz | head          # verify contents before extracting
tar xzf backup-2026-07-09.tgz -C /var/www/app # restore into place
sudo chown -R deploy:www-data /var/www/app    # re-apply ownership (see playbook 3)

# --- Confirm the app is actually serving the restored release ---
readlink -f /var/www/current                  # which release is live now?
curl -fsS http://localhost:8080/health && echo " OK"
```

The symlink swap is the whole point of keeping old releases on disk (playbook 4 warned against clearing them): rolling back is one `ln -sfn` plus a reload, not a re-deploy. Always `tar tzf` (list) before `tar xzf` (extract) so you know what you're about to overwrite. This is the same mechanism behind [zero-downtime deployments](https://www.deployhq.com/features/zero-downtime-deployments) — each release lands in its own directory and `current` swaps atomically.

---

## Common errors and fixes

| Error / symptom | Cause | Fix |
|---|---|---|
| `Permission denied` running a script | Missing execute bit, or wrong owner | `chmod +x script.sh`; check owner with `ls -l`, fix via `chown` |
| `bash: command not found` in cron/CI | Binary not on the minimal non-interactive `PATH` | Use an absolute path (`/usr/bin/php`), or set `PATH=` at the top of the script |
| `No space left on device` (but `df -h` shows free space) | Out of inodes, not bytes | Check `df -i`; delete many small files (old sessions, caches, logs) |
| `Text file busy` when overwriting a binary | The file is currently executing / mmap'd | Stop the process first (`systemctl stop`), then replace; or `mv` new in and restart |
| `bind: address already in use` | Old process still holds the port | `ss -ltnp \| grep :PORT`, `kill -TERM` the PID, then rebind (playbook 5) |
| `Argument list too long` on `rm *` / `chmod *` | Glob expands past `ARG_MAX` | Use `find . -maxdepth 1 -type f -delete` or `find … -exec cmd {} +` |
| `tail -f` goes silent after log rotation | Following the renamed inode, not the new file | Use `tail -F` (capital) — re-opens the path on rotation |
| `journalctl` hangs the SSH session | Output opened in the `less` pager | Add `--no-pager` (and `-n N` to bound the lines) |
| `cannot remove … Device or resource busy` | A mount or open file under the path | `lsof +D /path` to find the holder; unmount or stop it first |
| `rm -rf` silently did nothing | Trailing slash on a symlink, or a leading `-` parsed as a flag | Use `rm -rf ./-file` or `rm -rf -- -file`; check `readlink` on symlinks |
| Disk fills again minutes after clearing | A process still writing to a deleted-but-open file | `lsof \| grep deleted` — restart the writer to release the inode |

---

## Companion: full DeployHQ deploy workflow

These commands are the manual layer under an automated pipeline — the things you run by hand when a deploy needs diagnosis. The goal is to run them less often by moving the reliable ones into hooks: ownership resets, symlink swaps, and health checks belong in the deploy script, not in a 2 a.m. SSH session.

DeployHQ runs the same shell you do, so these playbooks map straight onto its hooks: put the permissions reset (playbook 3) and the health check (playbook 6) in post-deploy SSH commands, and let the [build pipeline](https://www.deployhq.com/features/build-pipelines) run them on every release. The Git-to-server flow is walked through in the [deploy from GitHub guide](https://www.deployhq.com/deploy-from-github), and when a release goes bad the symlink rollback becomes a one-click operation.

[Start a free DeployHQ trial](https://www.deployhq.com/signup) to move these manual steps into a repeatable deploy pipeline.

---

## Related cheatsheets

- [Bash scripting cheatsheet](https://www.deployhq.com/cheatsheets/bash) — for the `set -euo pipefail`, `trap`, and lockfile patterns that turn these one-liners into reliable deploy scripts.
- [cron and crontab cheatsheet](https://www.deployhq.com/cheatsheets/cron) — for scheduling the recurring jobs (backups, log rotation, cleanups) that use these commands.
- [SSH cheatsheet](https://www.deployhq.com/cheatsheets/ssh) — for the transport layer you run every one of these commands over on a remote host.
- [rsync cheatsheet](https://www.deployhq.com/cheatsheets/rsync) — for the file-sync step whose permission fallout playbook 3 cleans up.
- [systemd and journalctl cheatsheet](https://www.deployhq.com/cheatsheets/systemd) — for the service and log commands the failed-deploy playbook leans on.
- [Cheatsheets hub](https://www.deployhq.com/cheatsheets) — every DeployHQ cheatsheet in one place.

---

Need help? Email [support@deployhq.com](mailto:support@deployhq.com) or follow [@deployhq on X](https://x.com/deployhq).