systemd Cheatsheet

What it is

systemd is the init system on virtually every modern Linux server — Ubuntu 16.04+, Debian 8+, RHEL/CentOS 7+, Amazon Linux 2+. It owns PID 1, supervises every long-running service, manages timers (cron's replacement), captures logs into journald, and handles the start/stop/restart lifecycle that deploys hinge on. If your systemctl reload nginx runs after every deploy, systemd is what's actually doing the work.

This sheet covers the commands and unit-file patterns you reach for when systemd is part of a real deployment workflow — the systemctl invocations that show up in deploy hooks, the unit files that supervise long-running app processes (Node/Python/Go/Ruby servers, queue workers), the Restart= and WatchdogSec= flags that keep them alive across crashes, and the journalctl queries that find the root cause when a deploy doesn't come up.

Quick reference

systemctl basics

systemctl status                                    # overall system state
systemctl status nginx                              # one service: active state, recent logs, PIDs
systemctl is-active nginx                           # → active / inactive / failed (scriptable)
systemctl is-enabled nginx                          # → enabled / disabled / masked
systemctl is-failed nginx                           # → exit code 0 if failed

systemctl start nginx
systemctl stop nginx
systemctl restart nginx                             # full stop + start
systemctl reload nginx                              # SIGHUP — config reload, no restart
systemctl reload-or-restart nginx                   # reload if supported, restart otherwise
systemctl try-restart nginx                         # restart ONLY if currently running
systemctl try-reload-or-restart nginx
systemctl kill nginx                                # send SIGTERM (use --signal=SIGKILL for force)

systemctl enable nginx                              # start on boot
systemctl disable nginx                             # don't start on boot
systemctl enable --now nginx                        # enable + start immediately
systemctl disable --now nginx                       # disable + stop immediately
systemctl mask nginx                                # forbid starting (even by dependencies)
systemctl unmask nginx

systemctl reload is the deploy-friendly verb — it sends SIGHUP to nginx/php-fpm/postgres and they reread config without dropping connections. restart drops every active request. Use the right one.

Listing and inspecting units

systemctl list-units                                # all loaded, active units
systemctl list-units --type=service                 # only services
systemctl list-units --state=failed                 # only failed
systemctl list-units --all                          # include inactive
systemctl list-unit-files                           # all unit files on disk (enabled/disabled/masked)
systemctl list-unit-files --state=enabled

systemctl cat nginx                                 # show the unit file (and any drop-in overrides)
systemctl show nginx                                # all properties, key=value
systemctl show nginx --property=ExecStart,RestartSec
systemctl show nginx -p MainPID --value             # just the PID

systemctl list-dependencies nginx                   # what nginx depends on
systemctl list-dependencies --reverse nginx         # what depends on nginx

systemctl cat is the right way to inspect a unit — it concatenates the original .service file plus every .conf drop-in override in /etc/systemd/system/<unit>.d/. Reading just the original file misses overrides that change runtime behaviour.

Editing units

systemctl edit nginx                                # create drop-in override (preserves base file)
systemctl edit --full nginx                         # edit the full file (replaces base — avoid)
systemctl daemon-reload                             # reload unit files after manual edits
systemctl revert nginx                              # discard drop-in overrides

systemctl edit nginx opens $EDITOR against /etc/systemd/system/nginx.service.d/override.conf, which layers on top of the distro-shipped /lib/systemd/system/nginx.service. The override survives package upgrades; --full doesn't.

After any manual file edit (not via systemctl edit), run systemctl daemon-reload to make systemd reparse. Skip this and the running service stays on the old config until restart.

Journals (logs)

journalctl                                          # entire log, oldest first
journalctl -e                                       # jump to end (most recent)
journalctl -f                                       # follow (tail -f)
journalctl -r                                       # reverse (newest first)
journalctl -n 100                                   # last 100 lines

journalctl -u nginx                                 # only nginx logs
journalctl -u nginx -f                              # follow nginx
journalctl -u nginx --since "10 min ago"
journalctl -u nginx --since today --until "1 hour ago"
journalctl -u nginx --since "2026-06-13 14:00:00"
journalctl -u nginx -p err                          # priority filter (err / warning / notice / info / debug)
journalctl -u nginx -g "504 Gateway"                # grep within logs
journalctl -u nginx -o json | jq .                  # JSON output for scripting
journalctl -u nginx -o cat                          # message-only, no metadata

journalctl --boot                                   # logs from current boot only
journalctl --list-boots                             # all boots journalctl can see
journalctl -b -1                                    # logs from previous boot

journalctl --disk-usage                             # how much disk the journal occupies
journalctl --vacuum-size=500M                       # truncate to 500MB
journalctl --vacuum-time=7d                         # delete entries older than 7d

The single most useful journalctl invocation during a deploy: journalctl -u myapp -f --since "1 min ago". Tail the new process's logs in another terminal, push the deploy, watch the startup sequence in real time. Beats waiting for the systemd "failed to start" notification by ~30 seconds.

Failed units and recovery

systemctl --failed                                  # list every failed unit
systemctl reset-failed                              # clear "failed" state on ALL units
systemctl reset-failed nginx                        # clear it on one

After fixing the underlying problem, systemctl reset-failed is required before systemd will attempt to restart the unit again on the next dependency event. Forgetting this is the most common reason "I fixed it but it still won't start" lingers.

Targets (systemd's runlevels)

systemctl get-default                               # which target boots by default
systemctl set-default multi-user.target             # boot to text-mode (no GUI)
systemctl set-default graphical.target              # boot to GUI

systemctl isolate multi-user.target                 # switch target NOW (drops to text mode)
systemctl isolate rescue.target                     # single-user maintenance mode
systemctl isolate emergency.target                  # bare emergency shell

systemctl list-units --type=target

For servers, multi-user.target is the right default. graphical.target ships an X server you don't need.

Timers (cron's replacement)

systemctl list-timers                               # all active timers + next/last run
systemctl list-timers --all                         # include inactive
systemctl status backup.timer                       # one timer
systemctl start backup.service                      # manually trigger the underlying service
journalctl -u backup.service                        # last run's logs

Timer units pair with service units of the same name (e.g. backup.timer triggers backup.service). The Cron and crontab cheatsheet has the trade-off discussion of cron vs systemd timers — short version: timers integrate with journald, support calendar expressions, and run as proper systemd units; cron is simpler and ubiquitous.

Sockets, mounts, paths

systemctl list-sockets                              # socket-activated services
systemctl list-units --type=mount                   # systemd-managed mounts (from /etc/fstab)
systemctl list-units --type=path                    # path watchers (inotify-style triggers)

Socket activation lets systemd hold an open listening socket while the application is stopped, and hand it off when the app starts — eliminates the "port in use" race during a restart. Most modern Linux services (sshd, docker, postgres) support it.

Resource controls (cgroups)

systemd-cgtop                                       # like `top`, but grouped by unit (CPU/memory per service)
systemctl set-property nginx MemoryMax=512M         # transient — gone after reboot
systemctl set-property --runtime nginx CPUQuota=50% # persistent across reboot

# Or set permanently in the unit file's [Service] section
# CPUQuota=50%
# MemoryMax=512M
# IOWeight=200

systemd-cgtop is the right tool for "which service is eating my CPU/memory" — it groups process trees by their systemd unit, so you see the whole nginx + worker + cron family as one row.

Boot performance

systemd-analyze                                     # total boot time
systemd-analyze blame                               # per-unit boot time (sorted slowest first)
systemd-analyze critical-chain                      # blocking chain that gates boot completion
systemd-analyze plot > boot.svg                     # graphical boot timeline

systemd-analyze blame is the surprise-finder — discovers the 60-second timeout on a unit nobody knew was running, the network-online wait that never gets satisfied, the DNS lookup in a script that boots before DNS is up.

Unit files: the production patterns

A .service file lives in one of three places, in increasing override priority:

• /lib/systemd/system/foo.service — distro-shipped (don't edit)
• /etc/systemd/system/foo.service — system admin's full override
• /etc/systemd/system/foo.service.d/override.conf — drop-in (preferred for tweaks)

After any manual file change, run systemctl daemon-reload.

Minimal viable web app service

# /etc/systemd/system/myapp.service
[Unit]
Description=My App (Node.js)
After=network-online.target postgres.service
Wants=network-online.target

[Service]
Type=simple
User=deploy
Group=deploy
WorkingDirectory=/var/www/current
EnvironmentFile=/etc/myapp/env
ExecStart=/usr/bin/node /var/www/current/dist/server.js
Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal
SyslogIdentifier=myapp

[Install]
WantedBy=multi-user.target

Eight non-obvious decisions in this file:

• After=network-online.target delays start until the network is genuinely up (not just network.target, which fires when the interface is configured but routes may not exist yet).
• User=deploy runs the app as a non-root user. Forgetting this is the single most common security failure in hand-rolled unit files.
• WorkingDirectory=/var/www/current ties the service to the atomic-release symlink — a deploy that swaps the symlink (see the rsync cheatsheet's atomic-release pattern) doesn't need to touch this file.
• EnvironmentFile=/etc/myapp/env keeps secrets out of the unit file. Set this file's mode to 0600 and owner to root:root.
• Type=simple is the right default for most modern services that don't fork. Use Type=notify if your service uses sd_notify(); Type=exec for stricter start-tracking; Type=forking only if it actually forks (rare in 2026).
• Restart=always + RestartSec=5 keeps the service alive across crashes with a 5-second backoff. Without RestartSec, the default is 100ms — which can spiral into a tight restart loop and confuse journald.
• StandardOutput=journal captures every console.log / print() into journald, where journalctl -u myapp can find it.
• WantedBy=multi-user.target is what systemctl enable actually does — wires the service into the boot dependency graph.

Hardening: the defaults every production unit should set

The defaults above are correct; these are the hardening additions that turn it into a properly sandboxed service:

[Service]
# (everything from above, plus:)

# Filesystem
ProtectSystem=strict                                # /usr, /boot, /etc read-only
ProtectHome=true                                    # /home, /root not visible
ReadWritePaths=/var/log/myapp /var/lib/myapp        # explicit write whitelist
PrivateTmp=true                                     # /tmp is per-service
NoExecPaths=/tmp                                    # can't exec from writable paths

# Network
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX    # no Bluetooth, no Netlink
PrivateNetwork=false                                # set true for batch jobs that don't need net

# Kernel + ptrace
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectKernelLogs=true
ProtectControlGroups=true
RestrictNamespaces=true
LockPersonality=true
MemoryDenyWriteExecute=true                         # block JIT/shellcode tricks

# Privilege drop
CapabilityBoundingSet=
AmbientCapabilities=
NoNewPrivileges=true
RestrictSUIDSGID=true

# Resource limits
LimitNOFILE=65535
MemoryMax=1G
CPUQuota=200%                                       # 200% = 2 cores worth
TasksMax=512

Run systemd-analyze security myapp to grade your unit file. Stock units score around 9.0 (UNSAFE); after these additions, healthy production services land at 1.0-3.0 (OK / GOOD). The grade is heuristic, but it surfaces real attack surface every time.

Type-by-type quick reference

Type=simple        # ExecStart runs the main process directly (default)
Type=forking       # ExecStart forks; main process is the parent (legacy daemons)
Type=oneshot       # ExecStart runs to completion; no long-running process
Type=notify        # ExecStart starts; service sends sd_notify(READY=1) when ready
Type=exec          # Like simple, but Active state waits until execve() returns
Type=dbus          # Service is "started" once it claims a D-Bus name
Type=idle          # Like simple, but delays start until other jobs finish (boot screens)

For deploys that need health-gated startup (the service is up but not ready), Type=notify plus an sd_notify(READY=1) call after the database connection pool warms up is the right tool. Other units with After=myapp.service won't start until the notify fires.

Worker / background job service

For queue workers (Sidekiq, Celery, Laravel Horizon, BullMQ), the variant that holds up under load:

[Unit]
Description=Sidekiq for myapp
After=network-online.target redis.service
Requires=redis.service

[Service]
Type=notify
User=deploy
Group=deploy
WorkingDirectory=/var/www/current
EnvironmentFile=/etc/myapp/env
ExecStart=/usr/bin/bundle exec sidekiq -e production
ExecReload=/bin/kill -USR1 $MAINPID                 # gracefully reload on `systemctl reload`
KillSignal=SIGTERM
TimeoutStopSec=60                                   # wait 60s for graceful shutdown
SendSIGKILL=yes                                     # then SIGKILL
Restart=always
RestartSec=10
StartLimitIntervalSec=600
StartLimitBurst=5                                   # 5 failed starts in 600s = give up

[Install]
WantedBy=multi-user.target

The deploy-critical lines here are TimeoutStopSec=60 (gives the worker time to finish the in-flight job before getting killed) and KillSignal=SIGTERM (Sidekiq listens for SIGTERM and quiesces cleanly). The default SIGTERM then KILL sequence with 90s timeout is fine for most apps; tune for workers that take longer to drain.

StartLimitBurst=5 plus StartLimitIntervalSec=600 is the circuit breaker — if the worker fails 5 times in 10 minutes, systemd stops retrying. Without it, a startup bug puts the worker in an infinite restart loop that fills the journal.

Timer (cron replacement)

# /etc/systemd/system/db-backup.service
[Unit]
Description=Daily database backup

[Service]
Type=oneshot
User=postgres
ExecStart=/usr/local/bin/db-backup.sh

# /etc/systemd/system/db-backup.timer
[Unit]
Description=Trigger daily database backup

[Timer]
OnCalendar=*-*-* 03:30:00                           # daily at 03:30
RandomizedDelaySec=600                              # spread fleet to avoid thundering herd
Persistent=true                                     # run on next boot if missed
Unit=db-backup.service

[Install]
WantedBy=timers.target

Enable + start the timer (not the service):

systemctl daemon-reload
systemctl enable --now db-backup.timer
systemctl list-timers db-backup.timer               # confirm next-run timestamp

OnCalendar= accepts the full calendar grammar — *-*-* 03:30:00 is daily, Mon..Fri 09:00 is weekday mornings, *-*-01 00:00 is monthly. Run systemd-analyze calendar "OnCalendar=expression" to test before installing.

Socket activation

# /etc/systemd/system/myapp.socket
[Unit]
Description=Socket for myapp

[Socket]
ListenStream=127.0.0.1:3000
Accept=false

[Install]
WantedBy=sockets.target

# /etc/systemd/system/myapp.service
[Unit]
Description=My App (socket-activated)
Requires=myapp.socket

[Service]
ExecStart=/usr/bin/myapp
StandardInput=socket                                # systemd hands the socket on fd 0

The socket survives a service restart — connections don't get refused during the restart window because the kernel keeps queuing them on the socket systemd holds open. Useful for "deploy-without-disconnect" workflows where the brief restart would otherwise drop in-flight HTTP.

Deployment workflows (the moat)

1. The unit file lives outside the release directory

A common mistake: putting the .service file inside /var/www/releases/<sha>/. The first deploy works; the second deploy moves the release directory, breaking the unit's WorkingDirectory= and ExecStart= paths.

The right pattern: the unit file references a stable path (/var/www/current symlink), and the deploy never touches the unit file:

[Service]
WorkingDirectory=/var/www/current
ExecStart=/var/www/current/bin/server

Deploy script:

RELEASE="/var/www/releases/$SHA"
# ... rsync release into RELEASE ...
ln -sfn "$RELEASE" /var/www/current                 # atomic swap
systemctl reload myapp                              # or restart if app doesn't handle SIGHUP

The unit file becomes part of the machine image, deployed once with Ansible/Cloud-Init/Terraform — not part of the application release.

2. systemctl reload after every deploy

The deploy hook that turns a successful file transfer into a live release:

# In the post-deploy step
systemctl reload nginx                              # reread config (zero downtime)
systemctl restart myapp                             # if app doesn't support SIGHUP reload
systemctl reload-or-restart myapp                   # try reload first, fall back to restart

For nginx, php-fpm, postgres — always reload. They handle SIGHUP cleanly. For Node/Python/Go apps, you usually need a full restart. If your runtime supports a graceful-shutdown signal (most do), set KillSignal=SIGTERM in the unit and the restart drains in-flight requests cleanly.

Combine with zero-downtime deployments and the restart becomes invisible to traffic — the symlink swap puts new code in place; nginx fronting the app keeps connections warm; systemctl reload re-execs into the new release.

3. Sudo-free deploy hooks for systemctl

Deploy users (DeployHQ, Capistrano, plain SSH bots) typically don't have full sudo. Granting sudo systemctl reload nginx and similar per-command via /etc/sudoers.d/deploy is the right granularity:

# /etc/sudoers.d/deploy — install with `visudo -f`, mode 0440, root:root
deploy ALL=(root) NOPASSWD: /bin/systemctl reload nginx
deploy ALL=(root) NOPASSWD: /bin/systemctl restart myapp
deploy ALL=(root) NOPASSWD: /bin/systemctl restart myapp-worker
deploy ALL=(root) NOPASSWD: /bin/systemctl reload php8.2-fpm

Test with sudo -l -U deploy to confirm the entries are parsed. The deploy script then runs sudo systemctl reload nginx without prompting and without granting general root.

4. Health-gated deploys with Type=notify

The reliable pattern: app boots, connects to DB, warms caches, sends sd_notify(READY=1). systemd waits for that notification before treating the service as active. Dependent units (or a deploy hook waiting on systemctl is-active) only proceed once the app is genuinely serving.

# In the app — Python example
import os, socket
def notify_ready():
    sock_path = os.environ.get('NOTIFY_SOCKET')
    if not sock_path:
        return
    sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
    sock.connect(sock_path)
    sock.sendall(b'READY=1')

# Call after the app is genuinely ready (DB pool warm, migrations checked)
notify_ready()

[Service]
Type=notify
TimeoutStartSec=120                                 # systemd's max wait for READY=1
WatchdogSec=30                                      # app must ping systemd every 30s
NotifyAccess=main
ExecStart=/usr/bin/python /srv/app/main.py
Restart=on-failure

WatchdogSec=30 adds the bonus: the app must call sd_notify(WATCHDOG=1) at least every 30 seconds, or systemd kills + restarts it. Catches the hung-but-not-crashed failure mode.

Deploy hook waits for the readiness signal:

systemctl restart myapp
# Wait for systemd to confirm READY=1
for _ in $(seq 1 30); do
  if systemctl is-active myapp >/dev/null; then
    echo "myapp ready"
    break
  fi
  sleep 1
done

5. journalctl as the deploy debug entry point

When systemctl start myapp fails, the first command you run is always:

journalctl -u myapp --since "5 min ago" -n 200 --no-pager

That window is large enough to catch the start sequence, small enough to fit on screen. For an app that boots fine but crashes mid-deploy:

journalctl -u myapp -f --since now    # tail in another terminal
# In the first terminal: trigger the failing request

Real-time log tailing during deploys + smoke tests is the difference between a 90-second incident triage and a 30-minute incident.

For structured-log apps emitting JSON to stdout, pipe through jq:

journalctl -u myapp -o cat | jq -r '. | "[\(.level)] \(.message)"'

The jq cheatsheet covers the assertion patterns that turn a JSON log stream into deploy-gate checks.

6. Replace cron with systemd timers (when timer integration matters)

Cron is fine for "run this script daily" — it's universal, simple, and works. The reasons to replace it with a systemd timer:

• Logs go to journald, not a forgotten log file. journalctl -u backup.service --since today works out of the box.
• Calendar expressions are richer. OnCalendar=Mon..Fri 09:30 is harder to express in vanilla cron.
• Persistent=true runs a missed timer on next boot. Cron drops missed jobs silently when the host was off.
• Failures surface in systemctl --failed. Cron's failure mode (silent + email to root) is invisible on most modern servers.
• Resource limits + sandboxing. The same MemoryMax=, ProtectSystem=strict, PrivateTmp=true apply to timer-triggered services.

For mixed environments (some servers, some serverless cron from your platform), the Cron and crontab cheatsheet covers when each is the right tool.

7. Resource budgets per service

Production servers run multiple services. Without cgroup limits, one misbehaving worker can OOM-kill the database, take down the host with a fork bomb, or eat every CPU cycle and starve nginx of scheduling time.

[Service]
MemoryMax=512M                                      # hard cap; OOM-killed if exceeded
MemoryHigh=400M                                     # soft cap; throttle approaching MemoryMax
CPUQuota=200%                                       # 200% = 2 cores
CPUWeight=100                                       # relative scheduler priority (default 100)
IOWeight=100
TasksMax=256                                        # max threads/processes

These are kernel cgroup v2 controls, enforced by the scheduler — not advisory. An app that genuinely needs more memory hits MemoryMax and gets a clean OOM kill (visible in journalctl -u myapp -p err), not the silent host-wide memory pressure that produces "everything is randomly slow" pages.

Common errors and fixes

Companion: where systemd fits in a real deploy pipeline

Most of the systemd lifecycle work happens inside a deploy script — systemctl reload nginx, systemctl restart myapp, systemctl reload-or-restart php8.2-fpm. The deploy pipeline around those calls is what makes them reliable: a tested build artifact, an atomic release directory, a smoke test after systemctl is-active myapp confirms ready, and a rollback path when something fails.

A typical DeployHQ deploy that ends in systemctl calls:

1. Push to your main branch on GitHub or GitLab.
2. DeployHQ runs your build pipeline, produces the release artifact.
3. DeployHQ SSHes into your servers (using a per-project key, no shared credentials) and rsync's the artifact into /var/www/releases/<sha>/.
4. The deploy hook runs your install steps (composer install, npm ci --production, php artisan migrate --force), then ln -sfn /var/www/releases/<sha> /var/www/current for the atomic release swap.
5. Final hook: sudo systemctl reload-or-restart myapp && sudo systemctl reload nginx, followed by a smoke test against the health endpoint.
6. If anything fails, one-click rollback re-points the symlink at the previous release and reruns systemctl reload-or-restart — the rollback path is the same as the deploy path, just pointing at older code.

The end-to-end setup is in deploy from GitHub to your server. The zero-downtime deployments page covers the symlink-swap + systemctl reload mechanics that make this invisible to traffic.

Start a free DeployHQ trial to wire systemd-managed services into a production deploy pipeline.

Related cheatsheets

• Bash scripting cheatsheet — for the set -euo pipefail deploy scripts that orchestrate systemctl calls.
• SSH cheatsheet — for the deploy-key auth pattern that lets non-root deploy users hit sudo systemctl reload.
• Cron and crontab cheatsheet — for the cron vs systemd timers decision and the calendar expression syntax timers share.
• Docker cheatsheet — for the auto-restart behaviour systemd-on-host and Docker-Compose-restart-policies overlap on.
• Laravel Artisan cheatsheet — for the systemctl reload php-fpm and queue:restart sequence after Laravel deploys.
• Cheatsheets hub — every DeployHQ cheatsheet in one place.

Need help? Email support@deployhq.com or follow @deployhq on X.