Run Keycloak in Docker or Podman with SSL: Production Setup

Every Keycloak Docker tutorial that ranks today runs start-dev, one container, an embedded database and no TLS. That is a demo. Put real users on it and the first restart logs everyone out, the first upgrade has no rollback, and the admin console sits on the public internet.

Original content from computingforgeeks.com - post 62703

This guide builds the production version of Keycloak in Docker: an optimized image, PostgreSQL, two clustered nodes behind Nginx with a Let’s Encrypt certificate, a locked-down admin path, brute force protection, TOTP and passkeys, Grafana and Forgejo signing in over OpenID Connect, nightly backups with a tested restore, and a measured upgrade. The same stack then runs under Podman Quadlets on Rocky Linux with SELinux enforcing. Everything below was run in October 2026 on Keycloak 26.8.0 and PostgreSQL 18, with Docker on Ubuntu 26.04 and Ubuntu 24.04 and Podman on Rocky Linux 10.2. If you are still deciding whether Keycloak is the right identity provider at all, the Keycloak vs authentik vs Zitadel vs Authelia test compares it with the lighter options on one VM.

What the finished stack looks like

Nginx terminates TLS on port 443 and spreads requests across two Keycloak containers. Both nodes share one PostgreSQL database (the official container here, or a native PostgreSQL 18 install on its own host), find each other through it (the JDBC_PING discovery that is now the default), and replicate their caches over an encrypted JGroups channel. Sessions are persisted in the database, so a node can die without logging anyone out. Nothing except Nginx listens on a public interface.

Admin requests take the same path as everyone else’s: port 443 into Nginx, then loopback into a Keycloak node. Keycloak has no IP allow list for its admin paths, and a host firewall filters ports, not URL paths, so the rule that keeps /admin/ and the master realm off the internet has to live in the Nginx config.

Component	Listens on	Role
Nginx	0.0.0.0:80, 0.0.0.0:443	TLS, load balancing, blocks admin paths from outside
keycloak1, keycloak2	127.0.0.1:8081/8082 (HTTP), 127.0.0.1:9001/9002 (health, metrics)	Identity provider nodes
PostgreSQL 18	Docker network only	Realms, users, persisted sessions, cluster discovery
JGroups	Docker network, TCP 7800 and 57800	Cache replication between nodes, mTLS by default

Prerequisites and sizing

Keycloak sizing is driven by three numbers: login rate (password hashing is the CPU cost), the count of active sessions (cache memory, now backed by the database), and database connections. Each node holds a fixed JDBC pool, so the pool size times the node count must stay under the PostgreSQL max_connections limit, which defaults to 100. The Keycloak memory and CPU sizing guide is the reference for real numbers once you know your logins per second.

Measured on the test host after the full setup, each idle Keycloak node used 540 to 610 MiB, PostgreSQL about 160 MiB, and the whole host (including Grafana and Forgejo) sat at 2 GB used out of 8 GB. The lab ran 4 vCPU and 8 GB RAM. Treat that as a floor for following along, not a production recommendation. In production, run the two nodes on separate hosts with identical specs, because a cluster behaves like its slowest member.

• A Linux host with Docker Engine and the Compose plugin. Ubuntu 26.04 and 24.04 were both tested. The Docker CE install guide for Ubuntu covers the repository setup, and Rocky Linux readers can follow the Docker CE on Rocky Linux steps, or run the variable block below and then skip to the Podman section.
• A DNS A record for the Keycloak name (keycloak.example.com in this guide) pointing at this host, created before you request the certificate. Any DNS provider works. The optional Grafana example needs one more record, which its section covers.
• On Cloudflare, set the record to DNS only (grey cloud). Through the proxy, every request reaches Nginx from a Cloudflare address, so the admin console allow list no longer sees your real IP.
• Inbound TCP 80 and 443 for the HTTP-01 challenge and the site. A host without inbound port 80 can use DNS-01 instead, which needs an API token from your DNS provider.
• Outbound HTTPS to api.github.com, quay.io, Docker Hub and Let’s Encrypt (plus your DNS provider’s API, such as api.cloudflare.com, for DNS-01), and outbound TCP 587 to your mail relay if you follow the optional SMTP section.
• Optional: an SMTP relay account (host, username, password and an allowed sender address), only for the password reset section. Keycloak works without it.
• A TOTP authenticator app for the MFA step. A phone or security key is needed only to test passkeys.

Set the shell variables

Your domain and realm settings show up in dozens of commands, so they go into variables once. Every later command, config file and template reads them, which means this block is the only place you edit. Passwords and secrets are never stored in it, the commands that need them ask at a prompt. REALM is the short realm ID that appears in every URL (/realms/REALM/), so keep it lowercase with no spaces. REALM_NAME is the display name users see on the login page and in their passkey manager. ADMIN_CIDR is the address allowed to reach the admin console. On a VPS you manage from home or the office, use your workstation’s public IP as a /32 (run curl -4 ifconfig.me on that machine to see it). If your IP changes, update the allow line in the Nginx site. USER_EMAIL is the address of the first test user. The Keycloak version is read from the GitHub releases API, so the guide keeps working when the next release ships:

export KC_DOMAIN="keycloak.example.com"
export ADMIN_EMAIL="[email protected]"
export ADMIN_CIDR="203.0.113.10/32"
export REALM="example"
export REALM_NAME="Example Corp"
export USER_EMAIL="[email protected]"
export KC_VERSION=$(curl -s https://api.github.com/repos/keycloak/keycloak/releases/latest | grep '"tag_name"' | cut -d '"' -f 4)
echo "Keycloak ${KC_VERSION} on ${KC_DOMAIN}, realm ${REALM}"

The echo should print a clean version string, not an empty value:

Keycloak 26.8.0 on keycloak.example.com, realm example

Before requesting a certificate, check that the Keycloak name resolves to the address you expect. An empty result means the record is missing or has not propagated yet:

echo "${KC_DOMAIN} -> $(getent ahostsv4 "${KC_DOMAIN}" | awk 'NR==1 {print $1}')"

It should print the IP of this host. getent reads /etc/hosts first, so a 127.0.1.1 result means the server maps its own FQDN there, not that DNS is wrong:

keycloak.example.com -> 10.0.1.50

These variables live only in the current shell. Re-export them if you reconnect, and re-run the KCADM= and config credentials lines from the admin section before any later $KCADM command (after the go-live step, log in with the kcadm-automation client instead). The Grafana and Forgejo sections are optional examples, and each one sets its own hostname variable when you get to it.

Install Nginx and get a Let’s Encrypt certificate

Keycloak can terminate TLS itself, but a reverse proxy in front gives you load balancing across nodes and a place to block the admin paths. Install Nginx and certbot with its Nginx plugin:

sudo apt update
sudo apt install -y nginx certbot python3-certbot-nginx

Request the certificate with the HTTP-01 challenge. certonly leaves the Nginx config alone, because we write our own server blocks in a moment, and the deploy hook reloads Nginx after every renewal so it picks up the new files:

sudo certbot certonly --nginx -d "${KC_DOMAIN}" --non-interactive --agree-tos -m "${ADMIN_EMAIL}" \
  --deploy-hook "systemctl reload nginx"

The certificate lands in its own directory under /etc/letsencrypt/live/, which is the layout the Nginx files below expect. The optional Grafana section later reuses this command with its own name.

Private network or no port 80: use DNS-01

Our test hosts sit on a private LAN with no inbound port 80, so the certificates came from the DNS-01 challenge instead. Certbot has a plugin per DNS provider:

DNS provider	Ubuntu package	Certbot flag
Cloudflare	python3-certbot-dns-cloudflare	--dns-cloudflare
AWS Route 53	python3-certbot-dns-route53	--dns-route53
DigitalOcean	python3-certbot-dns-digitalocean	--dns-digitalocean
Google Cloud DNS	python3-certbot-dns-google	--dns-google
Linode	python3-certbot-dns-linode	--dns-linode
OVH	python3-certbot-dns-ovh	--dns-ovh
BIND / RFC 2136	python3-certbot-dns-rfc2136	--dns-rfc2136

The worked example uses Cloudflare. Substitute the plugin for your provider and credentials file if you are on something else. Create an API token scoped to Zone, DNS, Edit on that zone, then write it to a credentials file only root can read. The token is read at a prompt so it stays out of your shell history and off the screen:

sudo apt install -y python3-certbot-dns-cloudflare
read -rs -p "Cloudflare API token: " CF_TOKEN; echo
sudo install -m 600 /dev/null /etc/letsencrypt/cloudflare.ini
echo "dns_cloudflare_api_token = ${CF_TOKEN}" | sudo tee /etc/letsencrypt/cloudflare.ini >/dev/null

Then issue the certificate. If you follow the optional Grafana section later, run the same command again with its name:

sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini \
  --dns-cloudflare-propagation-seconds 30 -d "${KC_DOMAIN}" --non-interactive --agree-tos -m "${ADMIN_EMAIL}" \
  --deploy-hook "systemctl reload nginx"

Each run ends with the certificate path:

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/keycloak.example.com/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/keycloak.example.com/privkey.pem

Do not drop --deploy-hook. A DNS-01 certificate has no installer, so certbot renew writes the new files and never tells Nginx, which keeps serving the old certificate until it expires. Our first lab certificates were issued without the hook, and their renewal files had no installer and no renew_hook line. Certificates you already issued can take the hook without being reissued:

sudo certbot reconfigure --cert-name "${KC_DOMAIN}" --deploy-hook "systemctl reload nginx"
sudo grep renew_hook "/etc/letsencrypt/renewal/${KC_DOMAIN}.conf"

Prove the whole renewal chain with a dry run. A dry run skips deploy hooks unless you ask for them, and when certbot has no terminal attached it first sleeps a random delay of up to eight minutes (ours waited 411 seconds), so run it from an interactive shell:

sudo certbot renew --dry-run --cert-name "${KC_DOMAIN}" --run-deploy-hooks

The staging certificate is issued and the hook reloads Nginx:

Congratulations, all simulated renewals succeeded:
  /etc/letsencrypt/live/keycloak.example.com/fullchain.pem (success)

The certbot log shows the hook firing, and the Nginx journal shows the reload at the same second:

2026-10-04 20:01:13,452:INFO:certbot.compat.misc:Running deploy-hook command: systemctl reload nginx
Oct 04 20:01:13 idp01 systemd[1]: Reloaded nginx.service - A high performance web server and a reverse proxy server.

Build an optimized Keycloak image

The stock image re-runs its build step on every start when you change build-time options. Baking them in once with kc.sh build makes start --optimized fast and predictable. Create the project directory:

sudo mkdir -p /opt/keycloak
sudo vim /opt/keycloak/Dockerfile

Add this two-stage build. The database vendor, health checks and metrics are build-time options, which is why they live here and not in the Compose file:

ARG KC_VERSION=26.8.0
FROM quay.io/keycloak/keycloak:${KC_VERSION} AS builder
ENV KC_DB=postgres \
    KC_HEALTH_ENABLED=true \
    KC_METRICS_ENABLED=true
WORKDIR /opt/keycloak
RUN /opt/keycloak/bin/kc.sh build

FROM quay.io/keycloak/keycloak:${KC_VERSION}
COPY --from=builder /opt/keycloak/ /opt/keycloak/
ENTRYPOINT ["/opt/keycloak/bin/kc.sh"]

Write the Keycloak Docker Compose file

Open the Compose file:

sudo vim /opt/keycloak/compose.yaml

Paste the full file. Both nodes share one YAML anchor, so they cannot drift apart:

name: keycloak

x-keycloak: &keycloak
  build:
    context: .
    args:
      KC_VERSION: ${KC_VERSION}
  image: keycloak-optimized:${KC_VERSION}
  restart: unless-stopped
  environment:
    KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak
    KC_DB_USERNAME: keycloak
    KC_DB_PASSWORD: ${POSTGRES_PASSWORD}
    KC_DB_POOL_INITIAL_SIZE: 20
    KC_DB_POOL_MIN_SIZE: 20
    KC_DB_POOL_MAX_SIZE: 20
    KC_CACHE_EMBEDDED_CLUSTER_NAME: keycloak-prod
    KC_HOSTNAME: https://${KC_DOMAIN}
    KC_PROXY_HEADERS: xforwarded
    KC_HTTP_ENABLED: "true"
    KC_SERVER_ASYNC_BOOTSTRAP: "false"
    KC_BOOTSTRAP_ADMIN_USERNAME: tmpadmin
    KC_BOOTSTRAP_ADMIN_PASSWORD: ${KC_BOOTSTRAP_ADMIN_PASSWORD}
    JAVA_OPTS_KC_HEAP: "-XX:MaxRAMPercentage=70 -XX:InitialRAMPercentage=50"
  depends_on:
    postgres:
      condition: service_healthy
  healthcheck:
    test: ["CMD-SHELL", "exec 3<>/dev/tcp/127.0.0.1/9000 && printf 'GET /health/ready HTTP/1.1\\r\\nHost: localhost\\r\\nConnection: close\\r\\n\\r\\n' >&3 && head -n1 <&3 | grep -q ' 200 '"]
    interval: 15s
    timeout: 5s
    retries: 3
    start_period: 180s
    start_interval: 5s
  mem_limit: 1536m

services:
  postgres:
    image: postgres:18
    restart: unless-stopped
    environment:
      POSTGRES_DB: keycloak
      POSTGRES_USER: keycloak
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - pgdata:/var/lib/postgresql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U keycloak -d keycloak"]
      interval: 10s
      timeout: 5s
      retries: 10

  keycloak1:
    <<: *keycloak
    command: ["start", "--optimized", "--cache-embedded-node-name=keycloak1"]
    ports:
      - "127.0.0.1:8081:8080"
      - "127.0.0.1:9001:9000"

  keycloak2:
    <<: *keycloak
    command: ["start", "--optimized", "--cache-embedded-node-name=keycloak2"]
    ports:
      - "127.0.0.1:8082:8080"
      - "127.0.0.1:9002:9000"

volumes:
  pgdata:

Several lines in that file exist because a simpler version failed during testing:

• The healthcheck reads the HTTP status line. The image has no curl, so the check uses the bash /dev/tcp device. The version that circulates on GitHub greps the body for UP, and it passes while Keycloak is still down: the DOWN response contains "Graceful Shutdown", "status": "UP" as a sub-check. Our first node was marked healthy 20 seconds into a schema migration that took another minute.
• start_interval and retries 3. With retries: 20 and a 15 second interval, a node with a dead database took 312 seconds to turn unhealthy. The start period now covers slow first boots, and a running node fails within 45 seconds.
• KC_SERVER_ASYNC_BOOTSTRAP false. Since the 26.6 line, Keycloak opens its HTTP port before initialization finishes. Nginx (the open source build has no active health checks) sends traffic to the half-started node and users get 503s. More on the measurement in the upgrade section.
• A fixed pool of 20 per node. Two nodes at the default maximum of 100 each would exhaust the PostgreSQL default of 100 connections. With the fixed pool, PostgreSQL showed exactly 40 Keycloak connections.
• Named cluster and nodes. KC_CACHE_EMBEDDED_CLUSTER_NAME and --cache-embedded-node-name are new in the 26.8 line. Without them, node names are random on every start, which makes the logs useless.

Generate the secrets into .env, readable by root only:

printf 'KC_VERSION=%s\nKC_DOMAIN=%s\nPOSTGRES_PASSWORD=%s\nKC_BOOTSTRAP_ADMIN_PASSWORD=%s\n' \
  "${KC_VERSION}" "${KC_DOMAIN}" "$(openssl rand -hex 24)" "$(openssl rand -hex 24)" | sudo tee /opt/keycloak/.env >/dev/null
sudo chmod 600 /opt/keycloak/.env

Hex passwords are deliberate. Keycloak mangles passwords containing $$ or ${ when they arrive through environment variables, and hex can never contain either. Recent releases also accept a KCRAW_ prefix that passes a value through literally, but the KC_ variables used here still expand $ expressions. Validate the file and build the image:

cd /opt/keycloak
sudo docker compose config --quiet && echo "compose file OK"
sudo docker compose build --quiet

Start the cluster one node at a time

The first node creates the schema, so it starts alone. --wait blocks until the healthcheck passes:

sudo docker compose up -d --wait postgres keycloak1

On a fresh database this took 46 seconds on Ubuntu 24.04:

 Container keycloak-postgres-1 Healthy
 Container keycloak-keycloak1-1 Healthy 

Bring up the second node:

sudo docker compose up -d --wait keycloak2

It joins in about 11 seconds. The cluster view in its log lists both members:

sudo docker compose logs keycloak2 | grep ISPN000094

The (2) is the member count:

keycloak2-1  | 2026-10-04 19:25:07,183 INFO  [org.infinispan.CLUSTER] (db-index-checker) ISPN000094: Received new cluster view for channel keycloak-prod: [keycloak1(v=16.0.15)|1] (2) [keycloak1(v=16.0.15), keycloak2(v=16.0.15)]

The same number is exposed as a metric on the management port, which is what you alert on in production. A cluster size of 1 on a two-node deployment means a split:

curl -s http://127.0.0.1:9001/metrics | grep ^vendor_cluster_size

Both nodes see each other:

vendor_cluster_size{cache_manager="keycloak",node="keycloak1"} 2.0

The terminal view of the running cluster, from the Ubuntu host:

Both nodes are healthy and see each other, so the cluster is ready for a proxy.

Put Nginx in front of Keycloak

The Keycloak reverse proxy guide lists which paths belong on the internet. Only /realms/, /resources/ and /.well-known/ need to be public. The admin console (/admin/) and the master realm (/realms/master/, where administrators log in) stay on the admin network, and the health and metrics endpoints are never proxied at all. Create the server block:

sudo vim /etc/nginx/sites-available/keycloak.conf

Paste the configuration as is. KC_DOMAIN_HERE, ADMIN_CIDR_HERE and REALM_HERE are placeholders that the next command fills in from your shell variables:

upstream keycloak {
    server 127.0.0.1:8081 max_fails=3 fail_timeout=10s;
    server 127.0.0.1:8082 max_fails=3 fail_timeout=10s;
    keepalive 32;
}

server {
    listen 80;
    listen [::]:80;
    server_name KC_DOMAIN_HERE;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;
    server_name KC_DOMAIN_HERE;

    ssl_certificate     /etc/letsencrypt/live/KC_DOMAIN_HERE/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/KC_DOMAIN_HERE/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_session_cache shared:keycloak_tls:10m;

    add_header Strict-Transport-Security "max-age=31536000" always;

    proxy_buffer_size 128k;
    proxy_buffers 8 128k;
    proxy_busy_buffers_size 256k;

    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Forwarded-Host $host;
    proxy_set_header X-Forwarded-Port $server_port;

    location = / {
        return 302 /realms/REALM_HERE/account/;
    }

    location /realms/ {
        proxy_pass http://keycloak;
    }

    location /realms/master/ {
        allow ADMIN_CIDR_HERE;
        allow 127.0.0.1;
        deny all;
        proxy_pass http://keycloak;
    }

    location /.well-known/ {
        proxy_pass http://keycloak;
    }

    location /resources/ {
        proxy_pass http://keycloak;
    }

    location /admin/ {
        allow ADMIN_CIDR_HERE;
        allow 127.0.0.1;
        deny all;
        proxy_pass http://keycloak;
    }

    location / {
        return 404;
    }
}

On Ubuntu 24.04, read the next subsection first, because nginx -t in this block fails there. Note X-Forwarded-For $remote_addr. The usual $proxy_add_x_forwarded_for appends to whatever the client sent, and Keycloak requires the proxy to overwrite the header. Otherwise a client can forge the IP that brute force detection and the event log record. The 128k proxy buffers follow the values the Keycloak community settled on in issue 23824 for large redirect headers. Fill in the placeholders and enable the site:

sudo sed -i -e "s/KC_DOMAIN_HERE/${KC_DOMAIN}/g" -e "s#ADMIN_CIDR_HERE#${ADMIN_CIDR}#g" -e "s/REALM_HERE/${REALM}/g" /etc/nginx/sites-available/keycloak.conf
sudo ln -s /etc/nginx/sites-available/keycloak.conf /etc/nginx/sites-enabled/
sudo rm -f /etc/nginx/sites-enabled/default
sudo nginx -t && sudo systemctl reload nginx

Ubuntu 24.04: nginx 1.24 rejects “http2 on”

Ubuntu 26.04 ships nginx 1.28 and Rocky Linux 10 ships 1.26, which both accept the http2 on; directive. Ubuntu 24.04 ships nginx 1.24, and nginx -t fails:

nginx: [emerg] unknown directive "http2" in /etc/nginx/sites-enabled/keycloak.conf:17
nginx: configuration file /etc/nginx/nginx.conf test failed

The standalone directive arrived in nginx 1.25.1. On 24.04, move http2 back onto the listen lines:

sudo sed -i -e "/http2 on;/d" -e "s/listen 443 ssl;/listen 443 ssl http2;/" -e "s/listen \[::\]:443 ssl;/listen [::]:443 ssl http2;/" /etc/nginx/sites-available/keycloak.conf
sudo nginx -t && sudo systemctl reload nginx

Confirm Keycloak answers through the proxy with its public hostname as the issuer. No application realm exists yet, so the check reads the master realm, which Nginx only serves to ADMIN_CIDR and to loopback. --resolve sends the request to Nginx over 127.0.0.1 while TLS and the Host header still use your public name. Without it, a server outside ADMIN_CIDR gets a 403 from its own proxy:

curl -s --resolve "${KC_DOMAIN}:443:127.0.0.1" "https://${KC_DOMAIN}/realms/master/.well-known/openid-configuration" | python3 -m json.tool | grep -E '^    "(issuer|token_endpoint)"'

Both URLs must start with https:// and your domain. An http:// issuer here means the proxy headers are not reaching Keycloak:

    "issuer": "https://keycloak.example.com/realms/master",
    "token_endpoint": "https://keycloak.example.com/realms/master/protocol/openid-connect/token",

We tested the path rules from a container on a Docker bridge network, which sits outside ADMIN_CIDR. The admin console and master realm returned 403, the application realm returned 200, and /metrics and /health/ready returned 404 through the proxy. The admin console login page, reached from the admin network:

From any address outside ADMIN_CIDR, the same URL returns 403.

Replace the temporary admin account

The bootstrap account is meant to be temporary, and Keycloak shows a banner until it is gone. Create a permanent admin with kcadm.sh, then delete the bootstrap user. The -T flag matters: without it, Docker allocates a TTY and every captured ID ends in a carriage return that breaks the next command.

KCADM="sudo docker compose -f /opt/keycloak/compose.yaml exec -T keycloak1 /opt/keycloak/bin/kcadm.sh"
$KCADM config credentials --server http://localhost:8080 --realm master --user tmpadmin \
  --password "$(sudo grep -oP '^KC_BOOTSTRAP_ADMIN_PASSWORD=\K.*' /opt/keycloak/.env)"

Read the new admin password without echoing it into your shell history (the read -rs -p form is bash syntax), then create the account:

read -rs -p "New Keycloak admin password: " KC_ADMIN_PASS; echo
$KCADM create users -r master -s username=kcadmin -s enabled=true -s email="${ADMIN_EMAIL}" -s emailVerified=true
$KCADM set-password -r master --username kcadmin --new-password "${KC_ADMIN_PASS}"
$KCADM add-roles -r master --uusername kcadmin --rolename admin

Log in as the new admin and delete the bootstrap user:

$KCADM config credentials --server http://localhost:8080 --realm master --user kcadmin --password "${KC_ADMIN_PASS}"
TMP_ID=$($KCADM get users -r master -q username=tmpadmin --fields id --format csv --noquotes)
$KCADM delete users/${TMP_ID} -r master
$KCADM get users -r master --fields username

Only the permanent admin remains:

[ {
  "username" : "kcadmin"
} ]

The bootstrap password is still sitting in .env and in the container environment. Strip both bootstrap lines and roll the nodes one at a time, which also proves that a restart costs no downtime:

sudo sed -i '/KC_BOOTSTRAP_ADMIN_/d' /opt/keycloak/compose.yaml /opt/keycloak/.env
cd /opt/keycloak && sudo docker compose up -d --wait keycloak1 && sudo docker compose up -d --wait keycloak2

Recreating keycloak1 wipes the kcadm.sh login stored inside it, so log in again before the next section:

$KCADM config credentials --server http://localhost:8080 --realm master --user kcadmin --password "${KC_ADMIN_PASS}"

Create a realm with brute force protection and passkeys

Applications never use the master realm. Create one for your users with a password policy, brute force detection and passkeys switched on:

$KCADM create realms -s realm="${REALM}" -s enabled=true -s displayName="${REALM_NAME}" \
  -s bruteForceProtected=true -s failureFactor=5 -s rememberMe=true \
  -s loginWithEmailAllowed=true -s duplicateEmailsAllowed=false -s sslRequired=external \
  -s "passwordPolicy=length(12) and notUsername and notEmail and passwordHistory(3)" \
  -s webAuthnPolicyPasswordlessPasskeysEnabled=true -s webAuthnPolicyPasswordlessRpEntityName="${REALM_NAME}"

The last two options are easy to miss. The passkeys feature is enabled server-wide, but a new realm ships with the passkeys switch off, so the login page offers nothing until you turn it on. With it on, the browser offers saved passkeys straight from the username field (conditional mediation, the autofill prompt):

With failureFactor=5 from the realm command above, brute force detection locks an account temporarily after five failures. The Keycloak default is 30:

The lockout behaves differently than the number suggests. In our test, a script fired wrong passwords faster than one per second, and the account was locked after the second failure, well before the fifth. That is the “Quick login check” (1000 ms): failures closer together than that trigger the minimum lockout straight away. The user still sees “Invalid username or password.” on every attempt, by design, so an attacker cannot tell a locked account from a wrong password. The event log is where you see the difference:

      3 error="invalid_user_credentials"
      4 error="user_temporarily_disabled"

In the 26.8 line, login failures are stored in the database by default, so a lockout survives a full cluster restart. Turn on the realm event log too, which is where the passkey and lockout proof in this guide came from:

$KCADM update events/config -r "${REALM}" -s eventsEnabled=true -s eventsExpiration=2592000 -s adminEventsEnabled=true

Add a groups claim, users and MFA

Applications map roles from group membership, so every token needs a groups claim. Create a client scope with a group membership mapper. full.path=false sends grafana-admins instead of /grafana-admins, which is what most applications expect:

$KCADM create client-scopes -r "${REALM}" -s name=groups -s protocol=openid-connect -s 'attributes."include.in.token.scope"=true'
SCOPE_ID=$($KCADM get client-scopes -r "${REALM}" --fields id,name --format csv --noquotes | grep ",groups$" | cut -d, -f1)
$KCADM create client-scopes/${SCOPE_ID}/protocol-mappers/models -r "${REALM}" -s name=groups -s protocol=openid-connect \
  -s protocolMapper=oidc-group-membership-mapper -s 'config."claim.name"=groups' -s 'config."full.path"=false' \
  -s 'config."id.token.claim"=true' -s 'config."access.token.claim"=true' -s 'config."userinfo.token.claim"=true'
$KCADM update "realms/${REALM}/default-default-client-scopes/${SCOPE_ID}"
for g in grafana-admins grafana-editors forgejo-admins; do $KCADM create groups -r "${REALM}" -s name=$g; done

Making the scope a realm default means every client created afterwards gets the claim. Clients created before this point need the scope added by hand. The three groups are the ones the optional Grafana and Forgejo examples map to roles. If you connect other applications instead, name the groups after them; the claim works the same way. Create a user and put them in a group. USER_EMAIL from the first step only has to be a mailbox you can read if you follow the optional SMTP section, which sends the password reset test there:

$KCADM create users -r "${REALM}" -s username=alice -s enabled=true -s firstName=Alice -s lastName=Admin -s email="${USER_EMAIL}" -s emailVerified=true
read -rs -p "Password for alice: " USER_PASS; echo
$KCADM set-password -r "${REALM}" --username alice --new-password "${USER_PASS}"
ALICE_ID=$($KCADM get users -r "${REALM}" -q username=alice -q exact=true --fields id --format csv --noquotes)
GROUP_ID=$($KCADM get groups -r "${REALM}" -q search=grafana-admins --fields id --format csv --noquotes | head -1)
$KCADM update users/${ALICE_ID}/groups/${GROUP_ID} -r "${REALM}" -n

To force an authenticator app on the next login of that user, add the required action. Use webauthn-register-passwordless instead to make them register a passkey:

$KCADM update users/${ALICE_ID} -r "${REALM}" -s 'requiredActions=["CONFIGURE_TOTP"]'

Check it the way a user would. Open https://keycloak.example.com/ in a private browser window (Nginx redirects it to the account console of your realm) and sign in as alice. Keycloak shows the QR code before letting her through, so have a TOTP app ready: Aegis (Android), 2FAS (Android and iOS), Google or Microsoft Authenticator, or your password manager. Reaching the account console means the core setup works:

We tested the passkey path with a Chromium virtual authenticator. After registration, the next login completed from the passkey alone. The event below is for bob, a second test user we created the same way as alice:

LOGIN {'credential_type': 'webauthn-passwordless', 'web_authn_authenticator_user_verification_checked': 'true', 'auth_method': 'openid-connect', 'auth_type': 'code', 'username': 'bob', 'response_mode': 'query'}

Optional: configure SMTP for password reset mail

Keycloak runs fine without mail: users sign in with password plus MFA, and an admin resets a forgotten password in the admin console under Users, Credentials. That is why the realm command above leaves “Forgot Password?” switched off, since without a relay the link would accept a request and never send anything. If you have an SMTP account (relay host, username, password and an allowed sender address), set the non-secret values:

export SMTP_HOST="smtp.example.com"
export SMTP_USER="[email protected]"
export SMTP_FROM="[email protected]"

Then point the realm at the relay and switch on the “Forgot Password?” link in the same command. Port 587 with STARTTLS is the common case:

read -rs -p "SMTP password: " SMTP_PASS; echo
$KCADM update "realms/${REALM}" -s resetPasswordAllowed=true -s smtpServer.host="${SMTP_HOST}" -s smtpServer.port=587 \
  -s smtpServer.starttls=true -s smtpServer.auth=true -s smtpServer.user="${SMTP_USER}" \
  -s smtpServer.password="${SMTP_PASS}" -s smtpServer.from="${SMTP_FROM}" -s smtpServer.fromDisplayName="${REALM_NAME} SSO"

Our lab pointed the realm at a Mailpit container instead of a real relay. The reset request arrived with a link built from the public hostname, which is the part that breaks when the proxy headers are wrong:

If you test with Mailpit too, mount a volume for it. Its default store is in memory, and a host reboot wiped every message we had captured.

Connect Grafana to Keycloak with OpenID Connect

The core setup ends here: Keycloak runs clustered behind Nginx with MFA, passkeys and brute force protection. This section and the Forgejo one after it are optional examples of connecting applications, so follow the ones that match what you run. This example runs Grafana on the Keycloak host. Pick its hostname, create a DNS A record for it, and request its certificate with the same command as before (or the DNS-01 version):

export GRAFANA_DOMAIN="grafana.example.com"
sudo certbot certonly --nginx -d "${GRAFANA_DOMAIN}" --non-interactive --agree-tos -m "${ADMIN_EMAIL}" \
  --deploy-hook "systemctl reload nginx"

Grafana is a good first client because it maps groups to roles. It runs here as a container next to Keycloak, though a package install of Grafana on Ubuntu takes the same OAuth settings in grafana.ini. Register it as a confidential client with PKCE:

$KCADM create clients -r "${REALM}" -s clientId=grafana -s name=Grafana -s enabled=true -s publicClient=false \
  -s standardFlowEnabled=true -s directAccessGrantsEnabled=false -s rootUrl="https://${GRAFANA_DOMAIN}" \
  -s "redirectUris=[\"https://${GRAFANA_DOMAIN}/login/generic_oauth\"]" -s "webOrigins=[\"https://${GRAFANA_DOMAIN}\"]" \
  -s "attributes.\"post.logout.redirect.uris\"=https://${GRAFANA_DOMAIN}/login" -s 'attributes."pkce.code.challenge.method"=S256'

Write the Grafana .env straight from the client secret, so the secret never appears on screen:

sudo mkdir -p /opt/grafana
CLIENT_ID=$($KCADM get clients -r "${REALM}" -q clientId=grafana --fields id --format csv --noquotes)
printf 'KC_DOMAIN=%s\nGRAFANA_DOMAIN=%s\nREALM=%s\nGRAFANA_CLIENT_SECRET=%s\n' "${KC_DOMAIN}" "${GRAFANA_DOMAIN}" "${REALM}" \
  "$($KCADM get clients/${CLIENT_ID}/client-secret -r "${REALM}" --fields value --format csv --noquotes)" | sudo tee /opt/grafana/.env >/dev/null
sudo chmod 600 /opt/grafana/.env

Create the Grafana Compose file:

sudo vim /opt/grafana/compose.yaml

The role expression reads the groups claim from the scope created earlier. The Grafana Keycloak documentation reads a top-level roles claim instead, which Keycloak does not emit without an extra mapper (realm roles sit under realm_access.roles). PKCE is off by default in Grafana, hence USE_PKCE:

name: grafana

services:
  grafana:
    image: grafana/grafana:13.2.3 # https://github.com/grafana/grafana/releases
    restart: unless-stopped
    ports:
      - "127.0.0.1:3000:3000"
    environment:
      GF_SERVER_ROOT_URL: https://${GRAFANA_DOMAIN}/
      GF_AUTH_GENERIC_OAUTH_ENABLED: "true"
      GF_AUTH_GENERIC_OAUTH_NAME: Keycloak
      GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP: "true"
      GF_AUTH_GENERIC_OAUTH_CLIENT_ID: grafana
      GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET: ${GRAFANA_CLIENT_SECRET}
      GF_AUTH_GENERIC_OAUTH_SCOPES: openid email profile groups
      GF_AUTH_GENERIC_OAUTH_USE_PKCE: "true"
      GF_AUTH_GENERIC_OAUTH_AUTH_URL: https://${KC_DOMAIN}/realms/${REALM}/protocol/openid-connect/auth
      GF_AUTH_GENERIC_OAUTH_TOKEN_URL: https://${KC_DOMAIN}/realms/${REALM}/protocol/openid-connect/token
      GF_AUTH_GENERIC_OAUTH_API_URL: https://${KC_DOMAIN}/realms/${REALM}/protocol/openid-connect/userinfo
      GF_AUTH_SIGNOUT_REDIRECT_URL: https://${KC_DOMAIN}/realms/${REALM}/protocol/openid-connect/logout?client_id=grafana&post_logout_redirect_uri=https%3A%2F%2F${GRAFANA_DOMAIN}%2Flogin
      GF_AUTH_GENERIC_OAUTH_LOGIN_ATTRIBUTE_PATH: preferred_username
      GF_AUTH_GENERIC_OAUTH_EMAIL_ATTRIBUTE_PATH: email
      GF_AUTH_GENERIC_OAUTH_NAME_ATTRIBUTE_PATH: name
      GF_AUTH_GENERIC_OAUTH_ROLE_ATTRIBUTE_PATH: "contains(groups[*], 'grafana-admins') && 'Admin' || contains(groups[*], 'grafana-editors') && 'Editor' || 'Viewer'"
    volumes:
      - grafana-data:/var/lib/grafana

volumes:
  grafana-data:

Start Grafana, then add its server block. The certificate came from the earlier certbot run:

cd /opt/grafana && sudo docker compose up -d
sudo vim /etc/nginx/sites-available/grafana.conf

Paste this server block. The Upgrade headers carry the Grafana Live websocket:

server {
    listen 80;
    listen [::]:80;
    server_name GRAFANA_DOMAIN_HERE;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    http2 on;
    server_name GRAFANA_DOMAIN_HERE;

    ssl_certificate     /etc/letsencrypt/live/GRAFANA_DOMAIN_HERE/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/GRAFANA_DOMAIN_HERE/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;

    client_max_body_size 512m;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $http_connection;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Fill in the domain and enable it. On Ubuntu 24.04, apply the same http2 sed from the Keycloak section to this file before testing:

sudo sed -i "s/GRAFANA_DOMAIN_HERE/${GRAFANA_DOMAIN}/g" /etc/nginx/sites-available/grafana.conf
sudo ln -s /etc/nginx/sites-available/grafana.conf /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

The Grafana login page now has a Keycloak button:

Clicking it lands on the login page of the realm, branded with the realm display name:

We drove the login with a headless browser on both the Ubuntu 26.04 and 24.04 stacks. alice, in grafana-admins, arrived with the Admin role. bob, in no Grafana group, got Viewer. Grafana marks both accounts as synced from Generic OAuth:

Changing a user’s group in Keycloak changes the Grafana role at that user’s next login.

Connect Forgejo or Gitea

This optional example connects a Forgejo or Gitea you already run with HTTPS on its own name. It needs no DNS record or certificate on the Keycloak host, only the hostname:

export FORGEJO_DOMAIN="git.example.com"

Forgejo shows the groups claim working for a second kind of application. The commands below create a forgejo client and put alice in forgejo-admins, the group Forgejo will map to site admin. The last segment of the redirect URI must match the auth source name used below:

$KCADM create clients -r "${REALM}" -s clientId=forgejo -s name=Forgejo -s enabled=true -s publicClient=false \
  -s standardFlowEnabled=true -s directAccessGrantsEnabled=false -s rootUrl="https://${FORGEJO_DOMAIN}" \
  -s "redirectUris=[\"https://${FORGEJO_DOMAIN}/user/oauth2/keycloak/callback\"]" -s "webOrigins=[\"https://${FORGEJO_DOMAIN}\"]"
FJ_GROUP=$($KCADM get groups -r "${REALM}" -q search=forgejo-admins --fields id --format csv --noquotes | head -1)
$KCADM update users/${ALICE_ID}/groups/${FJ_GROUP} -r "${REALM}" -n
FJ_ID=$($KCADM get clients -r "${REALM}" -q clientId=forgejo --fields id --format csv --noquotes)
$KCADM get clients/${FJ_ID}/client-secret -r "${REALM}" --fields value --format csv --noquotes

The last command prints the client secret, which the next step reads at a prompt. Then, from the Compose directory of a running Forgejo (the Forgejo setup guide covers the install), add Keycloak as an OpenID Connect source:

export KC_DOMAIN="keycloak.example.com" REALM="example"
read -rs -p "Forgejo client secret: " FORGEJO_SECRET; echo
sudo docker compose exec -T -u git forgejo forgejo admin auth add-oauth --name keycloak --provider openidConnect \
  --key forgejo --secret "${FORGEJO_SECRET}" \
  --auto-discover-url "https://${KC_DOMAIN}/realms/${REALM}/.well-known/openid-configuration" \
  --scopes "openid email profile groups" --group-claim-name groups --admin-group forgejo-admins --skip-local-2fa

Set [oauth2_client] USERNAME = preferred_username and ENABLE_AUTO_REGISTRATION = true in the Forgejo config, otherwise the first login stops at a “complete your account” form. --skip-local-2fa is safe here because Keycloak already enforced TOTP. On a brand-new Forgejo instance, the first account becomes a site admin whatever its groups, so test the mapping with a second user. In our replay, dave (no group) got a 403 on /admin and erin (in forgejo-admins) got the admin dashboard. alice signed in with password plus TOTP and opened site administration:

Every client in the realm, including the built-in ones, is listed in the admin console:

Both application clients carry the groups claim from the realm default scope.

Test failover before you trust it

A cluster that has never lost a node is a hope, not a design. We logged a user into the account console, then broke things while reloading the page:

Action	Still logged in	Page load
Baseline, both nodes up	Yes	0.9 s
docker kill keycloak1	Yes	2.5 s (Nginx retries the dead upstream)
Second reload, keycloak1 still dead	Yes	0.8 s
keycloak1 back, then keycloak2 killed	Yes	2.3 s
Both nodes stopped and started again	Yes	3.3 s

The last row is the one older guides get wrong. Persistent user sessions have been the default since Keycloak 26.0, so sessions live in PostgreSQL and survive even a full cluster restart. No sticky sessions were configured: the 26.8 line deprecates the old trick of adding a route suffix to the AUTH_SESSION_ID cookie, and plain round robin worked through every test. The route is still attached by default, so each node logs a warning at startup that shouldAttachRoute is deprecated and names the option that turns it off, spi-sticky-session-encoder--infinispan--should-attach-route=false.

Back up the database and test a restore

The database is the whole state of Keycloak. The partial export in the admin console masks secrets and is not a backup, and kc.sh export is only consistent with every node stopped. Use pg_dump. Create the script:

sudo vim /usr/local/bin/keycloak-backup.sh

umask 077 is there for a reason. Our first version wrote world-readable dumps, and a dump contains every password hash and client secret in the realm:

#!/usr/bin/env bash
set -euo pipefail
umask 077
BACKUP_DIR=/var/backups/keycloak
KEEP_DAYS=14
STAMP=$(date +%Y%m%d-%H%M%S)

mkdir -p "$BACKUP_DIR"
cd /opt/keycloak
docker compose exec -T postgres pg_dump -U keycloak -d keycloak -Fc > "$BACKUP_DIR/keycloak-$STAMP.dump.tmp"
mv "$BACKUP_DIR/keycloak-$STAMP.dump.tmp" "$BACKUP_DIR/keycloak-$STAMP.dump"
find "$BACKUP_DIR" -name 'keycloak-*.dump' -mtime +"$KEEP_DAYS" -delete
echo "backup ok: $BACKUP_DIR/keycloak-$STAMP.dump ($(du -h "$BACKUP_DIR/keycloak-$STAMP.dump" | cut -f1))"

Make it executable and create the service unit:

sudo chmod 750 /usr/local/bin/keycloak-backup.sh
sudo vim /etc/systemd/system/keycloak-backup.service

Add the oneshot service:

[Unit]
Description=Back up the Keycloak PostgreSQL database
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
ExecStart=/usr/local/bin/keycloak-backup.sh

Then the timer that fires it nightly:

sudo vim /etc/systemd/system/keycloak-backup.timer

With Persistent=true, a run missed while the host was off happens at the next boot:

[Unit]
Description=Nightly Keycloak database backup

[Timer]
OnCalendar=*-*-* 02:30:00
RandomizedDelaySec=15m
Persistent=true

[Install]
WantedBy=timers.target

Enable the timer and take the first backup by hand:

sudo systemctl daemon-reload
sudo systemctl enable --now keycloak-backup.timer
sudo systemctl start keycloak-backup.service
sudo ls -la /var/backups/keycloak/

The directory and the dump are readable by root only:

drwx------ 2 root root   4096 Oct  4 19:28 .
drwxr-xr-x 3 root root   4096 Oct  4 19:28 ..
-rw------- 1 root root 260259 Oct  4 19:28 keycloak-20261004-192830.dump

Copy the dumps off the host as well, because a backup on the same disk is not a backup. A backup you have never restored is not one either. This drill restores the newest dump into a throwaway stack on other ports, next to the live one:

sudo mkdir -p /opt/keycloak-restore && cd /opt/keycloak-restore
sudo cp /opt/keycloak/Dockerfile /opt/keycloak/.env .
sed -e "s/^name: keycloak/name: kcrestore/" -e "s/keycloak-prod/kcrestore/" -e "s/127.0.0.1:808\([12]\)/127.0.0.1:809\1/" \
  -e "s/127.0.0.1:900\([12]\)/127.0.0.1:919\1/" /opt/keycloak/compose.yaml | sudo tee compose.yaml >/dev/null
DUMP=$(sudo ls -t /var/backups/keycloak/ | head -1)
sudo docker compose up -d --wait postgres
sudo cat "/var/backups/keycloak/${DUMP}" | sudo docker compose exec -T postgres pg_restore -U keycloak -d keycloak --clean --if-exists --no-owner
sudo docker compose up -d --wait keycloak1

Check the realms and the issuer on the restored node:

sudo docker compose exec -T postgres psql -U keycloak -d keycloak -tAc "select name from realm order by 1"
curl -s "http://127.0.0.1:8091/realms/${REALM}/.well-known/openid-configuration" | grep -o '"issuer":"[^"]*"'

Both realms came back and the node booted healthy from the dump:

example
master
"issuer":"https://keycloak.example.com/realms/example"

Tear the drill stack down with sudo docker compose down -v from /opt/keycloak-restore. The dump includes the JDBC_PING table, which still lists the live nodes under the keycloak-prod cluster name, so the drill renames its cluster to kcrestore. The restored node logged its own one-member view (ISPN000094 ... channel kcrestore ... (1)) while the live cluster kept reporting vendor_cluster_size 2.

Upgrade Keycloak without guessing

Nothing in this section runs on day one. Come back when a new Keycloak release is out. Rolling upgrades only work between patch releases of the same minor version, such as 26.8.0 to 26.8.1. Keycloak ships a checker that tells you which kind of upgrade you face. We built this lab on 26.7.5 and upgraded it to 26.8.0. Record the metadata of the running version first:

cd /opt/keycloak
sudo mkdir -p upgrade && sudo chmod 777 upgrade
sudo docker compose run --rm --no-deps -v /opt/keycloak/upgrade:/upgrade keycloak1 update-compatibility metadata --optimized --file=/upgrade/current.json

Refresh KC_VERSION in your shell first, because a new session has it empty or still set to the old release. Then write it to .env, rebuild, and run the check with the new image:

export KC_VERSION=$(curl -s https://api.github.com/repos/keycloak/keycloak/releases/latest | grep '"tag_name"' | cut -d '"' -f 4)
sudo sed -i "s/^KC_VERSION=.*/KC_VERSION=${KC_VERSION}/" .env
sudo docker compose build --quiet
sudo docker compose run --rm --no-deps -v /opt/keycloak/upgrade:/upgrade keycloak1 update-compatibility check --optimized --file=/upgrade/current.json; echo "exit=$?"

Exit code 0 means you can roll one node at a time. Ours returned 3, and the line before it names the reason:

[cacheEmbedded] Provider is compatible.
[cacheRemote] Provider is compatible.
[database] Provider is compatible.
[feature-compatibility] Rolling Update is not available. 'feature-compatibility.login-failures' is incompatible: null -> {"enabled":true,"version":2,"updatePolicy":"SHUTDOWN"}.
exit=3

That is the database-backed lockout store mentioned earlier, and its update policy is SHUTDOWN. So 26.7 to 26.8 needs every node stopped. Take a fresh backup, then stop both nodes and start them on the new image one at a time:

sudo systemctl start keycloak-backup.service
sudo docker compose stop keycloak1 keycloak2
sudo docker compose up -d --wait keycloak1
sudo docker compose up -d --wait keycloak2

A probe hitting the realm discovery URL every second through Nginx counted 19 seconds of downtime: 15 responses of 502 while no node was up, then the first node passed its schema migration and served traffic. The second node rejoined the cluster.

The same probe also caught 3 responses of 503 when the second node started. That is the async bootstrap problem from the Compose section: the port opens, Nginx sends traffic, and Keycloak answers 503 until it finishes initializing. We restarted one node with the probe running, once with each setting:

KC_SERVER_ASYNC_BOOTSTRAP	Requests during a node restart	Errors
true (default)	201	4 x 503
false	198	0

With async bootstrap off, the port stays closed until the node is ready. Nginx gets a refused connection and moves to the other node, which is exactly what you want from a load balancer without health checks.

Run Keycloak with Podman Quadlets on Rocky Linux

On RHEL family hosts the native way to run this stack is Podman with Quadlets: systemd generates a unit from each .container file, so the stack starts at boot without a Compose tool. Our Podman Quadlet walkthrough covers the unit types in more depth. Podman is in the base repositories. Certbot and its DNS plugins come from EPEL. Cloud and minimal images can ship without firewalld, so install it too:

sudo dnf -y install epel-release
sudo dnf -y install podman nginx certbot python3-certbot-nginx firewalld
sudo mkdir -p /opt/keycloak

Unlike Ubuntu, Rocky neither starts Nginx on install nor lets port 80 through the firewall, and the HTTP-01 challenge needs both. Open the ports and start Nginx before asking for the certificate:

sudo systemctl enable --now firewalld
sudo firewall-cmd --permanent --add-service=http --add-service=https
sudo firewall-cmd --reload
sudo systemctl enable --now nginx

Now issue the certificate with the same certbot command as the Docker section. For DNS-01 on Rocky, the Cloudflare plugin is sudo dnf -y install python3-certbot-dns-cloudflare from EPEL, and the --deploy-hook matters just as much. The EPEL certbot package enables its own certbot-renew.timer, which systemctl is-enabled certbot-renew.timer confirms. The Containerfile is the same file as the Dockerfile above:

sudo vim /opt/keycloak/Containerfile

Paste the two-stage build:

ARG KC_VERSION=26.8.0
FROM quay.io/keycloak/keycloak:${KC_VERSION} AS builder
ENV KC_DB=postgres \
    KC_HEALTH_ENABLED=true \
    KC_METRICS_ENABLED=true
WORKDIR /opt/keycloak
RUN /opt/keycloak/bin/kc.sh build

FROM quay.io/keycloak/keycloak:${KC_VERSION}
COPY --from=builder /opt/keycloak/ /opt/keycloak/
ENTRYPOINT ["/opt/keycloak/bin/kc.sh"]

Podman secrets replace the .env file. They are created once and injected at runtime, so no password lands in a unit file:

openssl rand -hex 24 | tr -d '\n' | sudo podman secret create kc_db_password -
openssl rand -hex 24 | tr -d '\n' | sudo podman secret create kc_bootstrap_admin_password -
sudo podman secret ls

Quadlet files for root live in /etc/containers/systemd/. Start with the network:

sudo vim /etc/containers/systemd/keycloak.network

One line names it:

[Network]
NetworkName=keycloak

The volume for PostgreSQL data:

sudo vim /etc/containers/systemd/keycloak-pgdata.volume

Add:

[Volume]
VolumeName=keycloak-pgdata

A .build unit builds the optimized image before any container needs it:

sudo vim /etc/containers/systemd/keycloak.build

The build argument key is BuildArg=. We first wrote Arg=, and the generator rejected it with unsupported key 'Arg' in group 'Build':

[Build]
ImageTag=localhost/keycloak-optimized:26.8.0
File=/opt/keycloak/Containerfile
SetWorkingDirectory=file
BuildArg=KC_VERSION=26.8.0

PostgreSQL reads its password from the secret file Podman mounts at /run/secrets/:

sudo vim /etc/containers/systemd/keycloak-postgres.container

Notify=healthy holds the unit in “activating” until the healthcheck passes, so anything ordered after it waits for a usable database:

[Unit]
Description=PostgreSQL for Keycloak

[Container]
ContainerName=keycloak-postgres
Image=docker.io/library/postgres:18
Network=keycloak.network
Volume=keycloak-pgdata.volume:/var/lib/postgresql
Environment=POSTGRES_DB=keycloak POSTGRES_USER=keycloak POSTGRES_PASSWORD_FILE=/run/secrets/kc_db_password
Secret=kc_db_password
HealthCmd=pg_isready -U keycloak -d keycloak
HealthInterval=10s
HealthRetries=10
Notify=healthy

[Service]
Restart=always
TimeoutStartSec=300

[Install]
WantedBy=multi-user.target

Both Keycloak nodes come from one template unit, where %i is the instance number:

sudo vim /etc/containers/systemd/[email protected]

Paste the template as is:

[Unit]
Description=Keycloak node %i
Requires=keycloak-postgres.service
After=keycloak-postgres.service

[Container]
ContainerName=keycloak%i
HostName=keycloak%i
Image=keycloak.build
Exec=start --optimized --cache-embedded-node-name=keycloak%i
Network=keycloak.network
PublishPort=127.0.0.1:808%i:8080
PublishPort=127.0.0.1:900%i:9000
Environment=KC_DB_URL=jdbc:postgresql://keycloak-postgres:5432/keycloak
Environment=KC_DB_USERNAME=keycloak
Environment=KC_DB_POOL_INITIAL_SIZE=20 KC_DB_POOL_MIN_SIZE=20 KC_DB_POOL_MAX_SIZE=20
Environment=KC_CACHE_EMBEDDED_CLUSTER_NAME=keycloak-prod
Environment=KC_HOSTNAME=https://KC_DOMAIN_HERE
Environment=KC_PROXY_HEADERS=xforwarded
Environment=KC_HTTP_ENABLED=true
Environment=KC_SERVER_ASYNC_BOOTSTRAP=false
Environment=KC_BOOTSTRAP_ADMIN_USERNAME=tmpadmin
Environment="JAVA_OPTS_KC_HEAP=-XX:MaxRAMPercentage=70 -XX:InitialRAMPercentage=50"
Secret=kc_db_password,type=env,target=KC_DB_PASSWORD
Secret=kc_bootstrap_admin_password,type=env,target=KC_BOOTSTRAP_ADMIN_PASSWORD
PodmanArgs=--memory=1536m
HealthCmd=bash -c "exec 3<>/dev/tcp/127.0.0.1/9000 && printf 'GET /health/ready HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n' >&3 && head -n1 <&3 | grep -q ' 200 '"
HealthInterval=15s
HealthTimeout=5s
HealthRetries=3
HealthStartPeriod=180s
Notify=healthy

[Service]
Restart=always
TimeoutStartSec=600

[Install]
WantedBy=multi-user.target

Then put your hostname in place of KC_DOMAIN_HERE:

sudo sed -i "s/KC_DOMAIN_HERE/${KC_DOMAIN}/" /etc/containers/systemd/[email protected]

One trap cost us a reboot. WantedBy= in a template file does not enable any instance, so after a reboot PostgreSQL was up and both Keycloak units were inactive. Enable the instances by symlinking the template once per node:

cd /etc/containers/systemd
sudo ln -s [email protected] [email protected]
sudo ln -s [email protected] [email protected]
sudo systemctl daemon-reload
systemctl list-unit-files 'keycloak*' --no-pager

All units are generated, including one per instance:

UNIT FILE                      STATE     PRESET
keycloak-network.service       generated -
keycloak-pgdata-volume.service generated -
keycloak-postgres.service      generated -
[email protected]              generated -
[email protected]             generated -
[email protected]             generated -

Start the first node. Thanks to Notify=healthy, systemctl start returns only once Keycloak is ready. The first start took 78 seconds, image build included:

sudo systemctl start [email protected]
sudo systemctl start [email protected]
sudo podman ps --format "{{.Names}} {{.Image}} {{.Status}}"

All three containers report healthy and the cluster view has two members:

keycloak-postgres docker.io/library/postgres:18 Up About a minute (healthy)
keycloak1 localhost/keycloak-optimized:26.8.0 Up 51 seconds (healthy)
keycloak2 localhost/keycloak-optimized:26.8.0 Up 16 seconds (healthy)

Nginx takes the same server block as the Docker setup. Rocky has no sites-enabled, so it goes in conf.d:

sudo vim /etc/nginx/conf.d/keycloak.conf

Paste the keycloak.conf content from the Nginx section, then fill in the placeholders and test the syntax:

sudo sed -i -e "s/KC_DOMAIN_HERE/${KC_DOMAIN}/g" -e "s#ADMIN_CIDR_HERE#${ADMIN_CIDR}#g" -e "s/REALM_HERE/${REALM}/g" /etc/nginx/conf.d/keycloak.conf
sudo nginx -t && sudo systemctl reload nginx

With SELinux enforcing, the first request fails with a 502 and this line in /var/log/nginx/error.log:

[crit] 53681#53681: *1 connect() to 127.0.0.1:8082 failed (13: Permission denied) while connecting to upstream

SELinux stops the httpd domain from opening outbound connections by default. Allow it with the boolean, not by disabling SELinux:

sudo setsebool -P httpd_can_network_connect 1

After a reboot with the symlinks in place, all three containers came back healthy with no manual start, and a token request through Nginx returned a signed access token. Every kcadm.sh step from the Docker sections works the same, using sudo podman exec keycloak1 /opt/keycloak/bin/kcadm.sh. Once the permanent admin exists, delete the two bootstrap lines from [email protected], run sudo podman secret rm kc_bootstrap_admin_password, reload systemd and restart the nodes one at a time.

Upgrades follow the same rules as on Docker, including the full stop for a minor release. The version sits in two places in keycloak.build, so update both with the freshly exported KC_VERSION, then restart the build unit before the nodes:

sudo sed -i -E "s/(:|=)[0-9]+\.[0-9]+\.[0-9]+$/\1${KC_VERSION}/" /etc/containers/systemd/keycloak.build
sudo systemctl daemon-reload
sudo systemctl restart keycloak-build.service

For a minor release, stop both nodes and bring them back one at a time so the first node runs the schema migration alone. Notify=healthy makes each start wait until the node is ready:

sudo systemctl stop [email protected] [email protected]
sudo systemctl start [email protected]
sudo systemctl start [email protected]

A patch release that the compatibility check clears can instead restart one node at a time with no downtime.

Errors we hit and how we fixed them

Unknown option: '--cache-embedded-node-name'

The node restarts in a loop with this error, followed by Possible solutions: --cache-embedded-mtls-enabled, .... The node and cluster name options exist only from the 26.8 line onward. On 26.7 and older, drop --cache-embedded-node-name and KC_CACHE_EMBEDDED_CLUSTER_NAME, or upgrade first and add them afterwards.

Container is healthy but /health/ready returns 503

Your healthcheck greps the body for UP. While Keycloak initializes, /health/ready returns 503 with "status": "DOWN" overall but "Graceful Shutdown": "UP" inside, so the grep matches. Check the status line for 200 as the Compose file above does.

Go-live checklist

Two steps protect the admin account before you go live. First, give scripts their own login. kcadm.sh cannot answer a one-time code, so once kcadmin has TOTP, its password login fails for good: in our test it returned Account is not fully set up [invalid_grant] while enrollment was pending and Invalid user credentials [invalid_grant] after it. A confidential client with a service account keeps the CLI working. The last command logs kcadm in with the new client. Copy the secret into your password manager from the admin console (Clients, kcadm-automation, Credentials), because later config credentials calls use it instead of the admin password:

$KCADM create clients -r master -s clientId=kcadm-automation -s enabled=true -s publicClient=false \
  -s serviceAccountsEnabled=true -s standardFlowEnabled=false -s directAccessGrantsEnabled=false
$KCADM add-roles -r master --uusername service-account-kcadm-automation --rolename admin
SA_ID=$($KCADM get clients -r master -q clientId=kcadm-automation --fields id --format csv --noquotes)
KCADM_SECRET=$($KCADM get clients/${SA_ID}/client-secret -r master --fields value --format csv --noquotes)
$KCADM config credentials --server http://localhost:8080 --realm master --client kcadm-automation --secret "${KCADM_SECRET}"

Then require TOTP for kcadmin. The next browser login to the admin console asks for the enrollment:

ADMIN_ID=$($KCADM get users -r master -q username=kcadmin -q exact=true --fields id --format csv --noquotes)
$KCADM update users/${ADMIN_ID} -r master -s 'requiredActions=["CONFIGURE_TOTP"]'

Run through this list before pointing real users at the stack. Every item was checked on the lab hosts:

• /admin/ and /realms/master/ return 403 from outside ADMIN_CIDR, and /metrics and /health are not reachable through the proxy.
• The bootstrap admin is deleted and KC_BOOTSTRAP_ADMIN_* is gone from compose.yaml and .env.
• vendor_cluster_size reads 2 on both management ports, and an alert fires when it drops.
• Killing either node keeps users logged in.
• A restore drill from the newest nightly dump boots a healthy node, and the dumps are also stored off the host.
• Brute force detection, the password policy and the passkeys switch are on in every application realm, not only the first one.
• Administrators use TOTP or passkeys on the master realm too.
• The certbot renewal timer is active (systemctl list-timers | grep certbot), every renewal file has a renew_hook or installer = nginx line, and certbot renew --dry-run --run-deploy-hooks passes.
• The Keycloak version is on a patch release you can roll, and you know how long a minor upgrade takes from the measured 19 seconds of downtime.

With Keycloak in place, the next step for most teams is pointing their user directory at it. If accounts already live in FreeIPA, the Keycloak LDAP federation can read them directly, and our FreeIPA server install guide sets up that side.