K3s installs on Rocky Linux 10 with one curl command, and the node even reports Ready. On a lean Rocky image the cluster behind it is still broken. Without kernel-modules-extra you get no br_netfilter, so CoreDNS is still not ready four minutes in, and with firewalld on and no rules your agents can’t join. SELinux mostly takes care of itself, because the installer pulls k3s-selinux for you.
You’ll build a two-node cluster the way you would run it on a real server, with SELinux enforcing, firewalld on, and each Rocky-specific step proven by breaking it first. For a single throwaway node, the K3s quickstart for Ubuntu and Rocky is shorter, and for upstream Kubernetes there’s our kubeadm guide for Rocky Linux 10. We ran every command below on Rocky Linux 10.2 with K3s v1.36.5+k3s1 from the stable channel in November 2026, on two Proxmox VMs.
Plan the K3s cluster on Rocky Linux
A K3s server runs the API server, scheduler, controller manager and the datastore in one process, and it also runs your workloads unless you taint it. Agents open an outbound connection to the server and keep it, pods on different nodes talk over flannel’s VXLAN overlay, and metrics-server scrapes each kubelet directly. So firewalld has to let these through:
- 6443/tcp on the server: API and agent join
- 8472/udp between nodes: flannel VXLAN
- 10250/tcp between nodes: kubelet metrics
- 80/tcp and 443/tcp: the bundled Traefik ingress
Here is how those pieces sit on the two lab nodes, with traffic you send through Traefik reaching a pod on either one.
We used two Rocky Linux 10.2 VMs on the same subnet, each patched with the usual Rocky Linux 10 post-install steps. Swap in your own names and addresses wherever these appear.
rocky-k3s01, 10.0.1.50: server, 2 vCPU, 4 GB RAMrocky-k3s02, 10.0.1.51: agent, 2 vCPU, 2 GB RAM- SELinux enforcing, firewalld active, user with sudo on both
Size the server from the K3s requirements page, which sets the floor at 2 cores and 2 GB for a server and 1 core with 512 MB for an agent. Treat that as a lab minimum. With Traefik, CoreDNS, metrics-server and two nginx pods running, our server process held about 897 MB resident and the node showed 1,571 MB used, while the agent sat at 498 MB used. For real workloads, start the server at 2 vCPU and 4 GB, which the K3s sizing guide rates for up to 350 agents, and size agents by what you schedule on them.
Install kernel-modules-extra on every node
For RHEL 10, the K3s docs carry one extra line: install kernel-modules-extra. It looks optional, and K3s does not refuse to start without it, which is exactly why you’ll be tempted to skip it. On Rocky 10 that package ships the netfilter modules kube-proxy and flannel write their rules with, and our cloud image had none of them:
br_netfilteriptable_natandiptable_filterxt_comment,xt_conntrack,xt_MASQUERADE
Patch the system first. The package installs modules for the newest installed kernel, not the one you are running, so a pending kernel update leaves you with modules that don’t match uname -r. Our clone booted 6.12.0-211.16.1 while the first-boot upgrade had already staged 211.72.1, so you reboot before anything else.
sudo dnf -y upgrade
sudo dnf -y install kernel-modules-extra
sudo systemctl rebootOnce the node is back, confirm the package version matches the running kernel. Both lines you get back must carry the same build string.
uname -r; rpm -q kernel-modules-extraOn the lab server they matched after the reboot.
6.12.0-211.72.1.el10_2.x86_64
kernel-modules-extra-6.12.0-211.72.1.el10_2.x86_64If you want to see which package owns the modules K3s loads, ask modinfo for each file and rpm for its owner.
for m in xt_comment xt_conntrack xt_MASQUERADE br_netfilter iptable_nat; do echo "$m -> $(rpm -qf $(modinfo -n $m))"; doneEvery one of them comes from kernel-modules-extra, not from the base kernel packages.
xt_comment -> kernel-modules-extra-6.12.0-211.72.1.el10_2.x86_64
xt_conntrack -> kernel-modules-extra-6.12.0-211.72.1.el10_2.x86_64
xt_MASQUERADE -> kernel-modules-extra-6.12.0-211.72.1.el10_2.x86_64
br_netfilter -> kernel-modules-extra-6.12.0-211.72.1.el10_2.x86_64
iptable_nat -> kernel-modules-extra-6.12.0-211.72.1.el10_2.x86_64One surprise from the lab. Installing firewalld on the bare cloud image pulled kernel-modules-extra in as a weak dependency, through iptables-nft and ipset. So a box that already has firewalld may have the modules by accident. Don’t rely on it, because install_weak_deps=False or a minimal kickstart drops them. Install the package by name on both nodes.
Error: “Module br_netfilter not found in directory /lib/modules”
To show what skipping the package costs, we removed it from the server and installed K3s anyway. The service started, the node went Ready, and the journal filled with module and iptables failures. Four minutes in, CoreDNS was 0/1, local-path-provisioner was in Error and metrics-server was crash-looping on failed readiness probes:
Watch for Extension conntrack revision 0 not supported, missing kernel module? followed by Failed to bootstrap IPTables. If your Ready node logs those lines, it is not a working node. Install kernel-modules-extra, make sure it matches the running kernel, and restart K3s with sudo systemctl restart k3s. After that the journal on our server read Module br_netfilter was already loaded, the iptables errors stopped, and all three system pods went healthy without being touched.
One warning stays after the fix: Failed to load kernel module nft-expr-counter with modprobe. modinfo finds no nft_counter module on this kernel and the rules applied cleanly with the warning present, so you can ignore it.
Open firewalld for K3s
K3s’s own docs recommend turning firewalld off. If your Rocky server faces anything other than a lab switch, keep it on and add the rules instead. The default public zone allows only ssh, cockpit and dhcpv6-client. K3s ships a checker, k3s check-config, and when we ran it on the server before adding any rules, its Firewall block listed every port K3s might use and marked each one blocked. Once K3s is installed you can run the same check yourself with sudo /usr/local/bin/k3s check-config.
Here is that report from the lab server. You need 6443, 10250 and 8472 for this cluster. 5001 is only for the embedded Spegel registry, 2379 and 2380 only for embedded etcd HA, and 51820/51821 only for the WireGuard flannel backend.
On the server, open the API port, the kubelet port, VXLAN and web traffic for Traefik. The last two rules put the pod CIDR (10.42.0.0/16) and the service CIDR (10.43.0.0/16) in the trusted zone, because pod traffic arrives on the host with those source addresses and firewalld would otherwise reject it:
sudo firewall-cmd --permanent --add-port=6443/tcp
sudo firewall-cmd --permanent --add-port=10250/tcp
sudo firewall-cmd --permanent --add-port=8472/udp
sudo firewall-cmd --permanent --add-service=http --add-service=https
sudo firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16
sudo firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16
sudo firewall-cmd --reloadOn the agent you open the same set minus 6443, since nothing connects to an agent’s API port.
sudo firewall-cmd --permanent --add-port=10250/tcp
sudo firewall-cmd --permanent --add-port=8472/udp
sudo firewall-cmd --permanent --add-service=http --add-service=https
sudo firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16
sudo firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16
sudo firewall-cmd --reloadCheck both zones with sudo firewall-cmd --list-all and sudo firewall-cmd --zone=trusted --list-all. On the server you should see the three ports and two services in public, and both CIDRs as sources in trusted:
VXLAN on 8472 carries unencrypted pod traffic, so it should only be reachable from the other nodes. On a flat lab subnet the public-zone rule is fine; on anything shared, restrict it with a rich rule or a dedicated zone bound to the cluster network. If you change cluster-cidr or service-cidr, change the trusted sources to match.
Install the K3s server with SELinux enforcing
Leave SELinux in enforcing mode. The K3s installer detects it on Rocky, adds the Rancher rancher-k3s-common-stable repository and installs k3s-selinux plus container-selinux from AppStream before it starts the service. What it does not do is tell containerd to label containers. That needs selinux: true in the K3s config, so write the config file before running the installer:
sudo mkdir -p /etc/rancher/k3s
sudo vim /etc/rancher/k3s/config.yamlAdd the following. tls-san adds extra names to the API server certificate (the node’s own IP is already in it, so the hostname is the entry that matters here, and you’d add a DNS name or VIP the same way), and node-ip pins the address flannel and the kubelet use on hosts with more than one interface:
write-kubeconfig-mode: "0600"
selinux: true
tls-san:
- rocky-k3s01
- 10.0.1.50
node-ip: 10.0.1.50Now run the installer from the stable channel. Setting INSTALL_K3S_CHANNEL=stable explicitly means you get the release K3s has marked stable, not whatever latest points at that day:
curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=stable sh -You’ll see the stable channel resolve to the current release, the Rancher repository GPG key get imported, and dnf install the two SELinux packages before the binary and systemd unit go in.
Notice the package is k3s-selinux-1.6-1.el9, built for EL9, while container-selinux comes from Rocky 10’s own AppStream. The repo URL the installer writes ends in centos/9/noarch. It loaded and enforced fine on Rocky 10.2, and the policy module showed up in semodule -l next to container, but expect that version string if you audit installed packages.
Confirm pods run confined
You prove SELinux is doing work by reading the label on a running container. Deploy anything, for example sudo /usr/local/bin/k3s kubectl create deployment selinux-test --image=nginx:alpine, then check the policy, the file contexts and the process labels. Run sudo /usr/local/bin/k3s kubectl delete deploy selinux-test when you’re done looking.
getenforce
sudo semodule -l | grep -E "k3s|container"
ls -Zd /var/lib/rancher/k3s /usr/local/bin/k3s
sudo grep -ri selinux /var/lib/rancher/k3s/agent/etc/containerd/config.toml
ps -eZ | grep -E 'k3s-server|containerd '
ps -eZ | grep nginxWith selinux: true, containerd’s generated config carries enable_selinux = true, the K3s binary runs as container_runtime_t, and every nginx process runs as container_t with its own MCS category pair:
We then removed selinux: true from the config, restarted K3s and started another nginx deployment. containerd’s config flipped to enable_selinux = false, and the new pod came up as spc_t, the super-privileged container type that SELinux does not confine:
system_u:system_r:spc_t:s0 39086 ? 00:00:00 nginx
system_u:system_r:spc_t:s0 39144 ? 00:00:00 nginx
system_u:system_r:spc_t:s0 39145 ? 00:00:00 nginxSo getenforce saying Enforcing and k3s-selinux being installed are not enough on their own. Without the flag your workloads run unconfined on an enforcing host, which is the opposite of what anyone who left SELinux on wanted. Put selinux: true in the config on every server and agent.
Set up kubectl for your user
Rocky’s sudoers sets secure_path = /sbin:/bin:/usr/sbin:/usr/bin, which leaves out /usr/local/bin where K3s puts its binary and the kubectl symlink. That is why sudo k3s kubectl get nodes fails with sudo: k3s: command not found on Rocky while it works on Ubuntu. Use the full path under sudo, or better, give your own user a kubeconfig and stop needing sudo for kubectl at all:
mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown $USER:$USER ~/.kube/config
chmod 600 ~/.kube/config
echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc
source ~/.bashrcDon’t skip the KUBECONFIG line. K3s’s bundled kubectl defaults to /etc/rancher/k3s/k3s.yaml rather than ~/.kube/config, so with only the copy in place our first kubectl get nodes failed with error loading config file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied. Keeping the system file at 0600 and exporting the variable is safer than relaxing the mode to 0644, which hands cluster-admin to every local user. Check the client and server versions:
kubectl versionBoth sides report the K3s build.
Client Version: v1.36.5+k3s1
Kustomize Version: v5.8.1
Server Version: v1.36.5+k3s1To manage the cluster from a workstation, copy the same file there and change server: https://127.0.0.1:6443 to https://10.0.1.50:6443. The node IP is already in the certificate; if you connect by hostname or a load balancer address instead, that name has to be in tls-san.
Join a worker node to the cluster
Agents authenticate with the server’s node token. Read it on the server and keep it out of shell history and screenshots, since anyone holding it can join a node to your cluster:
sudo cat /var/lib/rancher/k3s/server/node-tokenOn the agent, with kernel-modules-extra installed and the firewall rules from earlier in place, create the agent’s config first so it also starts with SELinux labelling:
sudo mkdir -p /etc/rancher/k3s
sudo vim /etc/rancher/k3s/config.yamlTwo lines are enough for an agent.
selinux: true
node-ip: 10.0.1.51Then run the installer with the server URL and the token. read -rs prompts for the token without echoing it or writing it to your shell history, and when K3S_URL is set the script installs an agent instead of a server and names the unit k3s-agent:
read -rs -p "Node token: " K3S_TOKEN; echo
curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=stable K3S_URL=https://10.0.1.50:6443 K3S_TOKEN="${K3S_TOKEN}" sh -Your agent pulls k3s-selinux the same way the server did. Back on the server, both nodes should be Ready within a few seconds of the installer returning.
kubectl get nodes -o wide
kubectl get pods -A -o wideThe agent had been Ready for 35 seconds in this capture, runs the same containerd build as the server, and already hosts a Traefik service load balancer pod (svclb-traefik), which is what lets ingress answer on its IP too. The restart counts on metrics-server and local-path-provisioner are left over from the missing-modules test, not from the join:
In the ROLES column the agent reads <none> and the server reads only control-plane. If you need a worker label for node selectors, add one with kubectl label node rocky-k3s02 node-role.kubernetes.io/worker=true.
Error: “failed to get CA certs” when the agent joins
Our first join attempt ran before the server’s firewall rules existed. The installer sat at systemd: Starting k3s-agent and never returned, because the unit waits until the agent reaches the server. The agent’s journal repeated one error every two seconds:
You’ll notice the message names 127.0.0.1:6444, the agent’s local load balancer, which is why it hides the real cause. A direct curl -sk https://10.0.1.50:6443/ping from the agent returned exit code 7, which means curl could not connect at all. Opening the ports on the server and running sudo systemctl restart k3s-agent on the agent was enough. There is no need to rerun the installer, and the node turned Ready within a minute.
Deploy a test app behind Traefik
A deployment spread over both nodes tests everything you just set up at once: scheduling on the agent, the VXLAN overlay, cluster DNS, and Traefik answering on port 80 through firewalld. Create the manifest:
vim ~/web-demo.yamlPaste this. The topology spread constraint asks the scheduler to put one replica on each node, and the Ingress routes web.example.com through the bundled traefik ingress class:
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-demo
spec:
replicas: 2
selector:
matchLabels:
app: web-demo
template:
metadata:
labels:
app: web-demo
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: ScheduleAnyway
labelSelector:
matchLabels:
app: web-demo
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: web-demo
spec:
selector:
app: web-demo
ports:
- port: 80
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-demo
spec:
ingressClassName: traefik
rules:
- host: web.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-demo
port:
number: 80Apply it, then hit the ingress on both node IPs with the right Host header and run a throwaway busybox pod that resolves the service name and fetches the page through it:
kubectl apply -f ~/web-demo.yaml
kubectl get pods -o wide -l app=web-demo
kubectl get ingress
curl -s -H 'Host: web.example.com' http://10.0.1.50/ | grep title
curl -s -H 'Host: web.example.com' http://10.0.1.51/ | grep title
kubectl run nettest --rm -i --restart=Never --image=busybox:1.36 -- sh -c 'nslookup web-demo.default.svc.cluster.local; wget -qO- http://web-demo | grep title'One replica landed on each node, the Ingress lists both node addresses, and all three paths returned the nginx welcome title. The busybox pod resolved web-demo through CoreDNS at 10.43.0.10, which proves pod-to-service traffic crosses firewalld correctly:
Point a real DNS name at either node and Traefik serves it the same way. To expose the Traefik dashboard as well, follow our guide on enabling the Traefik dashboard on K3s.
Error: “nslookup: read: No route to host” from a pod
A missing trusted-zone rule shows up as this error, and you can easily misread it as a CoreDNS bug because CoreDNS itself stays Running. To reproduce it, we removed the two CIDRs from the trusted zone on the server at runtime and ran the same test pod:
DNS lookups failed with No route to host from pods on both nodes, including one pinned to the agent, and wget could not resolve the service. Meanwhile kubectl top nodes kept working, because metrics-server reaches kubelets on 10250, which was still open. A firewall-cmd --reload restored the permanent rules and the next run passed. If pods on your cluster can’t resolve anything, run sudo firewall-cmd --zone=trusted --list-sources before you touch CoreDNS.
Check what changed since the K3s 1.30 builds
You run the same install command older guides show, and no install flag was added or removed in the 1.36 line, though 1.36.3 started accepting wildcards such as --tls-san=*.example.com. What changed is what you get after the install. The first rows below were read off the lab server; the rest come from the K3s v1.36 release notes:
| Item | What v1.36.5+k3s1 does | Source |
|---|---|---|
| containerd | v2.3.4-k3s1.36 | Lab |
| Traefik | image 3.7.13, chart traefik-40.1.5+up40.1.0 | Lab |
| CoreDNS / metrics-server / local-path | 1.14.7 / v0.9.0 / v0.0.37 | Lab |
| Datastore on a single server | SQLite through kine (state.db), no etcd | Lab |
| Server node role | control-plane only, no master label | Lab |
| Supervisor metrics and pprof | 403 for anonymous requests | Lab |
| Traefik chart v40 (from 1.36.2, or 1.35.6 on the 1.35 line) | ingress-nginx provider key renamed to kubernetesIngressNGINX | Release notes |
| Flannel | --flannel-cni-conf now reaches the embedded flannel (fix #14598) | Release notes |
The supervisor change is the one most likely to break monitoring. From the v1.34.12, v1.35.9, v1.36.5 and v1.37.1 patch releases, K3s rejects anonymous requests to the supervisor’s metrics, pprof and embedded registry routes on port 6443. Both endpoints are off by default, so we added supervisor-metrics: true and enable-pprof: true to the server config, restarted K3s, and called them once anonymously and once with the admin client certificate:
Both anonymous calls got 403, and the certificate got the k3s_certificate_expiration_seconds series. A Prometheus job that scraped the supervisor without credentials on an older build needs a client certificate or bearer token after the upgrade. We removed both settings again afterwards, since neither belongs on a cluster you are not actively profiling.
You only feel the Traefik rename if you carry a HelmChartConfig that enables the ingress-nginx compatibility provider. The old kubernetesIngressNginx spelling is ignored without an error, and our K3s on Debian guide shows the test. The latest channel pointed at v1.37.1+k3s1 on the same day, and the 1.37 line adds --write-kubeconfig-name and puts --advertise-address in the API server certificate. Nothing in this guide needs either, so stay on stable. Secrets encryption can also be switched on for a running cluster now; the steps are in our K3s on Ubuntu 24.04 guide and work the same on Rocky.
Uninstall K3s and clean up the Rocky nodes
Take an agent out of the cluster from the server side first, so its pods reschedule cleanly before the node disappears.
kubectl drain rocky-k3s02 --ignore-daemonsets --delete-emptydir-data
kubectl delete node rocky-k3s02Then run the uninstall script on the agent itself. On a server the script is /usr/local/bin/k3s-uninstall.sh instead.
sudo /usr/local/bin/k3s-agent-uninstall.shOn Rocky the script does more than stop the service. It runs dnf to remove the SELinux packages the installer added, then deletes the Rancher repo file:
Removed:
container-selinux-4:2.246.0-1.el10.noarch k3s-selinux-1.6-1.el9.noarch
Complete!
+ '[' -d /etc/zypp/repos.d ']'
+ '[' -d /etc/yum.repos.d ']'
+ rm -f /etc/yum.repos.d/rancher-k3s-common.repodnf only drops container-selinux because nothing else on the node needed it; if Podman were installed, it would stay. The script leaves your firewalld rules and kernel-modules-extra alone; drop the K3s ports and trusted sources yourself if the node is going back to other work. On a fresh Rocky 10 box, the order that works is the one above: modules first, firewall second, selinux: true in the config before the installer runs, and nothing reported as healthy until a pod on the agent has resolved a service name.
