Running k3s in Proxmox LXC Containers: The Pitfalls Nobody Documents

I’m going to break the usual format of a “how to run k3s in Proxmox LXC” post: I didn’t get it running. I gave it a bounded session on my real Proxmox host, hit a wall that turns out to be a known-but-poorly-documented one, and decided that writing down exactly what happened is more useful than padding out a guide for a setup I haven’t actually gotten to work. This is that writeup — what’s proven, what I ruled out, and what I’d try next.

The Setup

An unprivileged LXC, freshly created, with the two features every guide agrees you need for anything Docker- or Kubernetes-adjacent in a container:

pct create 996 local:vztmpl/debian-13-standard_13.1-2_amd64.tar.zst 
  --hostname k3s-test --cores 2 --memory 1024 --swap 512 
  --net0 name=eth0,bridge=vmbr0,ip=dhcp 
  --rootfs local-lvm:8 
  --unprivileged 1 --features nesting=1,keyctl=1

Debian 13, 2 cores, 1 GB RAM — deliberately small, since the point was to see what breaks, not to run a real workload. Before touching k3s at all, I checked the container itself was healthy: systemctl is-system-running came back running, zero failed units. Nesting was doing its job; this wasn’t a repeat of the “mount points fail without nesting” problem I ran into provisioning containers with Terraform the same afternoon.

The Install Looked Fine

curl -sfL https://get.k3s.io | sh -

Clean install, no errors: k3s v1.36.3+k3s1, binary in place, systemd unit created and enabled, service started. Nothing here suggested trouble.

Where It Actually Breaks

kubectl get nodes alternated between “server is currently unable to handle the request” and “connection refused” — the API server never settled into a state where it would answer. systemctl status k3s told the real story: the service was crash-looping, cleanly, every ~11-12 seconds:

k3s.service: Deactivated successfully.
k3s.service: Consumed 4.5s CPU time, 247M memory peak.
k3s.service: Scheduled restart job, restart counter is at 12.

“Deactivated successfully” is the detail worth sitting with. That’s systemd’s phrasing for a clean exit, not a crash — no segfault, no OOM-kill, no panic trace anywhere in the journal. Something inside k3s itself was choosing to give up after a few seconds of work, consistently, run after run.

What the Logs Actually Say

Two things stood out on closer reading. First, on every restart:

modprobe: FATAL: Module br_netfilter not found in directory /lib/modules/7.0.14-11-pve
modprobe: FATAL: Module overlay not found in directory /lib/modules/7.0.14-11-pve
level=warning msg="Failed to load kernel module br_netfilter with modprobe"

My first read was “there’s the cause” — except checking my actual Proxmox host’s kernel (lsmod, and /proc/sys/net/bridge/bridge-nf-call-iptables, which only exists if the functionality is active) suggests the capability is already present at the host level, likely built directly into this kernel rather than as a loadable .ko file — which is exactly why modprobe from inside an unprivileged container can’t find a module to load: there’s no module file, because it was never built as one. I’m flagging this as a plausible red herring rather than confirmed, because I didn’t chase it further — but it’s a good example of a log line that looks damning and might not be.

The more convincing lead was this one, appearing right alongside kube-proxy startup on every single restart:

level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072"
level=error msg="Failed to set sysctl: open /proc/sys/net/netfilter/nf_conntrack_max: permission denied"

k3s’s bundled kube-proxy wants to write a kernel sysctl on startup, and an unprivileged Proxmox LXC — nesting and keyctl notwithstanding — doesn’t grant write access to that specific /proc/sys path by default. nesting=1 unlocks the mount and cgroup namespacing that Docker and containerd need; it does not unlock arbitrary sysctl writes, which is a separate restriction enforced by the container’s AppArmor profile. That’s the actual gap between “runs Docker fine” and “runs a CNI-based orchestrator fine” inside an LXC, and it’s not obvious until you’re staring at a crash loop with no panic trace.

What I Didn’t Do, and Why

The community-documented fix for this class of problem is a custom AppArmor profile for the container that allows the specific sysctl writes k3s needs (sometimes paired with lxc.apparmor.profile: unconfined, which is a much bigger hammer than the problem calls for). I stopped short of building and applying that for two reasons that are specific to this being a real production Proxmox host and not a disposable lab box: I’d want to scope the AppArmor exception narrowly rather than reach for unconfined, and that’s real engineering, not a five-minute follow-up — worth doing properly in its own session, not squeezed into the same afternoon as everything else I was testing.

Cleanup

pct stop 996 && pct destroy 996 --purge 1

Gone. No lingering LVM volume, no orphaned config.

What This Is Actually Worth

Every k3s-in-Proxmox-LXC guide I’d skimmed before doing this made it look like curl | sh and done, with nesting=1 as the one gotcha to remember. My real result: nesting solved the problem it’s known for and a different, less-discussed one showed up right behind it — a sysctl permission wall that produces a clean, silent crash loop instead of a helpful error. If you’re planning to actually run this, the two things I’d check before you start: whether your k3s workload needs kube-proxy’s sysctl tuning at all (some CNI configurations don’t), and whether it’s worth the AppArmor customization versus just running the k3s node as a privileged container or a small VM instead — trading the container’s efficiency for not fighting its sandbox. I’ll follow up once I’ve actually built and tested that AppArmor profile properly.