Provisioning Proxmox Containers with Terraform: Two Real Breaks Before It Worked

I don’t run Terraform against my Proxmox box day to day — my containers get created directly with pct, and that’s honestly the right tool for a homelab of ~30 LXCs I mostly manage by hand. But “would Terraform make sense here” is a fair question, so I set aside an afternoon, wrote the smallest config that actually provisions something, and ran it against my real host. It broke twice before it worked, and both breaks are worth knowing about before you try this yourself.

The Setup

The bpg/proxmox provider (not the older, now-unmaintained Telmate one) talks to the Proxmox API directly — no SSH, no agent. It needs an API token, which on my box meant:

pveum user token add root@pam terraform-test --privsep 0

--privsep 0 means the token inherits the user’s full permissions instead of needing its own ACL — fine for a throwaway test, not what I’d use for anything that sticks around (a scoped role with just VM.Allocate/VM.Config.*/Datastore.AllocateSpace is the real answer, and I deleted this token the moment I was done).

provider "proxmox" {
  endpoint  = "https://127.0.0.1:8006"
  api_token = "root@pam!terraform-test=<secret>"
  insecure  = true   # self-signed cert, internal network only
}

resource "proxmox_virtual_environment_container" "test" {
  node_name = "proxmox"
  vm_id     = 997

  operating_system {
    template_file_id = "local:vztmpl/debian-13-standard_13.1-2_amd64.tar.zst"
    type              = "debian"
  }

  initialization {
    hostname = "tf-test"
    ip_config {
      ipv4 { address = "dhcp" }
    }
  }

  network_interface {
    name   = "eth0"
    bridge = "vmbr0"
  }

  disk {
    datastore_id = "local-lvm"
    size         = 4
  }

  cpu    { cores = 1 }
  memory { dedicated = 256 }

  unprivileged = true
  started      = true
}

An LXC container, not a QEMU VM — the provider does both, but on a box that already runs everything as LXC, matching that made more sense than introducing a second guest type just for this test. The initialization block is the provider’s equivalent of cloud-init for containers: hostname and network config injected at first boot, no template surgery required.

Break One: “Apply Failed” for a Container That Actually Got Created

terraform apply ran, then errored:

Error: error waiting for container created: task "...:vzcreate:997:..." failed
to complete with exit code: WARNINGS: 1

Checking the actual Proxmox task log told a different story — the container had been created and started fine:

WARN: Systemd 257 detected. You may need to enable nesting.
TASK WARNINGS: 1

The provider treats any non-empty warning count from the Proxmox task API as a failure and refuses to save the resource into state — even though the underlying operation succeeded. That leaves an orphan: a real container Terraform doesn’t know about. Running apply again just fails harder:

Error: error creating container: received an HTTP 500 response
- Reason: CT 997 already exists on node 'proxmox'

The fix in the moment was manual — pct destroy 997 --purge 1, clear the local state file, apply clean. The fix that actually matters is upstream of that: don’t ignore the warning, act on it.

Break Two: the Warning Was Real

Checking the fresh container confirmed the warning wasn’t noise:

$ pct exec 997 -- systemctl is-system-running
degraded
$ pct exec 997 -- systemctl --failed
dev-mqueue.mount  failed  POSIX Message Queue File System
run-lock.mount    failed  Legacy Locks Directory /run/lock
tmp.mount         failed  Temporary Directory /tmp

Debian 13’s systemd (257) tries to mount these inside the container at boot, and an unprivileged LXC without nesting enabled blocks exactly that class of mount. This is the same constraint that trips people up running Docker or k3s in an LXC — it’s not specific to Terraform, but Terraform’s default container resource doesn’t turn it on for you. One block fixes it:

resource "proxmox_virtual_environment_container" "test" {
  # ...
  features {
    nesting = true
  }
}

Reapplied clean, and confirmed properly this time instead of trusting the exit code:

$ pct exec 997 -- systemctl is-system-running
running
$ pct exec 997 -- systemctl --failed
0 loaded units listed.

Destroy Was the Easy Part

$ terraform destroy -auto-approve
proxmox_virtual_environment_container.test: Destroying... [id=997]
proxmox_virtual_environment_container.test: Destruction complete after 4s

Four seconds, clean, no surprises — which is exactly the asymmetry that makes Terraform worth it for anything you’ll create and destroy more than once: the failure modes are all on the apply side, and once the config is right, destroy just works.

Would I Actually Use This?

For the throwaway container in this post — no, pct is three lines and doesn’t need a state file. Where I can see it paying for itself: anything I’d want to spin up and tear down repeatedly (a CI runner, a test environment for a specific change), or the day I have more than one Proxmox node and want one config describing all of them instead of remembering what I ran by hand on each. The two breaks above are exactly what I’d want to know before that day, instead of on it.

The Checklist

  • Use bpg/proxmox, not the archived Telmate provider.
  • Scope the API token’s permissions for anything that isn’t a five-minute test — --privsep 0 is a shortcut, not a default.
  • Don’t trust a Terraform error at face value against Proxmox — check the actual task log; “failed” sometimes means “succeeded with a warning the provider didn’t like.”
  • Add features { nesting = true } for any container that’ll run Docker, k3s, or anything else that wants its own cgroups — and verify with systemctl is-system-running, not just “it started.”

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *