Ansible Idempotency in Practice: What -vvv Actually Shows You

I’ve written before about Ansible as part of a broader IaC toolchain without ever digging into this specific property on its own — worth fixing here. “Idempotent” is one of those words that gets used as a badge of honor for Ansible without much scrutiny of whether a given playbook actually earns it. It doesn’t come for free just because you wrote YAML instead of a bash script. I wanted to see the failure mode directly rather than take the usual advice on faith, so I wrote the smallest playbook that could demonstrate it, ran it three times in a row, and looked at what changed on disk each time.

The Setup: Two Ways to Do the Same Thing

Same goal — make sure a config line exists in a file — written two different ways in the same play:

- name: append a line with shell (looks fine, isn't idempotent)
  ansible.builtin.shell: echo "max_connections=200" >> /tmp/config.conf

- name: same intent, done with lineinfile (actually idempotent)
  ansible.builtin.lineinfile:
    path: /tmp/config.conf
    line: "max_connections=200"
    create: true

Both tasks look reasonable if you’re skimming a diff in a pull request. Only one of them is actually safe to run more than once.

Run It Three Times, Watch the File

$ ansible-playbook playbook.yml   # run 1
TASK [append a line with shell...] ***
changed: [localhost]
TASK [same intent, done with lineinfile...] ***
ok: [localhost]

$ ansible-playbook playbook.yml   # run 2 — nothing else touched the file in between
TASK [append a line with shell...] ***
changed: [localhost]
TASK [same intent, done with lineinfile...] ***
ok: [localhost]

$ cat /tmp/config.conf
max_connections=200
max_connections=200
max_connections=200

Three runs, three identical lines appended to the file, and the shell task cheerfully reported changed every single time — which is technically true (the file did change, it grew a line) and completely misses the point. The lineinfile task reported changed once, then ok on every subsequent run, because it actually checked whether the line already existed before deciding to act. That’s the entire difference between a playbook you can run on a schedule without thinking about it and one that quietly corrupts its own target a little more each time it fires.

Why shell and command Can Never Be Idempotent on Their Own

This isn’t a bug in the shell module — it’s doing exactly what it’s told, which is the actual problem. shell and command are dumb pass-through: Ansible hands your string to a shell and reports whatever exit code comes back, with no concept of “did this already happen.” Every purpose-built module — lineinfile, copy, package, service, user — carries its own check logic written in Python: read the current state, compare it to the desired state, only act if they differ, report the comparison result as changed or not. That check is the entire value proposition of using a module instead of a raw command, and it disappears completely the moment you reach for shell as a shortcut.

What -vvv Actually Shows You

Running the same playbook with -vvv makes the check-then-act pattern visible instead of assumed:

$ ansible-playbook playbook.yml -vvv
TASK [same intent, done with lineinfile...] ***
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p ... )'
Using module file /usr/lib/python3/dist-packages/ansible/modules/lineinfile.py
<127.0.0.1> PUT ... TO .../AnsiballZ_lineinfile.py
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3 .../AnsiballZ_lineinfile.py'
ok: [localhost] => {
    "changed": false,
    "diff": [...]
}

What’s actually happening on every task, module-based or not, even against localhost: Ansible packages the module into a self-contained Python script (the AnsiballZ_*.py file), copies it to a temp directory on the target, executes it there, and deletes it afterward. For lineinfile, that script reads the file, compares its content against the desired line, and only writes if something’s actually different — which is why the JSON result carries a diff key at all. shell has no equivalent step; there’s nothing to diff, because nothing in the module ever looked at the file before running your command.

Making shell/command Behave, When You Genuinely Need Them

Sometimes there’s no module for what you’re doing and a raw command is the honest answer — the fix isn’t to avoid shell entirely, it’s to give Ansible the check it can’t infer on its own:

# only runs if the marker file doesn't already exist
- name: run a one-time migration script
  ansible.builtin.shell: /opt/app/migrate.sh
  args:
    creates: /opt/app/.migrated

# tell Ansible how to interpret the command's own output
- name: check for pending updates
  ansible.builtin.command: apt list --upgradable
  register: result
  changed_when: "'upgradable' in result.stdout"

creates (and its counterpart removes) turns a stateless command into a stateful one by pointing at a side effect Ansible can check without understanding what the command actually did. changed_when is the more general tool: it lets you tell Ansible how to read the command’s exit code or output and decide for itself whether anything actually changed, instead of defaulting to “ran without error, so: changed.”

Why This Isn’t Just a Style Preference

The three-run test above matters more once a playbook stops being something you run once by hand and starts being something re-applied on a schedule, or by a CI job on every merge to keep infrastructure in sync with a repo — which is the actual, common way Ansible gets used past the first setup. A non-idempotent task that’s harmless run once becomes a slow leak the moment it’s re-applied automatically, weekly or on every deploy, with nobody watching the output line by line each time. The duplicate line in a config file from this post’s example is cosmetic; the same pattern applied to a task that appends a firewall rule, or grows a cron entry, compounds silently into a real problem precisely because the exit code never told anyone anything was wrong.

The Checklist

• Reach for a purpose-built module before shell/command — the idempotency check is the actual feature you’re paying for.
• If a raw command is genuinely necessary, add creates/removes or changed_when — don’t let it silently report changed on every run forever.
• Run a playbook twice in a row before trusting it, the same way you’d run a migration twice before trusting it in production. If the second run isn’t a no-op, something in it isn’t actually idempotent yet.
• -vvv is the fastest way to see whether a task actually checked state or just executed blindly — look for a diff key in the result, not just the changed flag.