GitOps with ArgoCD: How the Reconciliation Loop Actually Decides to Sync

ArgoCD showed up as one piece of the best-of-breed stack I wrote about replacing an all-in-one platform with, without ever explaining what it’s actually doing under the hood — worth fixing here. “GitOps” gets explained as “deploy by pushing to git instead of running a deploy script,” which is true but skips the part that actually makes it work differently from a CI pipeline that happens to be triggered by a commit. The interesting mechanism is the reconciliation loop — the thing running continuously inside the cluster, comparing two states and deciding what to do about the difference — and understanding it changes how you debug an ArgoCD app that won’t sync.

Push vs. Pull, and Why the Direction Actually Matters

A traditional CI/CD pipeline pushes: a GitHub Actions job builds an image, then runs kubectl apply or helm upgrade against your cluster directly, using credentials the pipeline holds. ArgoCD inverts this — nothing outside the cluster ever pushes changes into it. A controller running inside the cluster pulls the desired state from git and reconciles it against what’s actually running:

┌─────────┐   git pull    ┌──────────────────┐   compares to   ┌─────────────┐
│  Git    │◄──────────────│  ArgoCD           │◄────────────────│  Live       │
│  repo   │  (desired)     │  controller       │   (actual)      │  cluster    │
└─────────┘                └──────────────────┘                 └─────────────┘

The practical consequence: nothing outside the cluster needs deploy credentials to it at all — not a CI runner, not a developer’s laptop. A compromised CI pipeline can’t push a malicious deploy, because CI was never capable of deploying in the first place; it can only change what’s in git, and ArgoCD’s own controller — running with permissions scoped to what it’s meant to manage — is the only thing that ever actually talks to the Kubernetes API to apply changes.

The Loop Itself: Compare, Diff, Decide

Every few minutes (three by default) and on every webhook-triggered git push, ArgoCD’s controller does the same three-step cycle for every application it manages:

1. Fetch the manifests from the configured git path — rendering Helm charts or Kustomize overlays if that’s what the repo uses, producing a final, concrete set of Kubernetes objects.
2. Diff that rendered output against the live objects actually running in the cluster, field by field.
3. Report status: Synced if they match, OutOfSync if they don’t — and if the application’s sync policy is set to automated, act on that diff without waiting for a human to click anything.

This is the entire meaning of “desired state” in GitOps: not a philosophy, a literal comparison happening on a timer. An application sitting in OutOfSync for longer than expected almost always means one specific thing — the diff step found a real difference, and either automated sync is off (deliberately, waiting for manual approval) or something is actively fighting the sync and re-creating drift as fast as ArgoCD corrects it.

Where This Breaks in Practice: Something Else Is Also Writing to the Cluster

The reconciliation loop’s blind spot is anything that mutates cluster objects outside of what ArgoCD itself manages — most commonly a Horizontal Pod Autoscaler changing a Deployment’s replica count, or a mutating admission webhook injecting a sidecar container. ArgoCD sees the live replica count diverge from what’s in git, flags the app OutOfSync, and — if auto-sync is on — reverts it back to the git-specified value, immediately undoing whatever the autoscaler just did. The fix isn’t disabling autosync everywhere; it’s telling ArgoCD explicitly which fields it doesn’t own for a given resource:

metadata:
  annotations:
    argocd.argoproj.io/compare-options: IgnoreExtraneous
spec:
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jsonPointers:
        - /spec/replicas

ignoreDifferences tells the diff step to stop comparing that specific field at all, which is the correct fix precisely because it names the exact field something else legitimately owns — the same discipline as Terraform’s ignore_changes, applied to the same underlying problem of two systems both believing they own one value.

Sync Waves: Ordering Without a Pipeline

A single ArgoCD application often needs its resources applied in a specific order — a database migration Job before the Deployment that expects the new schema, a Secret before the pod that mounts it. There’s no imperative pipeline script to sequence this in; ordering is declared per-resource with an annotation:

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-1"   # applied before wave "0" (the default)

Lower numbers sync first; ArgoCD waits for each wave to reach a healthy state (not just “applied” — actually healthy, per its own health checks) before starting the next one. This replaces what would otherwise be explicit pipeline steps with a property of the resource itself, which is consistent with GitOps’ actual premise: the git repo describes the complete desired state, ordering included, rather than a script describing a sequence of actions to take.

What GitOps Doesn’t Solve, and People Expect It To

Worth being direct about this: reconciliation handles configuration drift, not application correctness. ArgoCD will happily and continuously reconcile a cluster into a broken state if that’s what the git repo describes — a typo in an image tag, a bad config value, a manifest that passes YAML validation but breaks the app at runtime. A `Synced` and `Healthy` status means “the cluster matches git and the health checks defined for these resources pass” — not “the application behind them is doing the right thing.” That’s still a testing and review problem, upstream of anything the reconciliation loop is designed to catch.

The Checklist

• ArgoCD pulls; nothing outside the cluster needs deploy credentials to it — that’s the actual security property, not just a style preference.
• OutOfSync that won’t clear usually means something else is writing to the same fields — check for HPAs, mutating webhooks, or manual kubectl edit before assuming ArgoCD itself is broken.
• Use ignoreDifferences for fields something else legitimately owns, the same discipline as Terraform’s ignore_changes.
• Sync waves replace pipeline ordering for resources that must apply in sequence — declared on the resource, not scripted externally.
• Synced and Healthy means the cluster matches git, not that the application is correct — that’s still on your tests and your review, not the reconciliation loop.