Skip to content

KB-02 — Migrating Classic App-of-Apps → ApplicationSet (zero teardown)

When you need this: converting a manually-managed App-of-Apps (one root Application that hardcodes each child's metadata.name) into an ApplicationSet with templated, traceable names (modgpt.cluster-addons.<app>) — without deleting the live workloads the old apps manage.

This is a one-time operation, not steady-state. The steady-state architecture (ApplicationSet, AppProject, root app, the two authoring gotchas) lives in App of Apps.

Why it's risky (read first)

Renaming an Application (vault → modgpt.cluster-addons.vault) means ArgoCD deletes the old Application and creates a new one. Because ArgoCD apps carry the resources-finalizer.argocd.io finalizer, a plain delete cascade-deletes every resource the app manages — including StatefulSets like Vault. The migration must therefore:

  1. Strip the finalizer before deleting each old app, so the delete is an orphan (workloads stay).
  2. Let the new ApplicationSet-generated app adopt the orphaned resources (same destination + same Helm releaseName → ArgoCD re-labels them, doesn't recreate).
  3. Go one app at a time, harmless apps first (pdb/reloader), stateful ones (vault/vso) last.
The runbook 
# 0. Orphan the OLD root App-of-Apps so it stops recreating children (children + workloads stay)
kubectl patch application <old-root> -n argocd -p '{"metadata":{"finalizers":null}}' --type merge
kubectl delete application <old-root> -n argocd

# 1. Restructure the repo to apps/<name>/app.yaml + push; create the ApplicationSet + root app.
#    (see App of Apps page for the ApplicationSet manifest and app.yaml schema)
#    New modgpt.cluster-addons.<name> apps ADOPT existing resources — IF releaseName matches
#    the old app's release name (use {{ .path.basename }} in the ApplicationSet).

# 2. Per old app — strip the finalizer FIRST, then delete (orphan, no cascade):
for app in pdb reloader; do            # harmless first
  kubectl patch application $app -n argocd -p '{"metadata":{"finalizers":null}}' --type merge
  kubectl delete application $app -n argocd
done
# verify modgpt.cluster-addons.pdb / .reloader are Synced+Healthy and workloads never restarted

# 3. Repeat for the STATEFUL apps one at a time (vault, vso), verifying pods do NOT restart:
kubectl get pods -n vault -w   # in another shell — confirm vault-0/1/2 stay Running throughout

Golden rule: always strip the finalizer before deleting an old Application during a rename — that one step is what prevents the cascade delete of its workloads.

Adoption requires a matching Helm release name

The new app only adopts (rather than duplicates) the old resources if its Helm releaseName equals the old app's release name. With the ApplicationSet that's helm.releaseName: {{ .path.basename }} — which also keeps resource/container names dot-free. See App of Apps → gotcha #1.

Cleanup — leftover dotted-name resources

If the first sync ran before you added helm.releaseName (i.e. the release name was the dotted app name), Helm/ArgoCD will have created dotted-named resources — and the Deployment will have failed (containers[0].name … must not contain dots) while the RBAC/ServiceAccount objects (which allow dots) succeeded. These leftovers are namespaced and cluster-scoped:

# namespaced
kubectl get sa,role,rolebinding -n <ns> | grep 'modgpt.cluster-addons'
# cluster-scoped
kubectl get clusterrole,clusterrolebinding | grep 'modgpt.cluster-addons'
Delete them, then argocd app sync <app> --prune so the app converges on the correctly-named resources.

Aftermath — SharedResourceWarning

After orphan-deleting old apps, scan for stale shared-resource warnings:

kubectl get applications -n argocd -o json | \
  jq -r '.items[].status.conditions[]? | select(.type=="SharedResourceWarning") | .message'

A resource's argocd.argoproj.io/tracking-id annotation still names the old (deleted) app, while the new app also claims it. The usual culprit is Helm hook resources (e.g. upgrade-crds, a pre-upgrade Job + its SA/ClusterRole/Binding) that carry a hook-delete-policy which didn't fire during the orphan delete.

Don't delete hook resources piecemeal — especially not the ServiceAccount

Deleting just the ServiceAccount of an in-flight hook leaves a Job whose pod can't mount its token → MountVolume.SetUp failed for volume "kube-api-access-…": failed to fetch token: serviceaccounts "…" not found.

Correct fix: delete the whole hook Job (kubectl delete job <hook-job> -n <ns>) and run a full sync (argocd app sync <app>) so the PreSync hook recreates the SA + RBAC + Job as a set; the hook-delete-policy: hook-succeeded then removes them cleanly. The warning clears and nothing is left stuck.

Verification checklist
  • [ ] All modgpt.cluster-addons.* apps Synced + Healthy
  • [ ] Live workloads never restarted (Vault pods kept their original age/identity)
  • [ ] No SharedResourceWarning (jq scan above returns nothing)
  • [ ] No leftover dotted-name SAs/Roles/ClusterRoles/Bindings
  • [ ] Old Applications gone; new ones own the resources (argocd.argoproj.io/tracking-id points to the new app)