fix(nvsnap): sync chart image tags and fail on drift

[balajinvda]

Fixes #731.

Why

The nvsnap Helm chart pinned agent.image.tag: v0.1.3 while scripts/versions.sh was at NVSNAP_APP_VERSION=v0.2.32. install-nvsnap.sh is the documented install path, so a fresh install deployed a 31-version-old agent. Caught while validating the chart on nvcf-dgxc-k8s-aws-usw2-dev2 -- test-e2e.sh refused to run:

[ERROR] Deployed agent (.../nvsnap-agent:v0.1.3) != expected (.../nvsnap-agent:v0.2.32)

sync-versions.sh already targeted the chart (DIRS=(deploy/k8s deploy)). The substitution matched a single registry/name:tag token, which is how deploy/k8s spells an image reference. Chart values split the same reference across repository: and tag: lines, so the pattern matched nothing and the sed no-opped -- while still printing Synced nvsnap-agent -> ...:v0.2.32.

The staleness check below used the same /name: anchor, so it also matched nothing in the chart and passed vacuously. Nothing in the pipeline could catch the drift. A verification step that passes without finding anything is the part that let this sit.

What changed

• chart_tag() / set_chart_tag() read and rewrite the split repository:/tag: form, preserving indentation. Run alongside the existing sed so both spellings stay in sync.
• Verification now compares every chart tag against its expected version, so drift fails loudly instead of passing silently.
• Verification also checks imageRegistry, since the chart composes refs as <imageRegistry>/<repository>:<tag> and a drifting registry breaks every image at once. sync-versions.sh never rewrites that field, so it is a genuinely reachable failure.
• values.yaml agent tag v0.1.3 -> v0.2.32 (the actual drift).

Deliberately not asserting that every image in IMAGES appears somewhere: nvsnap-init and pyzmq-builder currently appear in zero manifests, so that assertion would fail falsely.

Testing

On nvcf-dgxc-k8s-aws-usw2-dev2:

• Sync updates only the stale agent tag; no other chart line moves.
• Mutation test, chart-sync step disabled + tag reverted: fails with pins nvsnap-agent tag v0.1.3, expected v0.2.32.
• Mutation test, imageRegistry corrupted: fails with imageRegistry is stg.nvcr.io/zq9tgrjzrfpo, expected nvcr.io/0651155215864979/ncp-dev.
• Clean tree exits 0.
• ./scripts/test-e2e.sh vllm-small PASSES on the resulting install: 33G checkpoint, 8m51s total, post-restore inference OK. This is also the rule-10 gate that fix(nvsnap): migrate remaining workloads to criu-v2 and repair the helm chart #472 merged without.

Customer Release Notes

Fixed the nvsnap Helm chart installing an outdated agent image.

Plan Summary

Not applicable.

Usage

./scripts/sync-versions.sh -- now also rewrites chart values and exits non-zero on drift.

Notes

Workaround before this lands: ./scripts/install-nvsnap.sh --set agent.image.tag=v0.2.32.

nvsnap-l2-wait appears in the chart but has no version variable in versions.sh, so it is left alone. Worth deciding whether it should be managed.

References

#731

Related Merge Requests/Pull Requests

None

Dependencies

None