bundle/dstate: nullify local state when DMS recording enabled

[shreyas-goenka]

Summary

Fixes a bug where enabling DMS recording (experimental.record_deployment_history) on a deployment fails when the local state cache still has resources, even if the remote deployment was destroyed. The fix nullifies local state when recording is enabled, treating the state file as a tombstone that only carries the feature marker and header.

The Problem

When a DMS deployment is destroyed on the service, the local resources.json is a cache that can retain stale resources. Attempting to redeploy with DMS enabled would fail with:

Error: this deployment already exists and is not recorded with the deployment history
feature enabled, so it cannot be recorded without redeploying its resources

This error was caused by a guard that checked recording && !recorded && len(db.Data.State) > 0. While the guard was originally designed to prevent enabling DMS on existing non-DMS deployments (creating resources a second time), it also blocked the legitimate case of a destroyed DMS deployment being rebuilt.

The Solution

When opening a deployment state with recording enabled, nullify the local state. The service becomes the source of truth:

• If dmsDeploymentID is non-empty: ListResources fetches resources from the service
• If dmsDeploymentID is empty: no resources are loaded (fresh or destroyed deployment)

This allows the bootstrap path to be taken (recording && !recorded && len(db.Data.State) == 0) instead of erroring.

Tradeoff

Nullifying local state removes the guard against enabling DMS on an existing non-DMS deployment (no service node, but resources cached locally). This could:

• Allow migrating from non-DMS to DMS without explicit destroy
• Potentially orphan old resources if they have different IDs or names

However, this is an acceptable tradeoff to unblock the more common and important case: recovering destroyed DMS deployments. Users who need to migrate from non-DMS to DMS should explicitly destroy the bundle first.

Changes

• bundle/direct/dstate/state.go: Nullify local state when recording is true
• bundle/direct/dstate/state_test.go: Add unit test verifying preconditions
• acceptance/bundle/dms/existing-state/output.txt: Updated golden (reflecting changed guard behavior)

Testing

• Unit tests pass: go test ./bundle/direct/dstate/...
• Existing-state DMS acceptance test now reflects the new behavior (no error when enabling DMS on non-DMS deployment)

This pull request and its description were written by Isaac.

[eng-dev-ecosystem-bot]

Integration test report

Commit: 0336c85

Run: 34074400399

	Env	❌​FAIL	🟨​KNOWN	🔄​flaky	🙈​SKIP	✅​pass	🙈​skip	Time
❌​	aws linux	2	1	2	1	302	1224	7:26
❌​	aws windows	2	1		1	306	1222	7:49
❌​	azure linux	27	1		1	278	1224	5:35
❌​	azure windows	27	1		1	280	1222	5:31
❌​	gcp linux	27	1		1	279	1224	5:51
❌​	gcp windows	27	1		1	281	1222	5:38

29 interesting tests: 27 FAIL, 1 KNOWN, 1 SKIP

	Test Name	aws linux	aws windows	azure linux	azure windows	gcp linux	gcp windows
🟨​	TestAccept	🟨​K	🟨​K	🟨​K	🟨​K	🟨​K	🟨​K
❌​	TestAccept/bundle/destroy/jobs-and-pipeline	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/declined-deploy	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/declined-deploy/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/depends-on	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/depends-on/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/deployment-metadata-change	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/deployment-metadata-change/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/existing-state	❌​F	❌​F	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/existing-state/DATABRICKS_BUNDLE_ENGINE=direct/DMS=	❌​F	❌​F	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/failed-recreate	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/failed-recreate/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/multiple-resources	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/multiple-resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/no-drift	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/no-drift/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/no-resources	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/no-resources/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/provenance	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/provenance/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/record-failure	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/record-failure/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/stale-plan	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/stale-plan/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/READPLAN=	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/stale-plan/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true/READPLAN=1	✅​p	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/successful-recreate	🔄​f	✅​p	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/dms/successful-recreate/DATABRICKS_BUNDLE_ENGINE=direct/DMS=true	🔄​f	✅​p	❌​F	❌​F	❌​F	❌​F
🙈​	TestAccept/ssh/connection	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S	🙈​S