[bundle] Read deployment state from Deployment Metadata Service (step 4)#5355
Draft
shreyas-goenka wants to merge 3 commits into
Draft
[bundle] Read deployment state from Deployment Metadata Service (step 4)#5355shreyas-goenka wants to merge 3 commits into
shreyas-goenka wants to merge 3 commits into
Conversation
Step 4 of the DMS CLI integration plan. When DATABRICKS_BUNDLE_MANAGED_STATE is set, resource state is now fetched from the deployment metadata service on the server rather than the workspace-side resources.json file: - statemgmt/state_dms.go: new LoadStateFromDMS reads resources via the SDK Bundle.ListResourcesAll and seeds the in-memory state DB via DeploymentState.OpenWithData (no local file is touched). - statemgmt/state_pull.go: when DMS is active, pull only the deployment_id pointer from managed_service.json and short-circuit. The full state is loaded later via LoadStateFromDMS so file-state lineage/serial logic stays off the DMS path entirely. - statemgmt/state_push.go: no-op under DMS — the server owns the state, there is nothing useful to upload. - cmd/bundle/utils/process.go: route through LoadStateFromDMS instead of StateDB.Open when DMS is on; reject --plan under DMS because plan-vs-state lineage checks don't translate to the server's version-based locking. - bundle/bundle.go: add Bundle.DeploymentID, the in-memory carrier for the DMS deployment record id (populated by either state pull or lock acquire). - statemgmt/managed_service_json.go: promote managed_service.json constant and struct out of the lock package so the state-read path can share them. - deploy/lock/deployment_metadata_service.go: switch to the exported names and publish b.DeploymentID after acquiring the lock so subsequent reads in the same process see it without re-reading the file. Operation reporting (writing per-resource operations back to DMS during deploy) is intentionally deferred to step 5; today the resources list returned by ListResources is empty until that lands. The release-lock-error acceptance fixture loses the "Updating deployment state..." line because PushResourcesState is now a no-op under DMS. A new plan-and-summary acceptance test exercises the full read path end-to-end and verifies both bundle plan and bundle summary hit ListResources.
…ient
Adds three table-style tests:
* NoDeploymentID — empty b.DeploymentID short-circuits without any RPC and
leaves the in-memory state DB safe to read (ExportState returns empty).
* PopulatesFromList — resources land under "resources.<key>" with their
ID and State preserved verbatim; the nil-State path is exercised too.
* ListError — DMS-side failures are wrapped with the package-level prefix
so callers can tell them apart from local-filesystem errors.
Together these cover the three branches a code reviewer would otherwise have
to read the source to verify.
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
3 tasks
The deadcode lint check flagged this as unreachable on PR #5355's CI. ManagedState returned (string, bool) but no caller existed — every consumer uses IsManagedState (the bool wrapper). Drop it; reintroduce only if a caller needs the raw value. Co-authored-by: Isaac
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
shreyas-goenka
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
3 tasks
Collaborator
|
Commit: 0e706b8 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Step 4 of the DMS CLI integration plan (see #4856 for the kitchen-sink reference). Stacked on top of #5331.
When
DATABRICKS_BUNDLE_MANAGED_STATEis set, the CLI now reads resource state from the deployment metadata service instead of the workspace-sideresources.jsonfile:bundle/statemgmt/state_dms.go— newLoadStateFromDMSreads resources via SDKBundle.ListResourcesAlland seeds the in-memory state DB viaDeploymentState.OpenWithData.bundle/statemgmt/state_pull.go— pulls only thedeployment_idpointer frommanaged_service.jsonunder DMS, leaving file-state lineage/serial logic untouched.bundle/statemgmt/state_push.go— no-op under DMS (server owns the state).cmd/bundle/utils/process.go— routes throughLoadStateFromDMSinstead ofStateDB.Openwhen DMS is on; rejects--planunder DMS because plan-vs-state lineage checks don't translate to the server's version-based locking.bundle/bundle.go— newBundle.DeploymentIDcarrier, populated by either state pull or lock acquire.bundle/statemgmt/managed_service_json.go— promotemanaged_service.jsonconstant + struct out of the lock package so the read path can share them.bundle/deploy/lock/deployment_metadata_service.go— switch to the exported names and publishb.DeploymentIDafter acquiring the lock.Acceptance test
bundle/dms/plan-and-summaryexercises the full read path: deploy creates the DMS deployment record, thenbundle planandbundle summaryeach hitListResources(twoGET /resourceslines in the recorded requests). Unit tests instate_dms_test.gocover the three branches (no deployment id, populated list, list error).Out of scope (step 5)
Operation reporting — writing per-resource Create/Update/Delete operations back to DMS during deploy — is deferred. Today
ListResourcesreturns an empty list after a deploy so plan/summary render resources as(not deployed). That's expected for step 4; step 5 plugs theOperationReporterintodirect/bundle_apply.goand makes the resource list non-trivial.Test plan
go test ./bundle/...green (incl. newTestLoadStateFromDMS_*).go test ./acceptance -run TestAccept/bundle/dmsgreen (plan-and-summary + release-lock-error).go tool golangci-lint run ./bundle/...clean.This pull request and its description were written by Isaac.