Describe the issue
bundle deploy does not pass DATABRICKS_CONFIG_PROFILE to the Terraform subprocess when a profile is specified via workspace.profile in databricks.yml or --profile.
The CLI authenticates successfully for its own operations (validate, upload), but the Terraform apply phase fails because the Terraform provider calls databricks auth token --host <host> instead of databricks auth token --profile <profile>, which is ambiguous when multiple profiles in ~/.databrickscfg share the same host.
While, databricks auth env --profile <name> correctly returns DATABRICKS_CONFIG_PROFILE in its output, it follows a different code path than bundle deploy, meaning databricks auth and databricks bundle return inconsistent results for the same profile.
This appears to be the same class of issue as:
Two other callers in the codebase already work around this by manually appending the profile to the subprocess environment:
cmd/bundle/run.go:284-285cmd/apps/run_local.go:62-63
The Terraform code path in bundle/deploy/terraform/init.go:338 does not.
Steps to reproduce the behavior
- Have multiple profiles in
~/.databrickscfg sharing the same host:
[DEFAULT]
host = https://my-workspace.cloud.databricks.com
auth_type = databricks-cli
[my-bundle-profile]
host = https://my-workspace.cloud.databricks.com
auth_type = databricks-cli
- Run
databricks bundle deploy --profile my-bundle-profile
- See error during Terraform apply
Expected Behavior
bundle deploy should pass DATABRICKS_CONFIG_PROFILE to the Terraform subprocess when a profile is specified, so that Terraform's auth resolution can disambiguate profiles sharing the same host, consistent with how databricks auth env --profile behaves.
Actual Behavior
Validation and file uploads succeed. Terraform apply fails:
Error: cannot create job: failed during request visitor: databricks-cli auth:
cannot get access token: Error: DEFAULT and my-bundle-profile match
https://my-workspace.cloud.databricks.com in ~/.databrickscfg.
Use --profile to specify which profile to use.
Config: host=https://my-workspace.cloud.databricks.com. Env: DATABRICKS_HOST
OS and CLI version
- CLI: v0.290.1
- OS: macOS (Darwin 25.3.0)
Is this a regression?
The existing workarounds in cmd/bundle/run.go and cmd/apps/run_local.go as well as the aforementioned related issues suggest this gap has been known.
The fact that two (now three) callers need the same manual patch suggests the fix really belongs in either auth.Env() or b.AuthEnv() rather than at each call site. To that end, it's clear that more robust testing could be beneficial here.
Test Coverage Recommendations
There are no tests that assert the subprocess environment contains DATABRICKS_CONFIG_PROFILE when a profile is specified.
While this looks like a multi-profile host collision error, the underlying defect (profile silently dropped in favor of host-based inferenc) is independently testable:
-
Missing test (unit): When b.AuthEnv() is called and workspace.profile is set, assert DATABRICKS_CONFIG_PROFILE is present in the returned env map. (this issue)
-
Missing test (integration): When --profile is passed to bundle deploy, assert the Terraform subprocess environment includes DATABRICKS_CONFIG_PROFILE and does NOT fall back to host-based profile resolution.
- I'd argue there should be a warning issued any time a profile is specified but auth falls back to inference. having a single code path would make this a lot easier to test and maintain.
-
Missing assertion (existing tests): Any existing test that passes a profile and spawns a subprocess should assert that the profile propagates, not just that the command succeeds (which it will if only one profile matches the host).
Debug Logs
Debug: Environment variables for Terraform: DATABRICKS_HOST, DATABRICKS_AUTH_TYPE, HOME, PATH, TMPDIR, DATABRICKS_USER_AGENT_EXTRA
DATABRICKS_CONFIG_PROFILE is absent from this list.
Describe the issue
bundle deploydoes not passDATABRICKS_CONFIG_PROFILEto the Terraform subprocess when a profile is specified viaworkspace.profileindatabricks.ymlor--profile.The CLI authenticates successfully for its own operations (validate, upload), but the Terraform apply phase fails because the Terraform provider calls
databricks auth token --host <host>instead ofdatabricks auth token --profile <profile>, which is ambiguous when multiple profiles in~/.databrickscfgshare the same host.While,
databricks auth env --profile <name>correctly returnsDATABRICKS_CONFIG_PROFILEin its output, it follows a different code path thanbundle deploy, meaningdatabricks authanddatabricks bundlereturn inconsistent results for the same profile.This appears to be the same class of issue as:
Two other callers in the codebase already work around this by manually appending the profile to the subprocess environment:
cmd/bundle/run.go:284-285cmd/apps/run_local.go:62-63The Terraform code path in
bundle/deploy/terraform/init.go:338does not.Steps to reproduce the behavior
~/.databrickscfgsharing the same host:databricks bundle deploy --profile my-bundle-profileExpected Behavior
bundle deployshould passDATABRICKS_CONFIG_PROFILEto the Terraform subprocess when a profile is specified, so that Terraform's auth resolution can disambiguate profiles sharing the same host, consistent with howdatabricks auth env --profilebehaves.Actual Behavior
Validation and file uploads succeed. Terraform apply fails:
OS and CLI version
Is this a regression?
The existing workarounds in
cmd/bundle/run.goandcmd/apps/run_local.goas well as the aforementioned related issues suggest this gap has been known.The fact that two (now three) callers need the same manual patch suggests the fix really belongs in either
auth.Env()orb.AuthEnv()rather than at each call site. To that end, it's clear that more robust testing could be beneficial here.Test Coverage Recommendations
There are no tests that assert the subprocess environment contains
DATABRICKS_CONFIG_PROFILEwhen a profile is specified.While this looks like a multi-profile host collision error, the underlying defect (profile silently dropped in favor of host-based inferenc) is independently testable:
Missing test (unit): When
b.AuthEnv()is called andworkspace.profileis set, assertDATABRICKS_CONFIG_PROFILEis present in the returned env map. (this issue)Missing test (integration): When
--profileis passed tobundle deploy, assert the Terraform subprocess environment includesDATABRICKS_CONFIG_PROFILEand does NOT fall back to host-based profile resolution.Missing assertion (existing tests): Any existing test that passes a profile and spawns a subprocess should assert that the profile propagates, not just that the command succeeds (which it will if only one profile matches the host).
Debug Logs
DATABRICKS_CONFIG_PROFILEis absent from this list.