Skip to content

Commit 97235e2

Browse files
fix(kernel): forward full OAuth U2M app bundle into kernel (PECOBLR-4040) (#914)
* fix(kernel): forward full OAuth U2M app bundle into kernel (PECOBLR-4040) On the use_kernel path, OAuth U2M forwarded only whatever the caller explicitly set, sending a bare oauth-u2m otherwise. Since PECOBLR-4039 changed the kernel core default U2M app to databricks-sql-connector / sql offline_access / port 8030, a bare U2M connection authenticated as the wrong identity. The connector is an OVERRIDE of the kernel default, so it now forwards its full coupled bundle (client_id + oauth_scopes + redirect_port). Each field falls back to the connector's registered databricks-sql-python (or azure) default from the existing PYSQL_OAUTH_* constants, giving parity with the Thrift path. Explicit caller overrides and identity_federation_client_id forwarding are preserved. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 <e.wang@databricks.com> * ai: apply changes for #914 (1 review thread) Addresses: - #3799256419 at src/databricks/sql/backend/kernel/auth_bridge.py:266 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> * ai: apply changes for #914 (1 review thread) Addresses: - #3799336213 at src/databricks/sql/backend/kernel/auth_bridge.py:275 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> * ai: apply changes for #914 (1 review thread) Addresses: - #3799379619 at src/databricks/sql/backend/kernel/auth_bridge.py:283 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> * fix(kernel): don't handle azure-oauth U2M yet; reject it (PECOBLR-4120) Azure AD U2M can't work through the kernel today: the kernel resolves OAuth endpoints only from the workspace-native OIDC config and has no Azure AD path, so the Thrift azure-oauth flow (AAD token endpoint + /user_impersonation scope) cannot be reproduced. Rather than forward an azure bundle that authenticates against the wrong endpoints, reject auth_type='azure-oauth' up front with a clear NotSupportedError pointing at the Thrift backend. The kernel U2M path now handles databricks-oauth only. Azure support is tracked by PECOBLR-4120. Also fixes the stale scope tests the prior review left red: scopes are hardcoded to PYSQL_OAUTH_SCOPES for Thrift parity (not caller- overridable), and the tests now assert that. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 <e.wang@databricks.com> * ai: apply changes for #914 (2 review threads) Addresses: - #3799622645 at src/databricks/sql/backend/kernel/auth_bridge.py:254 - #3799650240 at src/databricks/sql/backend/kernel/auth_bridge.py:300 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> * ai: apply changes for #914 (1 review thread) Addresses: - #3799717722 at src/databricks/sql/backend/kernel/auth_bridge.py:275 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> * ai: apply changes for #914 (1 review thread) Addresses: - #3799762589 at src/databricks/sql/backend/kernel/auth_bridge.py:261 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> * ai: apply changes for #914 (1 review thread) Addresses: - #3799812091 at src/databricks/sql/backend/kernel/auth_bridge.py:258 Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> * fix(kernel): forward redirect_ports list to kernel U2M (PECOBLR-4144) databricks-sql-kernel #257 landed: the pyo3 Session now takes redirect_ports (a list) and no longer accepts the single redirect_port kwarg. Update the kernel auth bridge to emit redirect_ports for databricks-oauth U2M, forwarding the databricks-sql-python app's FULL registered port list (PYSQL_OAUTH_REDIRECT_PORT_RANGE, 8020-8024) so the kernel binds the first free port — busy-port fallback, matching the Thrift DatabricksOAuthProvider. A custom client_id + explicit port pins that single port ([port]). Bump KERNEL_REV to the merged #257 commit (45a0d6a) so kernel-e2e builds against the kernel that exposes redirect_ports. Tests updated. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 <e.wang@databricks.com> --------- Signed-off-by: eric-wang-1990 <e.wang@databricks.com> Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com> Co-authored-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
1 parent 45bfee8 commit 97235e2

4 files changed

Lines changed: 243 additions & 37 deletions

File tree

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# Release History
22

3+
# Unreleased
4+
- Kernel backend (`use_kernel=True`): OAuth U2M with `auth_type="databricks-oauth"` now forwards the connector's `databricks-sql-python` OAuth-app bundle (`client_id` + `sql offline_access` scopes + redirect port) into the kernel, so a bare U2M connection authenticates as `databricks-sql-python` — parity with the Thrift path — instead of inheriting the kernel's own `databricks-sql-connector` default. A caller-supplied `oauth_client_id` (with its coupled `oauth_redirect_port`) is honored, as is a caller-supplied `oauth_scopes`; absent one, the connector default (`sql offline_access`) is forwarded. Note: the kernel binds a single U2M redirect port, so unlike the Thrift path (which tries the full `8020..8024` range) the kernel path uses only one port and does not fall back to the next port if it is already bound — pass `oauth_redirect_port` (with `oauth_client_id`) to pick a free one on a port collision. `auth_type="azure-oauth"` (Azure AD) is not yet supported on the kernel path and raises `NotSupportedError` — use the Thrift backend for it (PECOBLR-4040; Azure tracked by PECOBLR-4120)
5+
36
# 4.4.0 (2026-07-22)
47
- Raised the minimum supported Python version to 3.10, dropping the end-of-life 3.8/3.9, to update the lockfile and clear CVE-flagged dependencies in the repo (databricks/databricks-sql-python#798)
58
- Fix: `REMOVE` staging operations no longer require `staging_allowed_local_path` to be set, since removing a remote file does not touch the local filesystem (databricks/databricks-sql-python#726)

KERNEL_REV

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
eff8950428f4e6cc9975c663ec919f334962f7d0
1+
45a0d6ae1de2f203220913ba96c994ebb2d7aae4

src/databricks/sql/backend/kernel/auth_bridge.py

Lines changed: 92 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,16 @@
1515
connector's own OAuth provider because the kernel re-mints tokens
1616
itself and the client secret is not recoverable from a built
1717
provider.
18-
- **OAuth U2M** — for ``auth_type`` ``databricks-oauth`` /
19-
``azure-oauth`` (the browser authorization-code flow), the optional
20-
``oauth_client_id`` / ``oauth_redirect_port`` are forwarded to the
21-
kernel's ``auth_type='oauth-u2m'`` and the kernel runs the browser
22-
flow itself.
18+
- **OAuth U2M** — for ``auth_type`` ``databricks-oauth`` (the browser
19+
authorization-code flow), the connector's ``databricks-sql-python``
20+
app bundle (``client_id`` + ``redirect_ports`` list, with the optional
21+
``oauth_client_id`` / ``oauth_redirect_port`` overriding it) is
22+
forwarded to the kernel's ``auth_type='oauth-u2m'`` and the kernel
23+
runs the browser flow itself. ``azure-oauth`` (Azure AD) is **not yet
24+
supported** on the kernel path and is rejected with
25+
``NotSupportedError`` — the kernel resolves OAuth endpoints only from
26+
the workspace-native OIDC config and cannot drive the Azure AD flow
27+
(PECOBLR-4120).
2328
2429
``identity_federation_client_id`` is forwarded with whichever auth shape
2530
wins resolution. It selects mandatory SP-wide workload-identity token
@@ -48,6 +53,11 @@
4853
import re
4954
from typing import Any, Dict, Optional
5055

56+
from databricks.sql.auth.auth import (
57+
PYSQL_OAUTH_CLIENT_ID,
58+
PYSQL_OAUTH_REDIRECT_PORT_RANGE,
59+
PYSQL_OAUTH_SCOPES,
60+
)
5161
from databricks.sql.auth.authenticators import AccessTokenAuthProvider, AuthProvider
5262
from databricks.sql.auth.token_federation import TokenFederationProvider
5363
from databricks.sql.exc import NotSupportedError, ProgrammingError
@@ -141,15 +151,24 @@ def kernel_auth_kwargs(
141151
rather than silently picking one flow (and failing later as a
142152
confusing 401 against the wrong principal):
143153
- a custom ``credentials_provider`` *and* M2M kwargs together;
144-
- a U2M ``auth_type`` (``databricks-oauth`` / ``azure-oauth``)
145-
*and* ``oauth_client_secret`` together.
154+
- a U2M ``auth_type`` (``databricks-oauth``) *and*
155+
``oauth_client_secret`` together.
156+
157+
(``azure-oauth`` is rejected as unsupported before these guards —
158+
PECOBLR-4120.)
146159
1. **OAuth M2M** — ``oauth_client_id`` + ``oauth_client_secret``
147160
both present → forward raw creds to the kernel's ``oauth-m2m``.
148161
2. **PAT** — the built provider is (or wraps) an
149162
``AccessTokenAuthProvider`` → extract the bearer token.
150-
3. **OAuth U2M** — ``auth_type`` is ``databricks-oauth`` /
151-
``azure-oauth`` → forward optional ``oauth_client_id`` /
152-
``oauth_redirect_port`` to the kernel's ``oauth-u2m``.
163+
3. **OAuth U2M** — ``auth_type`` is ``databricks-oauth`` → forward the
164+
connector's coupled ``databricks-sql-python`` bundle (``client_id``
165+
+ ``redirect_ports`` list, defaulting scopes to ``PYSQL_OAUTH_SCOPES``
166+
when the caller supplies none) to the kernel's ``oauth-u2m``, so a
167+
bare U2M connection authenticates as ``databricks-sql-python`` —
168+
forwarding the connector's own OAuth app rather than the kernel's
169+
``databricks-sql-connector`` default (PECOBLR-4039/4040). Unlike the
170+
Thrift path, a caller-supplied ``oauth_scopes`` is honored here.
171+
``azure-oauth`` is rejected as unsupported (PECOBLR-4120).
153172
4. **Custom credentials_provider** → ``NotSupportedError`` (opaque
154173
token source; no raw creds for the kernel to own).
155174
5. Anything else → ``NotSupportedError``.
@@ -169,6 +188,25 @@ def kernel_auth_kwargs(
169188
auth_type = opts.get("auth_type")
170189
has_m2m = bool(client_id and client_secret)
171190

191+
# azure-oauth (Azure AD U2M) is not yet supported on the kernel path.
192+
# Reject it up front — before any M2M/U2M routing — so ANY azure-oauth
193+
# request gets a clear "not supported" error rather than being silently
194+
# misrouted (e.g. azure-oauth + client_id + secret would otherwise look
195+
# like M2M). The kernel resolves OAuth endpoints only from the
196+
# workspace-native OIDC config and has no Azure AD path, so the Thrift
197+
# azure-oauth flow (AAD token endpoint + /user_impersonation scope, see
198+
# AzureOAuthEndpointCollection) cannot be reproduced here. Forwarding an
199+
# azure bundle would authenticate against the wrong endpoints, so we fail
200+
# loudly at session-open. Tracked by PECOBLR-4120.
201+
if auth_type == "azure-oauth":
202+
raise NotSupportedError(
203+
"use_kernel=True does not support auth_type='azure-oauth' (Azure "
204+
"AD U2M) yet: the kernel resolves OAuth endpoints only from the "
205+
"workspace-native OIDC configuration and cannot drive the Azure AD "
206+
"authorization/token flow. Use the Thrift backend (default) for "
207+
"azure-oauth. Tracked by PECOBLR-4120."
208+
)
209+
172210
# 0. Ambiguity guards — fail before any flow is chosen.
173211
if client_secret and opts.get("credentials_provider") is not None:
174212
raise NotSupportedError(
@@ -178,7 +216,7 @@ def kernel_auth_kwargs(
178216
"kernel-managed M2M, or use the Thrift backend (default) for "
179217
"credentials_provider."
180218
)
181-
if client_secret and auth_type in ("databricks-oauth", "azure-oauth"):
219+
if client_secret and auth_type == "databricks-oauth":
182220
raise NotSupportedError(
183221
f"Ambiguous auth on use_kernel=True: auth_type={auth_type!r} selects "
184222
"the U2M browser flow, but oauth_client_secret was also provided "
@@ -214,16 +252,33 @@ def kernel_auth_kwargs(
214252
return kwargs
215253

216254
# 3. OAuth U2M — browser authorization-code flow; the kernel runs it.
217-
if auth_type in ("databricks-oauth", "azure-oauth"):
218-
kwargs = {"auth_type": "oauth-u2m"}
219-
if client_id:
220-
kwargs["client_id"] = client_id
255+
# Only databricks-oauth reaches here (azure-oauth rejected up front).
256+
# Forward the connector's own databricks-sql-python bundle instead of
257+
# the kernel's databricks-sql-connector default, for parity with the
258+
# Thrift path. client_id + redirect ports are coupled per app (each
259+
# registers its own redirect URIs): a caller port only overrides the
260+
# default when an explicit client_id is also supplied. A caller may
261+
# override oauth_scopes; absent one we forward PYSQL_OAUTH_SCOPES as
262+
# the default. We forward the FULL PYSQL_OAUTH_REDIRECT_PORT_RANGE as
263+
# ``redirect_ports`` so the kernel binds the first free port (busy-port
264+
# fallback), mirroring the Thrift DatabricksOAuthProvider which retries
265+
# the next port when one is bound. A caller overriding client_id
266+
# supplies its own single registered port.
267+
if auth_type == "databricks-oauth":
221268
redirect_port = opts.get("oauth_redirect_port")
222-
if redirect_port is not None:
223-
kwargs["redirect_port"] = int(redirect_port)
269+
# Honor a caller-supplied oauth_scopes (normalized to a list of
270+
# strings); fall back to the connector default when none is given.
224271
scopes = _normalize_scopes(opts.get("oauth_scopes"))
225-
if scopes is not None:
226-
kwargs["oauth_scopes"] = scopes
272+
kwargs = {
273+
"auth_type": "oauth-u2m",
274+
"client_id": client_id or PYSQL_OAUTH_CLIENT_ID,
275+
"redirect_ports": (
276+
[_coerce_redirect_port(redirect_port)]
277+
if client_id and redirect_port is not None
278+
else list(PYSQL_OAUTH_REDIRECT_PORT_RANGE)
279+
),
280+
"oauth_scopes": scopes if scopes is not None else list(PYSQL_OAUTH_SCOPES),
281+
}
227282
if federation_client_id:
228283
kwargs["identity_federation_client_id"] = federation_client_id
229284
return kwargs
@@ -251,12 +306,29 @@ def kernel_auth_kwargs(
251306
raise NotSupportedError(
252307
f"use_kernel=True requires PAT (access_token), OAuth M2M "
253308
f"(oauth_client_id + oauth_client_secret), or OAuth U2M "
254-
f"(auth_type='databricks-oauth' / 'azure-oauth'), but got "
309+
f"(auth_type='databricks-oauth'), but got "
255310
f"{provider_desc} with auth_type={auth_type!r}. Use the Thrift "
256311
"backend (default) for other auth flows."
257312
)
258313

259314

315+
def _coerce_redirect_port(redirect_port: Any) -> int:
316+
"""Coerce an ``oauth_redirect_port`` value (which may arrive as a string,
317+
e.g. from a DSN) to an int.
318+
319+
A non-numeric value is a caller error; surface it as a PEP 249
320+
``ProgrammingError`` (as ``_normalize_scopes`` does for malformed
321+
``oauth_scopes``) rather than a bare ``ValueError``, so callers get a
322+
consistent, actionable exception type for garbled input."""
323+
try:
324+
return int(redirect_port)
325+
except (TypeError, ValueError):
326+
raise ProgrammingError(
327+
f"oauth_redirect_port must be an integer (or a string parseable as "
328+
f"one), got {redirect_port!r}."
329+
)
330+
331+
260332
def _normalize_scopes(scopes: Any) -> Optional[list]:
261333
"""Normalise an ``oauth_scopes`` value to a list of strings, or
262334
``None`` to let the kernel apply its defaults.

0 commit comments

Comments
 (0)