Close cancelled HTTP exchanges and harden auth validation

Review-feedback round on the conformance burn-down:

- Cancelled requests no longer leave the legacy streamable-HTTP POST
  hanging. The dispatcher emits a RequestSettled marker when a handler is
  cancelled without producing a response; the transport consumes it by
  closing the per-request stream, so the POST's SSE stream terminates
  without a response frame and JSON-response mode completes with 204 No
  Content (the client treats 202/204 alike). Per-request streams are
  released instead of leaking until session teardown, and a handler that
  survives the cancellation still delivers its normal response. The
  marker is type-visible on the dispatcher write stream and is stripped
  by every serializing transport, so it can never appear on a wire.
- A bearer token whose audience cannot be canonicalized (out-of-range or
  non-numeric port) is now rejected with the standard 401 invalid_token
  instead of raising through the auth middleware as a 500.
- The bundled authorization server's /register now accepts only https
  redirect URIs or http on a loopback host; other schemes on loopback
  hosts (ftp, ws, javascript, custom) are rejected.
- OAuth client scope selection falls back to the caller-configured
  OAuthClientMetadata.scope when neither the WWW-Authenticate challenge
  nor protected-resource metadata names scopes, matching the TypeScript
  SDK, so the documented migration path works as written.
- The cross-dispatcher contract that handler-raised MCPError subclasses
  surface to callers as plain MCPError is now pinned by an explicit test
  and documented; rehydrate with from_error when the subclass matters.
- Docs: migration notes for the bearer-challenge wire-shape changes and
  the cancellation wire spellings; story READMEs updated to the landed
  error contract; strict-capabilities doc corrected to state that
  resources/unsubscribe is gated by the base resources capability only.

Author: maxisbey

docs/client/index.md

@@ -145,7 +145,7 @@ The resource verbs come in pairs: two ways to list, one way to read.
 
 `read_resource` returns `contents`, a list of `TextResourceContents` or `BlobResourceContents`. Same idea as tool content: narrow with `isinstance`, then read `.text` (or `.blob`).
 
-A client can also **subscribe** to a resource and be told when it changes: `subscribe_resource(uri)` and `unsubscribe_resource(uri)`, same shape as everything else here. `MCPServer` doesn't implement that half. It says so up front (`server_capabilities.resources.subscribe` is `False`) and answers the request with an `MCPError`: `-32601`, *Method not found*. With `strict_capabilities=True` you get the same `-32601` without the round trip: the client sees `server_capabilities.resources.subscribe` is falsy and never sends the request. A server that does support subscriptions is built on the low-level `Server` (**The low-level Server**).
+A client can also **subscribe** to a resource and be told when it changes: `subscribe_resource(uri)` and `unsubscribe_resource(uri)`, same shape as everything else here. `MCPServer` doesn't implement that half. It says so up front (`server_capabilities.resources.subscribe` is `False`) and answers the request with an `MCPError`: `-32601`, *Method not found*. With `strict_capabilities=True` you get the same `-32601` for `subscribe_resource` without the round trip: the client sees `server_capabilities.resources.subscribe` is falsy and never sends the request. `unsubscribe_resource` is still sent — only the base `resources` capability gates it, matching the TypeScript SDK — so its `-32601` comes from the server. A server that does support subscriptions is built on the low-level `Server` (**The low-level Server**).
 
 ## Prompts
 

docs/migration.md

@@ -41,13 +41,19 @@ An unhandled exception in a request handler now produces JSON-RPC error `-32603`
 code `0` with `str(exc)` as the message, leaking handler internals to the peer; the
 exception is still logged server-side via `logger.exception`. To send a specific
 code and message, raise `MCPError` (unchanged); a pydantic `ValidationError` is
-still mapped to `INVALID_PARAMS`.
+still mapped to `INVALID_PARAMS`. An `MCPError` *subclass* raised by a handler
+(e.g. `UrlElicitationRequiredError`) reaches the caller as plain `MCPError` with
+the same `code`, `message`, and `data` on every dispatch path — including the
+in-process `Client(server)` — so catch `MCPError` and match on `error.code`
+rather than on the subclass type.
 
 A request cancelled via `notifications/cancelled` now receives no response at all,
 per the spec's SHOULD. v1 answered the cancelled request with an error
 (`code=0, message="Request cancelled"`). The sender's awaiting call already fails
 with anyio cancellation when its scope is cancelled, so no reply is needed to
-unblock it.
+unblock it. On legacy streamable HTTP this means the cancelled request's POST SSE
+stream now terminates without a response frame, and in JSON-response mode the POST
+completes with `204 No Content`.
 
 ### A second `initialize` on an already-initialized session is rejected
 
@@ -148,7 +154,7 @@ When no authorization-server metadata document could be discovered at all, the f
 
 The specification's scope-selection chain is two steps: the `scope` parameter from the `WWW-Authenticate` challenge, then `scopes_supported` from the Protected Resource Metadata document, *otherwise the `scope` parameter is omitted*. The SDK inserted an extra fallback between those two steps — the **authorization-server** metadata's `scopes_supported` — which over-requests (an authorization server may serve many resource servers, so its list is a superset of any one resource's) and caused real `access_denied` failures ([#1307](https://github.com/modelcontextprotocol/python-sdk/issues/1307)). That fallback is removed: when neither the challenge nor the PRM names scopes, the client now omits the `scope` parameter and lets the authorization server apply its defaults.
 
-This also affects the SEP-2207 `offline_access` augmentation, which only fires once a base scope was selected: if the authorization server's `scopes_supported` was your only scope source, the client now sends no `scope` at all (not even `offline_access`) and the authorization server's defaults decide whether a refresh token is issued. In either case, if you relied on the removed fallback, pass an explicit `scope` on the `OAuthClientMetadata` you give to `OAuthClientProvider`.
+This also affects the SEP-2207 `offline_access` augmentation, which only fires once a base scope was selected: if the authorization server's `scopes_supported` was your only scope source, the client now sends no `scope` at all (not even `offline_access`) and the authorization server's defaults decide whether a refresh token is issued. In either case, if you relied on the removed fallback, pass an explicit `scope` on the `OAuthClientMetadata` you give to `OAuthClientProvider`. That explicit scope ranks *below* the spec's two sources (matching the TypeScript SDK): a `scope` on the `WWW-Authenticate` challenge or a PRM `scopes_supported` still wins, so the explicit value only takes effect when both are silent.
 
 ### `get_session_id` callback removed from `streamable_http_client`
 
@@ -1583,6 +1589,12 @@ Leaving `resource_server_url=None` continues to disable the check entirely (ther
 
 `RefreshToken` gains an optional `resource` field so an `OAuthAuthorizationServerProvider` can propagate the original grant's audience binding through `exchange_refresh_token`; without it a refreshed access token would carry no audience and be rejected. `BearerAuthBackend.__init__` gains a keyword-only `resource_server_url: AnyHttpUrl | None = None`, wired automatically from `AuthSettings.enforced_audience`; `None` (the default, and what the SDK passes when `verifier_validates_audience` is set) means no audience is enforced.
 
+The error responses the bearer gate sends changed shape in the same release, following RFC 6750 §3:
+
+- A request presenting **no credentials at all** (no `Authorization: Bearer` header) is now answered with a bare challenge — `WWW-Authenticate: Bearer scope="…", resource_metadata="…"` and an empty JSON body `{}` — instead of `error="invalid_token", error_description="Authentication required"` in both the header and the body. RFC 6750 §3.1 says the `error` attribute should only appear when the request actually carried a token, so "no token" and "rejected token" are now distinguishable; anything matching on the literal `Authentication required` string or expecting a non-empty 401 body must be updated.
+- Every challenge — the `401`s and the `403` — now advertises the configured `required_scopes` in an RFC 6750 `scope="…"` parameter, which spec-conformant clients (including this SDK's OAuth client) read as the first step of scope selection and step-up authorization.
+- The `error_description` strings changed. A rejected token now states the failure: `The access token is malformed or unknown`, `The access token has expired`, `The access token carries no audience claim`, or `The access token was issued for a different resource` (previously all of these — where they were rejected at all — said `Authentication required`). The `403 insufficient_scope` description is now the fixed string `The access token lacks a required scope` instead of `Required scope: {scope}`; the required scope moved to the machine-readable `scope=` challenge parameter. Treat `error_description` as human-readable prose, not a contract.
+
 ### Bundled authorization server: RFC-correct redirect-URI handling
 
 Two fixes to the optional bundled OAuth authorization server (the `auth_server_provider=` path).

examples/clients/simple-auth-client/mcp_simple_auth_client/main.py

@@ -24,7 +24,7 @@
 from mcp.client.sse import sse_client
 from mcp.client.streamable_http import streamable_http_client
 from mcp.shared.auth import OAuthClientInformationFull, OAuthClientMetadata, OAuthToken
-from mcp.shared.message import SessionMessage
+from mcp.shared.message import RequestSettled, SessionMessage
 
 
 class InMemoryTokenStorage(TokenStorage):
@@ -248,8 +248,8 @@ async def _default_redirect_handler(authorization_url: str) -> None:
 
     async def _run_session(
         self,
-        read_stream: ReadStream[SessionMessage | Exception],
-        write_stream: WriteStream[SessionMessage],
+        read_stream: ReadStream[SessionMessage | Exception | RequestSettled],
+        write_stream: WriteStream[SessionMessage | RequestSettled],
     ):
         """Run the MCP session with the given streams."""
         print("🤝 Initializing MCP session...")

examples/stories/error_handling/README.md

@@ -35,9 +35,10 @@ uv run python -m stories.error_handling.client --http --server server_lowlevel
 ## Caveats
 
 - The "any other exception → `is_error` result" contract on `MCPServer` and the
-  "uncaught exception → `code=0`" behaviour on `lowlevel.Server` are **not
-  shown** — the contract is under design and the legacy code is a known spec
-  divergence. This story will grow those cases once the contract lands.
+  "uncaught exception → `-32603` `Internal server error`" behaviour on
+  `lowlevel.Server` are **not shown** here. The lowlevel reply is deliberately
+  opaque — handler internals never reach the peer; the exception is logged
+  server-side.
 - `MCPServer` prefixes the execution-error message with
   `"Error executing tool {name}: "`; build a `CallToolResult` directly from a
   lowlevel handler if you need verbatim control.

examples/stories/streaming/README.md

@@ -56,10 +56,10 @@ uv run python -m stories.streaming.client --http --server server_lowlevel
   OpenTelemetry instead of `notifications/message`. It is shown here because
   servers still need to support 2025-era clients during that window. Progress
   and cancellation are **not** deprecated. TODO(maxisbey): revisit before beta.
-- When a request is cancelled the server currently replies with
-  `ErrorData(code=0, message="Request cancelled")`; the spec says it should not
-  reply at all. The client never observes it (its awaiting task is already
-  cancelled), so this story does not assert on the reply.
+- When a request is cancelled the server sends no reply at all, per the spec's
+  SHOULD. The client's awaiting task is already cancelled, so there is nothing
+  to observe on that call; this story asserts only that the in-flight call was
+  cancelled and that the session survives for a follow-up call.
 
 ## Spec
 

src/mcp/client/__main__.py

@@ -12,7 +12,7 @@
 from mcp.client.session import ClientSession
 from mcp.client.sse import sse_client
 from mcp.client.stdio import StdioServerParameters, stdio_client
-from mcp.shared.message import SessionMessage
+from mcp.shared.message import RequestSettled, SessionMessage
 from mcp.shared.session import RequestResponder
 
 if not sys.warnoptions:
@@ -33,8 +33,8 @@ async def message_handler(
 
 
 async def run_session(
-    read_stream: ReadStream[SessionMessage | Exception],
-    write_stream: WriteStream[SessionMessage],
+    read_stream: ReadStream[SessionMessage | Exception | RequestSettled],
+    write_stream: WriteStream[SessionMessage | RequestSettled],
     client_info: types.Implementation | None = None,
 ):
     async with ClientSession(

src/mcp/client/_transport.py

@@ -6,11 +6,14 @@
 from typing import Protocol
 
 from mcp.shared._stream_protocols import ReadStream, WriteStream
-from mcp.shared.message import SessionMessage
+from mcp.shared.message import RequestSettled, SessionMessage
 
 __all__ = ["ReadStream", "WriteStream", "Transport", "TransportStreams"]
 
-TransportStreams = tuple[ReadStream[SessionMessage | Exception], WriteStream[SessionMessage]]
+TransportStreams = tuple[
+    ReadStream[SessionMessage | Exception | RequestSettled],
+    WriteStream[SessionMessage | RequestSettled],
+]
 
 
 class Transport(AbstractAsyncContextManager[TransportStreams], Protocol):

src/mcp/client/auth/oauth2.py

@@ -645,12 +645,16 @@ async def async_auth_flow(self, request: httpx.Request) -> AsyncGenerator[httpx.
                         self.context.client_info = None
                         self.context.clear_tokens()
 
-                    # Step 3: Apply scope selection strategy
+                    # Step 3: Apply scope selection strategy. The configured client-metadata
+                    # scope is the lowest-priority fallback, and the selection is written back
+                    # so registration (Step 4) and authorization (Step 5) read it — a later
+                    # re-auth therefore falls back to this selection, not the constructor value.
                     self.context.client_metadata.scope = get_client_metadata_scopes(
                         extract_scope_from_www_auth(response),
                         self.context.protected_resource_metadata,
                         self.context.oauth_metadata,
                         self.context.client_metadata.grant_types,
+                        self.context.client_metadata.scope,
                     )
 
                     # Step 4: Register client or use URL-based client ID (CIMD)

src/mcp/client/auth/utils.py

@@ -105,18 +105,25 @@ def get_client_metadata_scopes(
     protected_resource_metadata: ProtectedResourceMetadata | None,
     authorization_server_metadata: OAuthMetadata | None = None,
     client_grant_types: list[str] | None = None,
+    client_metadata_scope: str | None = None,
 ) -> str | None:
-    """Select effective scopes and augment for refresh token support."""
-    selected_scope: str | None = None
+    """Select effective scopes and augment for refresh token support.
 
-    # MCP spec scope selection priority:
+    Follows the MCP spec's scope-selection strategy (challenge scope, then PRM
+    `scopes_supported`), with the caller's pre-configured `OAuthClientMetadata.scope` as a
+    final SDK-defined fallback (TypeScript-SDK parity) before omitting the parameter.
+    """
+    # Scope selection priority (1-2 are the spec's chain; 3 is the SDK fallback):
     #   1. WWW-Authenticate header scope
     #   2. PRM scopes_supported
-    #   3. Omit scope parameter
+    #   3. Caller-supplied client metadata scope
+    #   4. Omit scope parameter
     if www_authenticate_scope is not None:
         selected_scope = www_authenticate_scope
     elif protected_resource_metadata is not None and protected_resource_metadata.scopes_supported is not None:
         selected_scope = " ".join(protected_resource_metadata.scopes_supported)
+    else:
+        selected_scope = client_metadata_scope
 
     # SEP-2207: append offline_access when the AS supports it and the client can use refresh tokens
     if (

src/mcp/client/session.py

@@ -45,7 +45,7 @@
     x_mcp_header_map,
 )
 from mcp.shared.jsonrpc_dispatcher import JSONRPCDispatcher
-from mcp.shared.message import ClientMessageMetadata, SessionMessage
+from mcp.shared.message import ClientMessageMetadata, RequestSettled, SessionMessage
 from mcp.shared.session import RequestResponder
 from mcp.shared.transport_context import TransportContext
 
@@ -215,8 +215,8 @@ class ClientSession:
 
     def __init__(
         self,
-        read_stream: ReadStream[SessionMessage | Exception] | None = None,
-        write_stream: WriteStream[SessionMessage] | None = None,
+        read_stream: ReadStream[SessionMessage | Exception | RequestSettled] | None = None,
+        write_stream: WriteStream[SessionMessage | RequestSettled] | None = None,
         read_timeout_seconds: float | None = None,
         sampling_callback: SamplingFnT | None = None,
         elicitation_callback: ElicitationFnT | None = None,