Support stateless streamable-HTTP remote servers by gkatz2 · Pull Request #4515 · stacklok/toolhive

gkatz2 · 2026-04-03T03:12:54Z

Summary

ToolHive had no way to proxy stateless streamable-HTTP MCP servers — servers that accept JSON-RPC only via POST and never open an SSE stream. When proxied, these servers showed zero tools because the proxy forwarded GET requests that stateless servers don't handle, preventing clients from completing tool discovery.
Adds a --stateless flag to thv run that: (1) returns 405 for GET, HEAD, and DELETE requests at the proxy instead of forwarding them, and (2) switches the health check to a POST-based JSON-RPC ping appropriate for servers without an SSE endpoint.

Type of change

Test plan

Unit tests (task test)
Linting (task lint-fix)
Manual testing (describe below)

Verified against a real stateless streamable-HTTP server: GET and HEAD requests to the proxy return 405 with method not allowed: server is stateless (POST only); POST requests are forwarded to the remote server. E2E tests are not included — no stateless MCP server fixture exists in the test infrastructure.

Changes

File	Change
`cmd/thv/app/run_flags.go`	Add `--stateless` CLI flag
`pkg/runner/config.go`	Add `Stateless bool` to `RunConfig`
`pkg/runner/config_builder.go`	Add `WithStateless()` builder option
`pkg/runner/runner.go`	Wire stateless config to HTTP transport; warn when flag used without `--url`
`pkg/transport/http.go`	Add `SetStateless()` setter and `buildProxyOptions()` helper
`pkg/transport/proxy/transparent/transparent_proxy.go`	Add `WithStateless()` option, method gate middleware, stateless pinger selection
`pkg/transport/proxy/transparent/pinger.go`	Add `StatelessMCPPinger` for POST-based health checks
`pkg/transport/proxy/transparent/pinger_test.go`	Unit tests for `StatelessMCPPinger`
`pkg/transport/proxy/transparent/method_gate_test.go`	Unit tests for `statelessMethodGate`

Does this introduce a user-facing change?

Yes. Adds --stateless flag to thv run. When set, the proxy returns 405 for GET, HEAD, and DELETE requests instead of forwarding them, enabling use with MCP servers that implement streamable-HTTP stateless mode.

Generated with Claude Code

Remote MCP servers implementing the streamable-HTTP transport in stateless mode (POST-only, no SSE) show zero tools when proxied by ToolHive. The transparent proxy forwards all HTTP methods including GET; stateless servers don't serve SSE on GET, so clients never complete tool discovery. Add a --stateless flag to thv run that gates GET, HEAD, and DELETE requests at the proxy with 405, and switches the health check to a POST-based JSON-RPC ping appropriate for servers without an SSE endpoint. Fixes stacklok#4514 Signed-off-by: Greg Katz <gkatz@indeed.com>

codecov · 2026-04-03T03:19:11Z

Codecov Report

❌ Patch coverage is 62.68657% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.90%. Comparing base (6adbe2b) to head (8a7267c).
⚠️ Report is 15 commits behind head on main.

Files with missing lines	Patch %	Lines
...g/transport/proxy/transparent/transparent_proxy.go	50.00%	5 Missing and 3 partials ⚠️
pkg/runner/runner.go	0.00%	5 Missing ⚠️
pkg/runner/config_builder.go	0.00%	4 Missing ⚠️
pkg/transport/http.go	63.63%	3 Missing and 1 partial ⚠️
pkg/transport/proxy/transparent/pinger.go	87.09%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4515      +/-   ##
==========================================
- Coverage   69.13%   68.90%   -0.23%     
==========================================
  Files         502      505       +3     
  Lines       51959    52440     +481     
==========================================
+ Hits        35921    36133     +212     
- Misses      13254    13509     +255     
- Partials     2784     2798      +14

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Regenerate CLI and Swagger docs to include the new --stateless flag added to the run command. Signed-off-by: Greg Katz <gkatz@indeed.com>

jerm-dro

Thanks for the PR and the well-written issue — this is a real gap for stateless streamable-HTTP servers. One blocker on the remote-only restriction, plus suggestions inline.

jerm-dro · 2026-04-03T20:44:29Z

pkg/runner/runner.go

+		// Configure stateless mode if requested. Remote workloads always use
+		// HTTPTransport, so a failed cast here is a programming error.
+		if r.Config.Stateless {
+			httpT, ok := transportHandler.(*transport.HTTPTransport)
+			if !ok {
+				return fmt.Errorf("internal error: remote transport is not an HTTPTransport")
+			}
+			httpT.SetStateless(true)
+		}


suggestion: SetStateless is called on the transport after construction but before Start. This works, but it's a mutation window — if calls get reordered, stateless mode silently doesn't apply. Consider passing it through as a transport constructor option (like WithRemoteBasePath) to make it impossible to miss. This follows the same setter pattern as SetOnHealthCheckFailed, so it's at least consistent with the current codebase — just noting the opportunity.

Agreed about the mutation window. The setter pattern here matches SetOnHealthCheckFailed, SetOnUnauthorizedResponse, SetTokenSource, and SetRemoteURL — all called between construction and Start() in the same block. Moving one to a constructor option while leaving the others as setters would be inconsistent. I think the right approach is a broader refactor that converts all of these to constructor options at once, rather than doing it piecemeal.

If you like, I can do that refactor as part of this PR. Or we can leave that for a follow-up. Which would you prefer?

jerm-dro · 2026-04-03T20:44:31Z

pkg/runner/runner.go

+	// Warn if stateless mode is set for a non-remote workload — it has no effect.
+	if r.Config.Stateless && r.Config.RemoteURL == "" {
+		slog.Warn("--stateless has no effect for local container workloads; use it only with remote URLs",
+			"server", r.Config.BaseName)
+	}


blocker: This is gated on RemoteURL != "", but a local container exposing stateless streamable-HTTP (not stdio) would have the same problem with GET-based health checks and GET forwarding. If a user runs thv run --transport streamable-http --stateless some-image, the proxy would still forward GETs to the container. The flag should work for any stateless streamable-HTTP server, not just remote ones.

Fixed in 9109460. The stateless configuration now runs after the remote-URL block, so it applies to any HTTPTransport regardless of whether RemoteURL is set. Also updated the comments, flag text, and generated docs to remove the remote-only language.

jerm-dro · 2026-04-03T20:44:33Z

pkg/transport/proxy/transparent/pinger_test.go

suggestion: The unit tests cover the pinger and method gate well, but an e2e test would give confidence this doesn't regress end-to-end. The existing e2e infrastructure already invokes thv run directly (e.g., health_check_zombie_test.go), so adding a test that stands up a minimal stateless HTTP server (POST-only, returns 405 on GET) and verifies the proxy returns 405 on GET and forwards POST correctly should be straightforward. Separately, adding a stateless mode to yardstick would make it easier to validate this locally — not a requirement for this PR, but it would help since there's no stateless MCP server fixture in the test infrastructure today.

Added in 8a7267c. The test starts an in-process mock MCP server (POST-only, returns valid JSON-RPC), runs thv run --stateless <mockURL>/mcp, and verifies GET→405, POST→forwarded, DELETE→405.

Move stateless configuration outside the remote-URL guard so --stateless works for both remote URLs and local container workloads. Add Allow header to 405 responses per RFC 9110. Update comments and flag text to remove remote-only language. Signed-off-by: Greg Katz <gkatz@indeed.com>

Verify that --stateless makes the proxy reject GET and DELETE with 405 and forward POST to the upstream MCP server. Uses an in-process mock server to avoid external dependencies. Signed-off-by: Greg Katz <gkatz@indeed.com>

gkatz2 requested review from ChrisJBurns, JAORMX, amirejaz, blkt, jhrozek, lujunsan, rdimitrov and yrobla as code owners April 3, 2026 03:12

github-actions bot added the size/M Medium PR: 300-599 lines changed label Apr 3, 2026

Add --stateless flag to generated docs

9d0fdea

Regenerate CLI and Swagger docs to include the new --stateless flag added to the run command. Signed-off-by: Greg Katz <gkatz@indeed.com>

github-actions bot added size/M Medium PR: 300-599 lines changed and removed size/M Medium PR: 300-599 lines changed labels Apr 3, 2026

jerm-dro self-requested a review April 3, 2026 19:44

jerm-dro requested changes Apr 3, 2026

View reviewed changes

gkatz2 added 2 commits April 3, 2026 15:50

Add E2E test for stateless proxy mode

8a7267c

Verify that --stateless makes the proxy reject GET and DELETE with 405 and forward POST to the upstream MCP server. Uses an in-process mock server to avoid external dependencies. Signed-off-by: Greg Katz <gkatz@indeed.com>

github-actions bot added size/L Large PR: 600-999 lines changed and removed size/M Medium PR: 300-599 lines changed labels Apr 3, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support stateless streamable-HTTP remote servers#4515

Support stateless streamable-HTTP remote servers#4515
gkatz2 wants to merge 4 commits intostacklok:mainfrom
gkatz2:feat/stateless-streamable-http

gkatz2 commented Apr 3, 2026

Uh oh!

codecov bot commented Apr 3, 2026 •

edited

Loading

Uh oh!

jerm-dro left a comment

Uh oh!

jerm-dro Apr 3, 2026

Uh oh!

gkatz2 Apr 3, 2026

Uh oh!

jerm-dro Apr 3, 2026

Uh oh!

gkatz2 Apr 3, 2026

Uh oh!

jerm-dro Apr 3, 2026

Uh oh!

gkatz2 Apr 3, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

gkatz2 commented Apr 3, 2026

Summary

Type of change

Test plan

Changes

Does this introduce a user-facing change?

Uh oh!

codecov bot commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jerm-dro left a comment

Choose a reason for hiding this comment

Uh oh!

jerm-dro Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

gkatz2 Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

jerm-dro Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

gkatz2 Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

jerm-dro Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

gkatz2 Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Apr 3, 2026 •

edited

Loading

gkatz2 Apr 3, 2026 •

edited

Loading