Skip to content

fix(sep-2322): stop two scored scenarios going green on a server that rejects everything - #537

Open
AmirK-S wants to merge 1 commit into
modelcontextprotocol:mainfrom
AmirK-S:fix/validate-input-baseline
Open

AmirK-S wants to merge 1 commit into
modelcontextprotocol:mainfrom
AmirK-S:fix/validate-input-baseline

Conversation

@AmirK-S

@AmirK-S AmirK-S commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Against a server that answers every request with -32000 Bad Request: Server not initialized, two scenarios in the 2026-07-28 requirement set score fully green on main (#451):

  • input-required-result-validate-input, 3/3: both checks accept any JSON-RPC error as proof that inputResponses were validated.
  • input-required-result-unsupported-methods, 2/2: its MUST NOT is never exercised when tools/list and prompts/list are both rejected.

This PR adds a control to each, following the #248 policy:

  • validate-input first calls test_input_required_result_elicitation without inputResponses and requires an InputRequiredResult, as A1 already does. If that fails, both checks are reported with untestableCheck at their existing WARNING severity.
  • unsupported-methods requires at least one probe to return a result. A server without prompts can still reject prompts/list. If neither returns a result, the check is reported with untestableCheck at FAILURE.

The reason names what the server returned, for example JSON-RPC error -32000: .... The catch path of validate-input now emits both check ids, so a throw no longer drops sep-2322-error-on-protocol-error.

Scope

This does not add the expected-vs-emitted count that #451 stays open for, nor the checks.json persistence on throw that @mohammedmessaoudene is taking.

Tests

  • New input-required-result-reject-all.test.ts: a local HTTP server that returns -32000 for every request. Both tests fail on main (expected 'SUCCESS' to be 'WARNING', expected 'SUCCESS' to be 'FAILURE') and pass here. It is a separate file so it does not collide with fix(scenarios): stop MRTR retries when no input is requested #498 in negative-mrtr.test.ts.
  • CLI, --requirements 2026-07-28 against that server: 2 fully green scenarios on main, 0 on this branch.
  • Against the everything-server, both scenarios give the same results as on main.
  • npm run typecheck, npm run lint and vitest run are green (48 files, 624 tests).

Refs #451

AI disclosure

Per AI_POLICY.md: this change, its tests and this description were written primarily by Claude Code under my direction, and I reviewed the measurements. Review replies may be AI-assisted as well.

🤖 Generated with Claude Code

… rejects everything

Against a server that answers every request with -32000 "Server not
initialized", two scenarios in the 2026-07-28 requirement set scored
fully green on main (modelcontextprotocol#451):

- input-required-result-validate-input: both checks accept any JSON-RPC
  error as proof that inputResponses were validated.
- input-required-result-unsupported-methods: the MUST NOT is never
  exercised when tools/list and prompts/list are both rejected.

validate-input now first calls the same tool without inputResponses, as
A1 does, and requires an InputRequiredResult. unsupported-methods now
requires at least one probe to return a result. Otherwise the checks are
reported through untestableCheck ("Not testable:"), per the modelcontextprotocol#248 policy,
at their existing severity (WARNING and FAILURE), instead of SUCCESS.

validate-input's catch path also emits both check ids, so a throw no
longer drops sep-2322-error-on-protocol-error from the emitted set.

Refs modelcontextprotocol#451

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pkg-pr-new

pkg-pr-new Bot commented Sep 29, 2026

Copy link
Copy Markdown

Open in StackBlitz

npx https://pkg.pr.new/@modelcontextprotocol/conformance@537

commit: 5c63aa1

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant