Cover the Ruby examples in the smoke tests - #183
Open
olaservo wants to merge 4 commits into
Open
Conversation
olaservo
force-pushed
the
ruby-smoke-tests
branch
from
August 10, 2026 03:53
0c29d05 to
b51f2f2
Compare
CI installed no Ruby toolchain and the suite ran no Ruby example, so nothing under mcp-client-ruby or weather-server-ruby was exercised by a green build. A dependency change there -- modelcontextprotocol#177 removes the base64 gem now that the anthropic gem declares it -- merges on the strength of a check that never resolved its Gemfile. The client is drivable without credentials: it connects before it looks for ANTHROPIC_API_KEY and exits 0 when there is none, the same shape the Python and TypeScript clients already rely on for their coverage. The Ruby weather server stays uncovered. The mcp gem's server does not emit the resultType field that protocol revision 2026-07-28 makes mandatory, so the test client rejects its responses; that is a fix for the gem rather than something an example can work around. The README said the gem could not negotiate the revision at all, which stopped being true at 1.1.0 -- it negotiates, then answers incompletely. Ruby 3.4 rather than the 3.2.0 floor the anthropic gem declares: 3.2 reached end of life in March 2026, and a gem's floor says what it tolerates, not what an example should tell people to run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
olaservo
force-pushed
the
ruby-smoke-tests
branch
from
August 10, 2026 04:11
b51f2f2 to
75796e0
Compare
olaservo
marked this pull request as draft
August 10, 2026 04:27
There was a problem hiding this comment.
Pull request overview
Adds Ruby MCP client coverage to the smoke-test suite and CI.
Changes:
- Installs Ruby 3.4 in CI.
- Adds Bundler preparation and Ruby client smoke testing.
- Updates smoke-test documentation and Ruby server limitations.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
.github/workflows/ci.yml |
Configures Ruby for CI. |
tests/utils.sh |
Adds Bundler dependency setup. |
tests/smoke-test.sh |
Runs the Ruby client smoke test. |
tests/README.md |
Documents Ruby coverage and requirements. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
olaservo
marked this pull request as ready for review
August 18, 2026 15:56
The client tests drive the path where no key is set: each client prints a notice and exits. They did not isolate the variable, so a key exported in the environment or sitting in a local .env sent the client into its interactive chat loop instead. Observed with a .env in mcp-client-ruby: the run reached "Query:" and waited on stdin. Empty rather than unset, because dotenv skips a name already present in ENV. All three clients check `nil? || empty?`, so an empty value takes the early exit. CI never hit this, having no key set. It affects local runs, and a run fed input rather than /dev/null would issue real API requests. Applied to the Python and TypeScript client tests too; all three share the pattern. Also correct the Ruby version comment: 3.3 is supported until March 2027, so 3.4 is not the oldest supported series. The tested version is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The server was excluded because the mcp gem did not stamp the resultType field that 2026-07-28 requires, so the test client rejected its responses. mcp 1.2.0 stamps it. Gemfile.lock is gitignored, so a fresh bundle install resolves the existing ">= 1.1.0" to 1.2.0 with no manifest change. Observed against the built test client: on 1.1.0 the run fails with "missing required resultType"; on 1.2.0 it connects and lists both tools. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README still said the Ruby weather server was uncovered and described the resultType gap as unfixable from an example. Both stopped being true when the server test landed. Also move the no-API-key comment below the Ruby server test. It describes the client tests, and sat where it read as describing the server one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
olaservo
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI installed no Ruby toolchain and the suite ran no Ruby example, so nothing under
mcp-client-rubyorweather-server-rubywas exercised by a green build — including #177, which changes the client's Gemfile.Adds
ruby/setup-rubyand tests for both Ruby examples. The client connects before it checksANTHROPIC_API_KEYand exits 0 without one, so it runs credential-free like the Python and TypeScript clients.The Ruby weather server was previously excluded: the
mcpgem negotiated2026-07-28but never emitted theresultTypefield that revision makes mandatory, so the test client rejected its responses. The gem stamps it as of 1.2.0 (2026-08-15).Gemfile.lockis gitignored, so a freshbundle installresolves the existing>= 1.1.0to 1.2.0 and no manifest change is needed here. Observed against the built test client: on 1.1.0 the run fails withInvalid result for tools/list: missing required resultType; on 1.2.0 it connects and lists both tools.Also pins
ANTHROPIC_API_KEYempty for all three client tests. They drive the path where no key is set, but did not isolate the variable, so an exported key or a local.envsent the client into its interactive chat loop instead. Empty rather than unset, because dotenv skips a name already present inENV.Verified in a fork with #177 cherry-picked on top: 8/8 pass. On Ruby 3.4
base64is no longer a default gem, so that run doubles as confirmation of #177 — without the transitive dependency the client would fail to load.The Go and Rust clients stay uncovered: both abort when no
.envfile is present, so they cannot be driven without credentials. That is a change in their own directories.