Skip to content

Commit 07e8a0c

Browse files
committed
fix(client,server): correct SEP-2549 cache invalidation and make caching configurable
Listings are scoped private, not public. tools/list is filtered per caller (#1111), so a shared cache must never serve one principal's listing to another; do not restore PUBLIC without gating it on the absence of a list filter. Client-side, the cache lookup moved inside Mono.defer so an assembled Mono stays cold, a generation counter drops a response that was in flight when its group was invalidated, and a listing spanning several pages is not cached at all, because stitching a cached first page to a freshly fetched later page mixes two server snapshots. ttlMs is validated on the result builders rather than in the compact constructor: a bad server hint must not make the whole listing unparseable. McpClientCacheStore keeps a Caffeine or shared-store implementation a user choice, so mcp-core stays at its three compile dependencies.
1 parent 01981b9 commit 07e8a0c

18 files changed

Lines changed: 1313 additions & 159 deletions

docs/client.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -496,6 +496,62 @@ var client = McpClient.sync(transport)
496496
.build();
497497
```
498498

499+
### Result Caching
500+
501+
When a server marks a response cacheable, the client stores it and answers later identical calls
502+
from that store instead of going back to the server. This covers `listTools`, `listPrompts`,
503+
`listResources`, `listResourceTemplates`, and `readResource`.
504+
505+
Caching is on by default and does nothing until a server opts in, because only a response that
506+
carries a time to live (TTL) is ever stored. An entry is dropped when its TTL lapses, when the
507+
matching `*_changed` or `resources/updated` notification arrives, and when the client reconnects
508+
or closes.
509+
510+
To read current server state without waiting for either the TTL or a notification, drop every
511+
entry:
512+
513+
```java
514+
client.invalidateCache();
515+
ListToolsResult fresh = client.listTools();
516+
```
517+
518+
To ignore server TTLs altogether, turn caching off:
519+
520+
```java
521+
var client = McpClient.sync(transport)
522+
.enableResultCaching(false)
523+
.build();
524+
```
525+
526+
**Choosing where entries are kept**
527+
528+
Entries live in a bounded in-memory store that holds 512 of them and evicts the oldest first. To
529+
use a cache library or share one store across several clients, implement `McpClientCacheStore` and
530+
pass it to the builder:
531+
532+
```java
533+
// MyCacheStore is your own implementation, for example over Caffeine
534+
var client = McpClient.sync(transport)
535+
.cacheStore(new MyCacheStore())
536+
.build();
537+
```
538+
539+
A store receives an `McpClientCacheKey` identifying the request, the value, and a TTL in
540+
milliseconds. It must be safe for concurrent use, and it must not return an entry whose TTL has
541+
lapsed. The client decides what may be cached and when an entry has to go, so a store only has to
542+
honor those decisions.
543+
544+
**What the client won't cache**
545+
546+
- A listing that spans several pages. Pairing a cached first page with a later page fetched fresh
547+
would mix two different views of the server's catalog.
548+
- A response whose TTL is missing, zero, or negative.
549+
- A response that arrives after the notification that invalidates it, which would otherwise pin a
550+
stale listing for the whole TTL.
551+
552+
A TTL longer than 24 hours is capped at 24 hours, so a client can't be left serving a response the
553+
server has no way to invalidate.
554+
499555
### Pagination
500556

501557
`listTools`, `listResources`, `listResourceTemplates`, and `listPrompts` all accept an optional opaque `cursor` string, and their results carry a `nextCursor` that is non-null while more pages remain. Loop until `nextCursor` is `null` to collect every page:

docs/server.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -537,6 +537,68 @@ The same `addToolFilter(...)` method is available on the stateless builders.
537537
- With STDIO there is no per-request metadata, so the filter receives `McpTransportContext.EMPTY` and has nothing to key
538538
on.
539539

540+
### Caching Hints for Listings
541+
542+
A server can tell clients how long they may reuse a listing before asking for it again, so that a
543+
client polling `tools/list` on a stable catalog stops paying for a round trip each time. The hint
544+
travels on the response as a time to live (TTL) in milliseconds and a cache scope; a client that
545+
honors it serves the cached listing until the TTL lapses or a `*_changed` notification arrives.
546+
547+
Listings carry no TTL by default. Set one with `listCache(...)`, which applies to `tools/list`,
548+
`prompts/list`, `resources/list`, and `resources/templates/list`:
549+
550+
=== "Sync"
551+
552+
```java
553+
McpServer.sync(transportProvider)
554+
.tools(calculatorTool, weatherTool)
555+
.listCache(Duration.ofMinutes(5), CacheScope.PRIVATE)
556+
.build();
557+
```
558+
559+
=== "Async"
560+
561+
```java
562+
McpServer.async(transportProvider)
563+
.tools(calculatorTool, weatherTool)
564+
.listCache(Duration.ofMinutes(5), CacheScope.PRIVATE)
565+
.build();
566+
```
567+
568+
The same `listCache(...)` method is available on the stateless builders.
569+
570+
!!! warning "`PUBLIC` means any caller may be served the response"
571+
572+
`CacheScope.PUBLIC` tells a shared cache, such as an MCP gateway, that it may serve one
573+
principal the response it stored for another. Use it only for a listing that is identical for
574+
every caller. Any registered [tool filter](#filtering-the-tool-listing-per-request) makes the
575+
listing caller-specific, so keep the scope `PRIVATE` whenever you filter.
576+
577+
**How a client uses the hint**
578+
579+
- The TTL is a ceiling, not a promise. A client may re-fetch sooner, and it drops the entry as
580+
soon as it receives the matching `notifications/tools/list_changed`,
581+
`notifications/prompts/list_changed`, or `notifications/resources/list_changed`.
582+
- Set a TTL only on listings the server can invalidate. With `listChanged` disabled, a client has
583+
no way to learn about a change before the TTL lapses.
584+
- A listing that spans several pages isn't cached: pairing a cached first page with a later page
585+
fetched fresh would mix two different views of the catalog.
586+
587+
**Caching a resource read**
588+
589+
`resources/read` carries its own hint, because how long a resource stays valid is a property of
590+
that resource rather than of the server. Set it on the result the read handler returns:
591+
592+
```java
593+
ReadResourceResult.builder(contents)
594+
.ttlMs(Duration.ofMinutes(1).toMillis())
595+
.cacheScope(CacheScope.PRIVATE)
596+
.build();
597+
```
598+
599+
A client drops the entry when it receives `notifications/resources/updated` for that URI. Results
600+
that set neither field default to a TTL of `0` with a `private` scope, which means no caching.
601+
540602
### Resource Specification
541603
542604
Specification of a resource with its handler function.
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
/*
2+
* Copyright 2026-2026 the original author or authors.
3+
*/
4+
5+
package io.modelcontextprotocol.client;
6+
7+
import java.util.Iterator;
8+
import java.util.LinkedHashMap;
9+
import java.util.Map;
10+
import java.util.function.Predicate;
11+
import java.util.function.Supplier;
12+
13+
import io.modelcontextprotocol.util.Assert;
14+
15+
/**
16+
* The {@link McpClientCacheStore} used when none is configured: a bounded, insertion
17+
* ordered map guarded by its own monitor.
18+
*
19+
* @author Sylwester Lachiewicz
20+
*/
21+
class InMemoryMcpClientCacheStore implements McpClientCacheStore {
22+
23+
static final int DEFAULT_MAX_ENTRIES = 512;
24+
25+
private record CacheEntry(Object value, long expiresAtMillis) {
26+
27+
boolean isExpired(long now) {
28+
return now >= this.expiresAtMillis;
29+
}
30+
31+
}
32+
33+
/**
34+
* Insertion-ordered, so that eviction drops the oldest entry.
35+
*/
36+
private final Map<McpClientCacheKey, CacheEntry> cache = new LinkedHashMap<>();
37+
38+
private final int maxEntries;
39+
40+
private final Supplier<Long> timeProvider;
41+
42+
InMemoryMcpClientCacheStore(int maxEntries, Supplier<Long> timeProvider) {
43+
Assert.isTrue(maxEntries > 0, "maxEntries must be positive");
44+
Assert.notNull(timeProvider, "timeProvider must not be null");
45+
this.maxEntries = maxEntries;
46+
this.timeProvider = timeProvider;
47+
}
48+
49+
@Override
50+
public Object get(McpClientCacheKey key) {
51+
synchronized (this.cache) {
52+
CacheEntry entry = this.cache.get(key);
53+
if (entry == null) {
54+
return null;
55+
}
56+
if (entry.isExpired(this.timeProvider.get())) {
57+
this.cache.remove(key);
58+
return null;
59+
}
60+
return entry.value();
61+
}
62+
}
63+
64+
@Override
65+
public void put(McpClientCacheKey key, Object value, long ttlMs) {
66+
long now = this.timeProvider.get();
67+
synchronized (this.cache) {
68+
// Re-insert so that insertion order stays age order.
69+
this.cache.remove(key);
70+
this.cache.put(key, new CacheEntry(value, saturatedAdd(now, ttlMs)));
71+
this.evict(now);
72+
}
73+
}
74+
75+
@Override
76+
public void removeIf(Predicate<McpClientCacheKey> matcher) {
77+
synchronized (this.cache) {
78+
this.cache.keySet().removeIf(matcher);
79+
}
80+
}
81+
82+
@Override
83+
public void clear() {
84+
synchronized (this.cache) {
85+
this.cache.clear();
86+
}
87+
}
88+
89+
int size() {
90+
synchronized (this.cache) {
91+
return this.cache.size();
92+
}
93+
}
94+
95+
private void evict(long now) {
96+
if (this.cache.size() <= this.maxEntries) {
97+
return;
98+
}
99+
this.cache.values().removeIf(entry -> entry.isExpired(now));
100+
Iterator<McpClientCacheKey> oldestFirst = this.cache.keySet().iterator();
101+
while (this.cache.size() > this.maxEntries && oldestFirst.hasNext()) {
102+
oldestFirst.next();
103+
oldestFirst.remove();
104+
}
105+
}
106+
107+
private static long saturatedAdd(long left, long right) {
108+
long sum = left + right;
109+
return ((left ^ sum) & (right ^ sum)) < 0 ? Long.MAX_VALUE : sum;
110+
}
111+
112+
}

0 commit comments

Comments
 (0)