Skip to content

Bounded the memory a graphql-transport-ws client can make the server hold - #631

Open
xperiandri wants to merge 4 commits into
devfrom
websocket-message-limits
Open

xperiandri wants to merge 4 commits into
devfrom
websocket-message-limits

Conversation

@xperiandri

Copy link
Copy Markdown
Collaborator

Problem

A graphql-transport-ws client could make the server hold memory without limit, per connection, in three ways:

  • Oversized messages. The reader assembled a message of any size.
  • Unbounded inbox. The inbox from the reader to the control loop had no bound, so messages sent faster than they were handled, for example while a query executed, queued without limit.
  • Unbounded outbound queue. It stayed unbounded, and the server never waited for the client to read. A client that sent pings and read nothing made the connection hold every pong: an independent review measured 312 MB in under half a second.

Changes

  • Message size (breaking).
    • GraphQLTransportWSOptions.MaxReceiveMessageSize is 4 MiB by default. A message over it closes the connection with 1009 (MessageTooBig) as soon as one byte beyond the limit is received, without buffering more of it.
    • A value that is not positive fails when the middleware is created, instead of aborting every accepted connection.
  • Inbox.
    • At most 16 client messages wait for the control loop, with BoundedChannelFullMode.Wait. While it is full, the reader does not read the socket, so TCP flow control slows the client down.
    • The messages wait as their bytes and are deserialized when handled, because a parsed message of dense JSON takes about ten times its size.
  • Outbound.
    • Messages are serialized when they are queued and counted in bytes.
    • Once more than 16 MiB would wait for a client that does not read, the pending send is cancelled, which aborts the socket.
    • A single larger message is still sent when nothing else waits.
  • No deadlocks. Producers never wait for room:
    • workers report their end through a queue of their own, which the control loop reads before every client message;
    • the inbox is completed once the control loop stops, which fails a write the reader waits on.

Tests

The tests are xUnit, in AspNetCore/WebSocketConnectionTests.fs:

  • messages at the limit, over it, and over the default limit, including connection_init;
  • the reader stopping while the inbox is full;
  • the connection ending while the reader waits;
  • the ids of subscriptions that ended while the inbox was full being free again;
  • a client that does not read being aborted;
  • the byte limit of the sender queue;
  • the startup check.

The new tests fail when the behaviour is removed. The WebSocket tests passed 5 of 5 repeated runs.

An independent review ran a real Kestrel host and covered:

  • limits exactly at and over the size, across frames and split characters;
  • backpressure and close codes;
  • host shutdown, client close and client abort with a full inbox.

It found no deadlock, hang or leaked worker.

Verification:

  • Unit tests: the whole project passes, 782 tests with 5 skipped.
  • FAKE: the full pipeline passes, including the integration tests.

Left for later

These predate this PR:

  • Ping handler: a custom ping handler and AsyncExecute get no cancellation token, so a hanging one delays shutdown.
  • Id reuse: a worker sends complete before it reports its end, so a client reusing an id at once could still, rarely, get 4409.

🤖 Generated with Claude Code

xperiandri and others added 2 commits October 3, 2026 04:32
A client could make the server hold an arbitrary amount of memory per
connection in two ways: the reader assembled a message of any size, and
the inbox from the reader to the control loop was unbounded, so messages
sent faster than the control loop handled them queued without limit.

- Added GraphQLTransportWSOptions.MaxReceiveMessageSize, 4 MiB by default
  (GraphQLOptionsDefaults.WebSocketMaxReceiveMessageSize). The reader
  requests at most one byte past the limit, and a message exceeding it
  closes the connection with 1009 (MessageTooBig) without the rest of it
  being read. Breaking change: the record has a new field.
- Bounded the inbox to 16 client messages with BoundedChannelFullMode.Wait:
  while it is full the reader does not read from the socket, so TCP flow
  control slows the client down.
- Kept the single-owner loops free of deadlocks: subscription workers
  report their end through a separate unbounded queue, since a worker must
  never wait and the shutdown awaits the workers without reading it; the
  control loop handles those ends before every client message. The inbox
  is completed once the control loop stops, which fails a write the reader
  is waiting on. The reader stops after the first protocol failure, since
  the connection is closed on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The messages waiting for a client are serialized when they are queued and
  counted in bytes: once more than 16 MiB would wait for a client that does
  not read them, the pending send is cancelled, which aborts the socket. A
  client sending pings without reading the pongs made the connection hold
  every pong, without limit. A single larger message is still sent when
  nothing else waits.
- The received messages wait in the inbox as their bytes and are
  deserialized by the control loop when it handles them: a parsed message
  of dense JSON took about ten times its size, so the inbox held far more
  than its documented bound.
- A MaxReceiveMessageSize that is not positive now fails when the
  middleware is created instead of aborting every accepted connection.
- Corrected the docs, which said the rest of a message over the size limit
  is never read: it is never buffered, but closing the socket may read and
  discard it.
- Tests for the ids of subscriptions that ended while the inbox was full,
  a client that does not read, the byte limit of the sender queue and the
  startup check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 10, 2026 09:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive concurrency, cancellation, and socket-abort behavior warrants final human review.

0 open findings

What changed in this PR

Bounds per-connection WebSocket memory usage through message-size limits and queue backpressure.

Changes:

  • Limits incoming messages to 4 MiB by default.
  • Bounds inbound messages and outbound serialized bytes.
  • Adds validation and comprehensive WebSocket tests.
File Description
RELEASE_NOTES.md Documents behavioral and breaking changes.
GraphQLOptions.fs Adds the receive-size option and default.
GraphQLWebsocketMiddleware.fs Validates the configured limit.
ServiceCollectionExtensions.fs Applies the default limit.
WebSocketMessaging.fs Introduces serialized queued messages.
WebSocketTransport.fs Enforces size and outbound byte limits.
WebSocketConnection.fs Adds bounded queues and backpressure handling.
WebSocketConnectionTests.fs Covers limits, shutdown, and slow clients.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

xperiandri and others added 2 commits October 10, 2026 11:49
…onnection tests

The tests name the close codes through `CustomWebSocketStatus` instead of
spelling the numbers out, and assert with `Assert.Equal` and
`Assert.InRange` instead of `Assert.True` on a comparison.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nnection tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Test Results

    9 files      9 suites   14m 41s ⏱️
  890 tests   885 ✅  5 💤 0 ❌
2 670 runs  2 655 ✅ 15 💤 0 ❌

Results for commit af1fc52.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants