Skip to content

environmentd: bound HTTP and SUBSCRIBE output buffering to prevent OOM (SQL-428, SQL-423) - #37905

Open
mtabebe wants to merge 2 commits into
MaterializeInc:mainfrom
mtabebe:ma/sql-428-oom-buffered-output
Open

environmentd: bound HTTP and SUBSCRIBE output buffering to prevent OOM (SQL-428, SQL-423)#37905
mtabebe wants to merge 2 commits into
MaterializeInc:mainfrom
mtabebe:ma/sql-428-oom-buffered-output

Conversation

@mtabebe

@mtabebe mtabebe commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Problem:

A single query can drive environmentd to OOM, even when the client read as
fast as it could, because two paths buffered unbounded output in the shared
process.

  • HTTP SELECT. The peek path already streams results out of the replica in
    bounded batches, but the HTTP SQL layer re-buffered every batch into a
    serde_json::Value tree roughly 10-15x the wire size. A large SELECT over
    /api/ws or /api/sql grew memory without limit. The /api/sql size guard
    was a stale TODO and was never enforced.
  • SUBSCRIBE / COPY-TO. The replica -> controller -> coordinator response path
    is unbounded at every hop, and the subscribe producer runs on the
    non-blockable coordinator main loop, so it cannot apply backpressure to a slow
    client. One slow or stopped SUBSCRIBE consumer could make the coordinator
    buffer the entire subscribe output in memory.

Solution:

HTTP SQL layer (SQL-428).

  • WebSocket /api/ws streams rows straight through. SELECT now flows through a
    new StatementResult::Rows variant whose sender emits Rows(desc) plus
    per-row Row messages, flushing between stash batches so a slow client applies
    backpressure and only one batch is resident. max_result_size is enforced
    incrementally. Large results over WebSocket now get the same bounded memory
    profile as pgwire SELECT.
  • Plain JSON /api/sql is a single buffered document. Rows are accumulated as
    pre-serialized RawValues instead of a Value tree, and max_result_size is
    enforced on the actual serialized byte count. An oversized result fails fast
    with ResultSize.
  • MCP and prometheus consumers of SqlResult::Rows parse each RawValue on
    demand.

SUBSCRIBE / COPY-TO buffer (SQL-423).
Bound the terminal buffer and retire the sink when it overflows. ActiveSubscribe
carries a shared byte counter and a per-sink budget. Each send adds the batch's
byte size and a receiver wrapper subtracts it as the client writer drains, so the
counter tracks the currently buffered depth. After each response the coordinator retires the subscribe if the depth exceeds the budget, surfacing AdapterError::SubscribeFellBehind
with a hint to use a non-buffering client or COPY (SUBSCRIBE ...) TO STDOUT.

The budget is a new dyncfg subscribe_max_buffered_bytes (default 128 MiB).
COPY (SUBSCRIBE ...) TO STDOUT shares the channel and is covered too.

Testing:

New integration tests in src/environmentd/tests/server.rs:

  • test_ws_select_streams_rows: a SELECT over /api/ws emits one Row message
    per row before CommandComplete, confirming the non-buffering path.
  • test_http_sql_result_size_limit: with max_result_size lowered to 1 MB, an
    oversized /api/sql result fails with a result-size error while a small one succeeds.
  • test_subscribe_buffer_bound: with subscribe_max_buffered_bytes set to 1 KiB,
    a COPY-wrapped SUBSCRIBE whose client never reads surfaces the "fell behind"
    error instead of growing without bound.

@mtabebe
mtabebe force-pushed the ma/sql-428-oom-buffered-output branch 4 times, most recently from 4cc5337 to 13ce959 Compare July 28, 2026 18:18
mtabebe added 2 commits July 28, 2026 16:24
…output (SQL-428)

Problem:
A single `SELECT` over the HTTP endpoints could make environmentd buffer the
entire result in memory and OOM, even when the client reads as fast as possible.
The peek path already streams results out of the replica in bounded batches, so
the growth was entirely the HTTP layer re-buffering those batches as an inflated
`serde_json::Value` tree (~10-15x the wire size).

Solution:
- WebSocket: stream rows straight through. `SELECT` now flows through
a new `StatementResult::Rows` variant whose sender streams `Rows(desc)` +
per-row `Row` messages, flushing between stash batches for backpressure
and enforcing `max_result_size` incrementally. Only one batch is resident, so
large results over WebSocket get the same bounded memory profile as pgwire
`SELECT`. The row descriptor is sent lazily so a query that errors before
producing rows still emits only an `Error`.

- Plain JSON: the response is one buffered document, so it stays buffered.
But its footprint is smaller and its cap is now accurate. Rows are
accumulated as pre-serialized `Vec<Box<serde_json::value::RawValue>>` instead
of a `Value` tree, and `max_result_size` is enforced on the real serialized
byte count. A genuinely oversized result fails fast with `ResultSize` instead
of OOMing.

The MCP and prometheus consumers of `SqlResult::Rows` parse each `RawValue` on
demand. A streamed `SELECT` that errors mid-peek now reports `has_rows: true`
over WebSocket (it commits to being row-returning before streaming), matching
the `SUBSCRIBE` arm and pgwire.
Problem:

A single slow or stopped SUBSCRIBE client could make environmentd buffer the
whole subscribe output in memory and OOM. The replica -> controller ->
coordinator response path is unbounded at every hop, and the producer runs on
the non-blockable coordinator main loop, so it cannot apply backpressure to a
slow client.

Solution:
Bound the terminal buffer and retire the sink.

`ActiveSubscribe` gains a shared byte counter and a per-sink budget.
Each `send()` adds the batch's byte size and carries it on the channel.
A thin receiver wrapper, installed where the subscribe `rx` is built, subtracts
it as the client writer drains each batch, so the counter tracks the currently
buffered depth.

After each `process_response`, the coordinator retires the subscribe if the
buffered depth exceeds the budget, surfacing a new
`AdapterError::SubscribeFellBehind` ("SUBSCRIBE fell behind ...") with a hint to
use a non-buffering client or `COPY (SUBSCRIBE ...) TO STDOUT`.

The budget is a new dyncfg `subscribe_max_buffered_bytes`, default 128 MiB.
@mtabebe
mtabebe force-pushed the ma/sql-428-oom-buffered-output branch from 13ce959 to 5854351 Compare July 28, 2026 20:24
@mtabebe
mtabebe marked this pull request as ready for review July 28, 2026 21:09
@mtabebe
mtabebe requested review from a team as code owners July 28, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant