Skip to content

Inflater#readpartial can return more than maxlen bytes #854

Description

@sebas2day

Response::Inflater#readpartial wraps connection.readpartial with zstream.inflate, but doesn't account for the fact that decompression can produce more (or possibly fewer?) output bytes than input bytes. The inflated result is returned directly, violating the maxlen contract of IO#readpartial:

  • Contains maxlen bytes from the stream, if available.

From https://docs.ruby-lang.org/en/3.3/IO.html#method-i-readpartial

This breaks streaming consumers that depend on bounded reads (e.g. Oj's sc_parse).

Possible fix would be to Buffer the inflate output inside Inflater#readpartial and slice the output to respect the requested size limit.

Current implementation:

# Read and inflate a chunk of the response body
#
# @example
# inflater.readpartial # => "decompressed data"
#
# @return [String]
# @raise [EOFError] when no more data left
# @api public
def readpartial(*)
chunk = @connection.readpartial(*)
zstream.inflate(chunk)
rescue EOFError
unless zstream.closed?
zstream.finished? ? zstream.finish : zstream.reset
zstream.close
end
raise
end

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions