Response::Inflater#readpartial wraps connection.readpartial with zstream.inflate, but doesn't account for the fact that decompression can produce more (or possibly fewer?) output bytes than input bytes. The inflated result is returned directly, violating the maxlen contract of IO#readpartial:
- Contains maxlen bytes from the stream, if available.
From https://docs.ruby-lang.org/en/3.3/IO.html#method-i-readpartial
This breaks streaming consumers that depend on bounded reads (e.g. Oj's sc_parse).
Possible fix would be to Buffer the inflate output inside Inflater#readpartial and slice the output to respect the requested size limit.
Current implementation:
|
# Read and inflate a chunk of the response body |
|
# |
|
# @example |
|
# inflater.readpartial # => "decompressed data" |
|
# |
|
# @return [String] |
|
# @raise [EOFError] when no more data left |
|
# @api public |
|
def readpartial(*) |
|
chunk = @connection.readpartial(*) |
|
zstream.inflate(chunk) |
|
rescue EOFError |
|
unless zstream.closed? |
|
zstream.finished? ? zstream.finish : zstream.reset |
|
zstream.close |
|
end |
|
|
|
raise |
|
end |
Response::Inflater#readpartialwrapsconnection.readpartialwithzstream.inflate, but doesn't account for the fact that decompression can produce more (or possibly fewer?) output bytes than input bytes. The inflated result is returned directly, violating themaxlencontract ofIO#readpartial:From https://docs.ruby-lang.org/en/3.3/IO.html#method-i-readpartial
This breaks streaming consumers that depend on bounded reads (e.g. Oj's
sc_parse).Possible fix would be to Buffer the inflate output inside
Inflater#readpartialand slice the output to respect the requested size limit.Current implementation:
http/lib/http/response/inflater.rb
Lines 30 to 48 in a210c0a