Skip to content

Native TCP: CompressionMethod::ZSTD is not honored for server -> client blocks (network_compression_method never sent) #556

Description

@claude

Description

When a client is configured with ClientOptions().SetCompressionMethod(CompressionMethod::ZSTD), only the client → server direction actually uses ZSTD. The server → client direction still uses the server's default codec (LZ4).

The reason is that the native TCP query packet only carries a boolean "compression enabled" flag; the selected codec is never propagated to the server:

  • clickhouse/client.cpp:349-351compression_ is set to CompressionState::Enable for any method other than None; the specific method is discarded.
  • clickhouse/client.cpp:1087WireFormat::WriteUInt64(*output_, compression_); writes only that enable flag.
  • clickhouse/client.cpp:447CompressedOutput correctly uses options_.compression_method for the outgoing (client → server) data blocks.

To make the server compress its responses with ZSTD, the client must send the network_compression_method (and, for a configured level, network_zstd_compression_level) setting in the per-query settings section of the query packet (clickhouse/client.cpp:1067-1080). It never does.

The read path is not broken — CompressedInput::Decompress (clickhouse/base/compressed.cpp:105-131) accepts both LZ4 and ZSTD method bytes — so nothing fails; the selected codec is just silently not honored in one direction.

Additionally, there is no option to configure a ZSTD compression level at all: ClientOptions (clickhouse/client.h:107) only exposes compression_method.

This is the C++ equivalent of ClickHouse/clickhouse-go#1993.

ClickHouse server version

26.7.5.10 (verified against a running server over the native protocol on port 9000).

Reproduction

The server-side effective value of network_compression_method for the query is what determines how the server compresses the result blocks it sends back, so it can be observed directly from the client:

#include <clickhouse/client.h>
#include <iostream>

using namespace clickhouse;

int main() {
    Client client(ClientOptions()
                      .SetHost("localhost")
                      .SetPort(9000)
                      .SetCompressionMethod(CompressionMethod::ZSTD));

    client.Select(
        "SELECT name, value FROM system.settings WHERE name IN "
        "('network_compression_method', 'network_zstd_compression_level')",
        [](const Block& block) {
            for (size_t i = 0; i < block.GetRowCount(); ++i) {
                std::cout << (*block[0]->As<ColumnString>())[i] << " = "
                          << (*block[1]->As<ColumnString>())[i] << std::endl;
            }
        });
    return 0;
}

Expected output (codec honored in both directions):

network_compression_method = ZSTD
network_zstd_compression_level = <configured level>

Actual output:

network_compression_method = LZ4
network_zstd_compression_level = 1

i.e. the server compresses everything it sends back to this client with LZ4, even though ZSTD was requested.

Suggested fix

In Client::Impl::SendQuery (clickhouse/client.cpp, per-query settings block around lines 1067-1080), when options_.compression_method != CompressionMethod::None and the setting is not already present in query.GetQuerySettings(), send:

  • network_compression_method = "ZSTD" / "LZ4" matching options_.compression_method
  • network_zstd_compression_level = the configured level, if a new ClientOptions field for it is added (e.g. SetCompressionLevel), when the method is ZSTD

Link

Original report against the Go client: ClickHouse/clickhouse-go#1993

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions