What would you like to happen?
Problem
ClickHouseIO has no notion of ClickHouse's fixed-point Decimal family — Decimal(P, S),
Decimal(P), bare Decimal, and the width aliases Decimal32(S), Decimal64(S),
Decimal128(S), Decimal256(S). The column-type parser rejects these type strings, so
ClickHouseIO.getTableSchema() throws on the DESCRIBE TABLE output and pipelines cannot
write to any table containing a Decimal column — a blocker for monetary and other
fixed-point data.
Proposed solution
-
Schema model
- Add
TypeName.DECIMAL.
- Extend
ColumnType with a scale field alongside the existing precision field
(introduced for DateTime64), plus a validated factory
ColumnType.decimal(int precision, int scale) (P in [1, 76], S in [0, P]).
-
Parser
- Grammar rules for
Decimal[(P[, S])] (bare Decimal = Decimal(10, 0),
Decimal(P) = Decimal(P, 0), matching ClickHouse) and the width aliases
Decimal32/64/128/256(S) (precisions 9/18/38/76), reachable through
Nullable(...) and Array(...).
-
Beam field type mapping
Decimal(P, S) maps to the FixedPrecisionNumeric.of(P, S) logical type
(base type DECIMAL, values are BigDecimal), preserving the declared
precision/scale through getEquivalentSchema — the same logical type JdbcIO
uses for NUMERIC columns.
-
Writer
- Serialize as ClickHouse's RowBinary representation: the unscaled value as a
little-endian signed integer whose width (32/64/128/256 bits) is chosen from the
declared precision, exactly per ClickHouse's Decimal(P, S) → Decimal32/64/128/256
synonym rule.
-
Defaults
- Support
DEFAULT <literal> on Decimal columns in
ColumnType.parseDefaultExpression — without it, schema fetch fails for any table
with a defaulted Decimal column.
-
Tests
- Parser/mapping unit tests, byte-level writer tests, and round-trip tests against the
ClickHouse test container for all four width buckets, Nullable(Decimal),
Array(Decimal), DEFAULT handling, and getTableSchema on the server's canonicalized
DESCRIBE output.
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components
What would you like to happen?
Problem
ClickHouseIO has no notion of ClickHouse's fixed-point
Decimalfamily —Decimal(P, S),Decimal(P), bareDecimal, and the width aliasesDecimal32(S),Decimal64(S),Decimal128(S),Decimal256(S). The column-type parser rejects these type strings, soClickHouseIO.getTableSchema()throws on theDESCRIBE TABLEoutput and pipelines cannotwrite to any table containing a Decimal column — a blocker for monetary and other
fixed-point data.
Proposed solution
Schema model
TypeName.DECIMAL.ColumnTypewith ascalefield alongside the existingprecisionfield(introduced for
DateTime64), plus a validated factoryColumnType.decimal(int precision, int scale)(P in [1, 76], S in [0, P]).Parser
Decimal[(P[, S])](bareDecimal=Decimal(10, 0),Decimal(P)=Decimal(P, 0), matching ClickHouse) and the width aliasesDecimal32/64/128/256(S)(precisions 9/18/38/76), reachable throughNullable(...)andArray(...).Beam field type mapping
Decimal(P, S)maps to theFixedPrecisionNumeric.of(P, S)logical type(base type
DECIMAL, values areBigDecimal), preserving the declaredprecision/scale through
getEquivalentSchema— the same logical type JdbcIOuses for NUMERIC columns.
Writer
little-endian signed integer whose width (32/64/128/256 bits) is chosen from the
declared precision, exactly per ClickHouse's
Decimal(P, S)→Decimal32/64/128/256synonym rule.
Defaults
DEFAULT <literal>on Decimal columns inColumnType.parseDefaultExpression— without it, schema fetch fails for any tablewith a defaulted Decimal column.
Tests
ClickHouse test container for all four width buckets,
Nullable(Decimal),Array(Decimal), DEFAULT handling, andgetTableSchemaon the server's canonicalizedDESCRIBEoutput.Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components