feat(filter): exact integer-float comparisons and Float32 double literals keep the scalar index

[jonasdedden]

Problem

Two numeric filter shapes lose the scalar index (issues #9317, #9318, reproduced on 13.0.0-beta.4):

• i > 1.5 on an integer column errors (could not convert Float64(1.5) to Int64); CAST(i AS double) > 1.5 plans but casts the column (CAST(i AS Float64) > ...), so BTREE/BITMAP never apply.
• x > CAST(0.5 AS double) and x < CAST('-inf' AS double) on a Float32 column cast the column to Float64 and skip the index, while x > 0.5 coerces the literal to Float32(0.5) and uses it.

Chosen rule

New lance-datafusion/src/numeric_coercion.rs, hooked into resolve_expr before the existing literal coercion:

• Integer vs float, exact. i > 1.5 → i > 1, i >= 1.5 → i > 1, i < 1.5 / i <= 1.5 → i <= 1 (floor; negatives round toward −∞), i = 1.5 → false and i != 1.5 → true, both null-preserving (col IS NULL AND NULL / col IS NOT NULL OR NULL) so NOT/AND/OR keep three-valued semantics. Integral floats in range become integer literals with the same operator; out-of-range clamps to constants. CAST(i AS double) unwraps to i first. BETWEEN uses ceil/floor with single-sided reduction and empty-range detection; IN drops never-matching elements and keeps typed nulls.
• Float32 vs double, exactly convertible only. A CAST-wrapped Float64 that round-trips through f32 (including ±inf, NaN with sign) lowers to Float32; CAST('-inf' AS double) folds (including string spellings Arrow rejects) to Float32(-inf). Anything inexact stays as a column cast — correct, just without the index. Bare x > 0.5 keeps its existing always-downcast path.

The integer rewrite is mathematical exactness, not DataFusion's lossy int-to-double coercion, mirroring DataFusion's unwrap-cast-in-comparison rule for integer casts. For small literals the two agree; for huge integers with a very close literal they can differ, and the exact form is what stays on the column.

Alternatives considered

• Coerce the float to int with as (truncation): wrong for negatives and fractional orderings.
• Downcast every typed double to Float32 (like bare literals): fast but changes results for inexact values such as 0.1 (Float32(0.1) vs Float64(0.1) differ on the boundary row).
• Keep erroring on i > 1.5 and document the CAST workaround: preserves status quo but permanently gives up the index for generated filters.

Compatibility / release

• No file-format change; filter-only rewrite in lance-datafusion.
• No new MSRV or dependency; half already used for f16.
• Target: next 13.0.0 beta (branched from main post-beta.4); test release 13.0.0-beta.4 reproduces both issues, Rust tests below prove the fix on main.
• Local polars-pylance impact (not changed here): integer-vs-float pushdown and Float32 double-literal pushdown can stop casting the column / stop declining and keep index-friendly SQL once this lands.

Tests

• cargo test -p lance-datafusion — 188 passed (7 new in numeric_coercion: floor/ceil, integral, out-of-range constants, exact f64→f32, special-string folding, plus parse_filter/optimize/physical checks that i > 1.5 → i > 1, CAST(i AS double) > 1.5 unwraps, null/NOT preserved, x > CAST(0.5 AS double) → Float32(0.5), inexact 0.1 stays Float64).
• cargo test -p lance --lib dataset::tests::dataset_index::test_numeric_coercion_uses_scalar_index — BTREE and BITMAP, 11 selective predicates assert ScalarIndexQuery plus indexed-vs-unindexed (use_scalar_index(false)) row equality with nulls, 3 always-true/false/empty predicates assert equality only, plus Float32(0.5) plan shape without CAST(x AS Float64).
• cargo clippy -p lance-datafusion --all-targets -- -D warnings clean; cargo fmt applied.
• Python baseline on pylance==13.0.0b4 reproduces the issue text verbatim (error + CAST(x AS Float64) plans).

Limitations

• IsDistinctFrom / IsNotDistinctFrom with mixed int-float are left to the existing path (still error); only the six standard comparisons rewrite.
• Complex right-hand sides (i = 1.5 + 0.5) still coerce sub-literals before folding and can error; simple and CAST-wrapped literals are covered.
• Float16 typed doubles are not rewritten (left as column casts, correct without index).

Fixes #9317, fixes #9318.

[lance-gatekeeper]

❌ Gate recommendation: request changes.

Keep the comparison-aware literal rewrites for bare integer/float comparisons and exactly representable Float64-to-Float32 literals, but only perform reductions that preserve the written filter under explicit casts and Lance/Arrow truth tables. The current implementation has three independent wrong-result paths at those semantic boundaries.

[lance-gatekeeper]

extract_column_cast removes every float-target cast without checking whether it is lossless, so an explicit cast can change query results rather than only the plan. On this head, with i = [16_777_216, 16_777_217], evaluating CAST(i AS float) = 16777216.0 as written yields [true, true] because both integers round to the same f32; after parse_filter it yields [true, false] because this path rewrites it to integer equality. The same extraction feeds the binary, BETWEEN, and IN rewrites. Please restrict unwrapping to casts proven lossless over the source domain, or leave the cast intact and forgo the index; the literal-side rewrite can still solve the reported no-cast case.

Reproducer

let parsed = planner
    .parse_filter("CAST(i AS float) = 16777216.0")
    .unwrap();
let written = Expr::Cast(Cast::new(Box::new(col("i")), DataType::Float32))
    .eq(lit(16_777_216.0_f64));
assert_eq!(
    evaluate(&planner, parsed, &batch),
    evaluate(&planner, written, &batch)
);
// head: [true, false]; written expression: [true, true]

Run as cargo test -p lance-datafusion --test gate_numeric_coercion explicit_float32_column_cast_keeps_its_rounding_semantics with an Int64 batch containing those two values.

[lance-gatekeeper]

Mapping every out-of-range companion bound to NULL breaks three-valued logic for NOT BETWEEN. With i = [0, NULL], the written filter i NOT BETWEEN CAST(NULL AS double) AND -1e100 evaluates to [true, NULL]: for the non-null row, the impossible upper comparison makes BETWEEN false, so NOT makes it true. This branch instead turns both bounds into typed nulls and produces [NULL, NULL], silently filtering out 0. Please preserve the always-false side when combining it with a null bound, or reduce the whole predicate (including negated) with the correct null behavior.

Reproducer

let parsed = planner
    .parse_filter("i NOT BETWEEN CAST(NULL AS double) AND -1e100")
    .unwrap();
let cast_i = Expr::Cast(Cast::new(Box::new(col("i")), DataType::Float64));
let written = cast_i
    .clone()
    .gt_eq(Expr::Literal(ScalarValue::Float64(None), None))
    .and(cast_i.lt_eq(lit(-1e100_f64)))
    .not();
assert_eq!(
    evaluate(&planner, parsed, &batch),
    evaluate(&planner, written, &batch)
);
// head: [NULL, NULL]; written expression: [true, NULL]

Run as cargo test -p lance-datafusion --test gate_numeric_coercion not_between_null_and_impossible_upper_bound_keeps_sql_null_logic with an Int64 batch containing Some(0) and None.

[lance-gatekeeper]

This NaN arm conflicts with the Arrow total-order comparisons used by Lance's float path and scalar indices: it forces every ordering comparison to false, regardless of NaN sign. On this head, with i = 1 and x = 1.0f32, i < CAST('nan' AS double) evaluates to false while the same predicate on x evaluates to true. The result therefore changes solely with the numeric column width. Please apply the existing Arrow/Lance NaN ordering when reducing integer comparisons (including BETWEEN bounds), or leave NaN predicates unreduced instead of returning a different answer.

Reproducer

let integer = planner
    .parse_filter("i < CAST('nan' AS double)")
    .unwrap();
let float32 = planner
    .parse_filter("x < CAST('nan' AS double)")
    .unwrap();
assert_eq!(
    evaluate(&planner, integer, &batch),
    evaluate(&planner, float32, &batch)
);
// head: integer [false], Float32 [true]

Run as cargo test -p lance-datafusion --test gate_numeric_coercion integer_and_float32_columns_agree_on_nan_ordering with a batch containing Int64(1) and Float32(1.0).