Epic: Row-oriented scalar functions

[connortsui20]

Right now, functions whose natural implementation operates on one row still have to own much of the machinery for executing that operation over columnar Vortex arrays.

This Epic tracks a RowFn definition that separates those concerns. A scalar function implementor can choose typed row elements and define the computation/operation, while the framework handles batch decoding, dtype validation, constants, null rows, output construction, and more.

Status

In progress.

The private executor tracked by #9130 is complete. The author-facing API tracked by #9129 remains experimental.

The executor, generic filter-and-scatter fallback, primitive arithmetic, and spatial users are merged. Current work includes the tensor migrations in #9347 and #9348, primitive comparisons in #9703, and UTF-8 elements in #9715. #9645 explores a different valid-row fallback and does not block the completed executor.

RowFn adoption is automatic through a blanket ScalarFnVTable implementation. A type that needs custom vtable hooks can keep them on a separate public type and delegate to a private RowFn kernel through row_fn_return_dtype and execute_rows. Every sealed element-tuple arity supports indexed traversal, while unary and binary kernels keep specialized sources.

InputElement and OutputSink are unsafe to implement. InitializedElement::write is unsafe because its token is not lifetime-branded to one callback row. OutputSink::Params describes physical storage, while RowVisitor::with_output_dtype declares the logical output dtype.

Performance decisions remain specific to each migration. Primitive arithmetic uses RowFn where matched x86 measurements and generated code support it. Primitive comparisons remain open in #9703. A function can keep a columnar path when its row kernel produces worse native code.

Subissues

• Tracking Issue: Define the RowFn API #9129
• Tracking Issue: Execute RowFn over Vortex arrays #9130

Goal

A scalar function whose natural implementation operates on one typed row should be able to define that operation without also implementing the surrounding array machinery.

The goals are:

• Define a stable RowFn API for choosing typed inputs, optionally preparing batch state, computing rows, and building outputs.
• Share validation, batch decoding, constant handling, null propagation, output allocation, and validity handling across row functions.
• Allow crates such as vortex-tensor and vortex-spatial to add row representations without changing vortex-array.
• Preserve encoding-aware shortcuts for functions that have a better answer for a specific encoding.
• Avoid a performance penalty for already-efficient row kernels, while making commonly missed optimizations reusable.
• Support optional row outputs for strict functions that may return null from otherwise valid inputs.

Note that we will focus solely on strict functions. as the semantics around non-strict functions are complicated enough that it's probably not worth extending this already-somewhat-complicated API further.

RowFn is also not intended for columnar or zero-copy kernels (not, list_length), kernels with state shared across rows (like), or heterogeneous variadic kernels.

Motivation

We have a good number of scalar function implementations in the Vortex codebase, and because it is simply a trait implementation of ScalarFnVTable, anyone can add their own scalar function.

However, writing a good implementation is not exactly trivial. Suppose we want to add a scalar function Hypot to Vortex that is similar to hypot. This simply gets distance from the origin to the point (x, y) via sqrt(x^2 + y^2).

Even though this is arguably a "basic" scalar function, there are MANY things that the implementor needs to worry about w.r.t. correctness and performance.

• Depending on what the implementor wants, they might need to implement this for when x and y are arbitrary floating-point types such as f16, f32, and f64. Maybe they want to accept integers without an explicit cast!
• How should they deal with null values / validity? What should the result of hypot(null, 5.0) be?
• Depending on how they want their null semantics, they might want to do a bunch of stuff with validity up front before moving onto the compute (perhaps intersect the validity of the 2 inputs).
• This sqrt(x^2 + y^2) operation is small enough that the implementor would want to ensure auto-vectorization can happen, so they need to make sure there is no branching in hot loops.
• How do they pre-allocate memory correctly?
• If one or both columns of x and y are ConstantArray, then they definitely want to precompute x^2 and y^2 rather than doing it n times.

Note that in practice we would probably want to decompose hypot() into an expression that does sqrt(x^2 + y^2) as a tree of numeric expressions for better optimizations, but hopefully you get the idea that there are several things that many scalar function implementations have to worry about, even if the function is "simple".

More often than not, the scalar function implementor is only going to worry about a subset of these and leave potential performance optimizations on the table.

So this begs the question: Can scalar functions be defined in terms of their natural operation on a single typed row, without giving up Vortex’s array semantics, extension types, encoding-aware shortcuts, or performance?

RowFn rationale

Every row-oriented scalar function execution follows roughly the same execution pipeline:

1. Validate the input types.
2. Handle degenerate inputs, such as all null or entirely constant arguments.
3. Decode each input column once.
4. Compute anything that is constant/static for the batch.
5. Read and compute one row at a time (hopefully in a vectorized manner).
6. Build the output and apply its validity.

Without a shared definition, every scalar function has to assemble this pipeline itself.

RowFn makes the owner of each step explicit:

1. RowFn::ARG_NAMES declares the arity, and InputElement validates each element dtype.
2. Null constants and entirely constant inputs are handled automatically.
3. InputElement::decode prepares each varying column once. InputElement::decode_constant extracts one constant representation without constructing a one-row column.
4. The prepare closure on the visit_prepared* methods computes state derived from constant arguments once per batch.
5. The sealed tuple adapter reads each row and the function's closure performs the actual computation.
6. An OutputElement or OutputSink builds the result, after which the framework applies the input validity.

Every RowFn receives the standard ScalarFnVTable implementation automatically. A function that needs custom vtable hooks can keep them on a separate public type and delegate row planning and execution to a private RowFn kernel.

Prior Art

Velox's simple function framework lifts a row-at-a-time C++ call method into vectorized execution with the same division of labor: the author writes one typed row, and the framework owns decoding, nulls, constants, and output construction. The main differences: Velox binds types at registration and resolves by signature, while RowFn dispatches on runtime dtypes inside the function. Velox covers a broader surface (strings, nested types, variadics, per-row nullable outputs), while RowFn is stricter, keeps its row loops branch-free for vectorization, and lets encodings reach the function through reduce_encoded. Both frameworks keep columnar fallbacks where the row form loses, notably primitive comparisons.

A detailed comparison with side-by-side examples lives in research/velox-row-fn-comparison on ct/velox-row-fn-cmp.

Unresolved questions

• Are nullable outputs from otherwise valid inputs required before the first version is complete? This is needed for strict but non-total functions such as list_sum and variant_get.
• InputElement, OutputElement, and OutputSink are supported downstream extension points. InputElement is unsafe to implement, and OutputSink::finish is unsafe to call. ElementTuple and SinkResult remain sealed executor mechanics.

Implementation history

• Harden indexed lane sources #9358, Define RowFn and RowVisitor #9386, and Implement RowFn row execution #9353 add the lane sources, the author-facing contracts, and shared row execution.
• Simplify RowFn execution contracts #9496 and Replace ScalarFnVTable::is_fallible with is_infallible #9511 establish the positive INFALLIBLE contracts used by the executor.
• Implement RowFn batch execution #9450 adds batch execution and strategy selection. Execute owned RowFn outputs over valid rows #9500 adds owned valid-row execution, and Support nullary RowFn execution #9469 adds nullary execution.
• Use dense retry for nullable row functions #9517 adds dense retry and lands the primitive arithmetic work started in Execute primitive numeric operators with RowFn #9345.
• Carry partial RowFn validity with MaskValues #9534 and Extend MaskValuesRef to filter execution #9542 carry materialized partial validity through valid-row execution.
• Add filter-and-scatter RowFn execution #9521 adds the generic filter-and-scatter fallback.
• Let a row dispatch declare a runtime output dtype #9581 and Add physical parameters to OutputSink #9585 separate logical output dtypes from physical sink parameters.
• refactor(vortex-spatial): execute distance with RowFn #9616 and refactor(vortex-spatial): execute geo kernels with RowFn #9623 migrate the spatial functions and validate downstream input and output types.
• Pack Boolean RowFn output directly #9620 and Pack Boolean RowFn output directly #9626 add direct packed Boolean output.
• Map constant RowFn inputs directly #9628, Use indexed sources for fallible constant rows #9629, and Decode RowFn constants directly #9680 specialize constant input traversal and decoding.
• Execute tensor L2 norm with RowFn #9347 and Execute tensor product functions with RowFn #9348 contain the open tensor migrations and the proposed encoding-aware hook.
• Compare primitive values with RowFn #9703 replaces the earlier primitive comparison migration. Add Utf8 types for RowFn  #9715 adds UTF-8 input and output types.
• Automate RowFn benchmark comparisons #9351 prototyped benchmark comparison tooling.

[connortsui20]

@mhk197 gave me the idea that this could potentially work for gpu execution too 👀