[connortsui20]

Summary

• Tracking Issue: Tracking Issue: Fix Extension Types #11114
• Progress towards Work with Arrow fields instead of Arrow data types #11286, because the export entry point now takes a target Field instead of a DataType.

Stacked on #11301.

An Arrow export could use two different ArrowSessions at once, and the API made that easy to do by accident. ArrowSession::execute_arrow took the session as its receiver and also an ExecutionCtx, which carries its own VortexSession and so its own ArrowSession. The top-level dispatch and the target checks used the receiver, but plugins such as Vector and Json export their storage through the session of the ctx. So ArrowSession::default().execute_arrow(array, None, &mut session.create_execution_ctx()) compiled, and which plugins ran depended on which part of the export was looking.

A session that arrow() created on first use also could not import what it exported. It was a bare default without the dtype registry, so it wrote an extension type without a plugin under its Vortex ID but read the same field back as plain storage. Only vortex_arrow::initialize fixed that, and it replaced the existing session, so calling it after vortex_uuid::initialize silently dropped the UUID plugins.

Changes

Exports now go through ExecuteArrowExt on the array, which takes its session from the ctx, so there is no second session to pass. Keeping session.arrow().execute_arrow(..) and using the session of the ctx inside was rejected, because the receiver would then be silently ignored. A runtime check that both are the same session was rejected too, because it only fails on the paths that tests happen to cover. The old methods are removed instead of deprecated, since a deprecated method that takes a session keeps the same trap open.

arrow() now builds the session from the dtype registry on first use, through the new SessionExt::get_or_insert_with, so every session imports what it exports. That leaves nothing for initialize to do, so it is removed instead of fixed.

Note that arrow() now also creates the DTypeSession, so .with::<DTypeSession>() after the first arrow() call panics, as it already did after initialize. New tests cover a UUID session that imports by Vortex ID without initialize, and a dtype that is registered after the first arrow() call.

API Changes

• ArrowSession::execute_arrow, ArrowSession::exporter, and ArrowExporter are removed. Use execute_arrow or execute_arrow_with_options from vortex_arrow::ExecuteArrowExt on the array instead.
• The deprecated ArrowArrayExecutor is removed.
• vortex_arrow::initialize is removed.
• ArrowSession has no Default or public constructor, so every ArrowSession comes from a VortexSession.
• vortex-session gains SessionExt::get_or_insert_with.

[codspeed]

Merging this PR will regress 1 benchmark

1 warning

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚡ 4 improved benchmarks
❌ 1 regressed benchmark
✅ 1911 untouched benchmarks
⏩ 48 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	CPU Simulation	ArrowExportVTable_to_arrow_field	45.9 µs	57.2 µs	-19.83%
⚡	CPU Simulation	string_export[view/view/flat/nonnull]	24.7 µs	21.4 µs	+15.5%
⚡	CPU Simulation	encode_primitives[f32, (2000, 2)]	211.2 µs	183.2 µs	+15.31%
⚡	Walltime	scalar_subtract_neon	13.4 µs	11.8 µs	+13.53%
⚡	CPU Simulation	decimal_export[canonical_i128_p38]	18.4 µs	16.6 µs	+10.88%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/arrow-session-from-ctx (6c7f48f) with ct/arrow-ext-export-guard (21414d4)}

Footnotes

1. 48 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[connortsui20]

I think the main files to review are:

• vortex-arrow/src/executor/mod.rs (though mostly just refactor, logic is the same)
• vortex-arrow/src/session.rs
• vortex-session/src/lib.rs
• vortex-session/src/session.rs

[connortsui20]

this is where the new session method is called