[ChristopherWirt]

Summary

• Tracking Issue: [Spark] Let VortexOutputWriter store footer metadata #11136

VortexWriter.Builder can store user metadata segments in the file footer, but VortexOutputWriter doesn't expose this. A caller writing files from Spark outside the commit protocol has to keep its own copy of the writer just to set footer keys.

This adds a VortexOutputWriter constructor that takes footer metadata.

Changes

• New constructor VortexOutputWriter(filePath, schema, options, writable, metadata), which passes metadata to VortexWriter.Builder.metadata.
• The existing constructor delegates with an empty map, so Spark's write path is unchanged.

Tests

• VortexOutputWriterTest.storesFooterMetadata writes a file with one metadata key and reads it back with NativeFiles.readMetadata.

On the fork (quantcast#77, same change), the Java changes are the same. develop's VortexOutputWriter differs around the builder call, so this branch reapplies the change by hand. Only git diff --check has been run on the port.

API Changes

• New public VortexOutputWriter constructor taking Map<String, byte[]> metadata. Existing callers are unaffected.

AI assistance

Disclosed per the AI policy: AI tools helped write and review this change. Claude Code prepared this upstream port, issue and PR description at my request. I am the responsible contributor.

🤖 Generated with Claude Code

[codspeed]

Merging this PR will degrade performance by 11.13%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

❌ 1 regressed benchmark
✅ 1916 untouched benchmarks
⏩ 47 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Walltime	lt_i64_nullable_avx2	9 µs	10.2 µs	-11.13%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing quantcast:cwirt/upstream-spark-writer-footer-metadata (0a2d05a) with develop (69ab721)}

Footnotes

1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩