[CORE][VL] Add columnar EmptyRelationExec offload to Velox backend - #12766
Open
minni31 wants to merge 5 commits into
Open
[CORE][VL] Add columnar EmptyRelationExec offload to Velox backend#12766minni31 wants to merge 5 commits into
minni31 wants to merge 5 commits into
Conversation
Offload EmptyRelationExec (a leaf node AQE's Propagate Empty Relations optimization creates on Spark 4.0+ when it proves a subtree produces no output) to a native EmptyRelationExecTransformer. The transformer produces an empty RDD[ColumnarBatch] so surrounding columnar operators no longer need to be wrapped in ColumnarToRow / RowToColumnar transitions around the empty relation. EmptyRelationExec only exists on Spark 4.0+ (SPARK-47217), so the node is never referenced from version-agnostic modules: detection goes through the new SparkShims.isEmptyRelationExec, overridden only in the Spark 4.0 and 4.1 shims and defaulting to false elsewhere. The shared OffloadOthers rule and the SparkPlanExecApi trait therefore compile unchanged against Spark 3.3-3.5. The offload is gated by spark.gluten.sql.columnar.emptyRelation (default true). The Velox backend implements isSupportEmptyRelationExec; other backends inherit the trait default and keep vanilla execution. Adds VeloxEmptyRelationSuite (empty-result correctness on all supported Spark versions, plus plan-shape and config-gate assertions gated to Spark 4.0+) and re-enables the upstream SPARK-35585 AQE test on Spark 4.0/4.1 with a Gluten-aware assertion that accepts either EmptyRelationExec or the transformer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 459758d8-f68d-4c1f-8069-b4df4eca2175
|
Run Gluten Clickhouse CI on x86 |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a Spark 4.0+ shim-based detection path and a new columnar leaf transformer for EmptyRelationExec, allowing Gluten (Velox backend) to avoid unnecessary ColumnarToRow / RowToColumnar transitions around proven-empty subtrees produced by AQE’s “Propagate Empty Relations”.
Changes:
- Add
SparkShims.isEmptyRelationExec(defaultfalse) with Spark 4.0/4.1 overrides, and wireEmptyRelationExecTransformerintoOffloadOthers. - Introduce
EmptyRelationExecTransformer(emptyRDD[ColumnarBatch]) and a new dynamic configspark.gluten.sql.columnar.emptyRelation(defaulttrue). - Add/adjust Spark 4.0/4.1 Velox tests to validate plan shape + correctness and handle SPARK-35585 expectations.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| shims/spark41/src/main/scala/org/apache/gluten/sql/shims/spark41/Spark41Shims.scala | Adds Spark 4.1 shim detection for EmptyRelationExec. |
| shims/spark40/src/main/scala/org/apache/gluten/sql/shims/spark40/Spark40Shims.scala | Adds Spark 4.0 shim detection for EmptyRelationExec. |
| shims/common/src/main/scala/org/apache/gluten/sql/shims/SparkShims.scala | Adds version-agnostic isEmptyRelationExec API with safe default. |
| gluten-substrait/src/main/scala/org/apache/spark/sql/execution/EmptyRelationExecTransformer.scala | New columnar leaf transformer producing an empty RDD[ColumnarBatch]. |
| gluten-substrait/src/main/scala/org/apache/gluten/extension/columnar/offload/OffloadSingleNodeRules.scala | Adds EmptyRelationExec → transformer offload case guarded by shims + backend support. |
| gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala | Adds new dynamic config accessor + entry for empty relation offload. |
| gluten-substrait/src/main/scala/org/apache/gluten/backendsapi/SparkPlanExecApi.scala | Adds backend API hooks to support and create the transformer. |
| backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxSparkPlanExecApi.scala | Implements support gate + transformer construction for Velox backend. |
| docs/Configuration.md | Documents the new spark.gluten.sql.columnar.emptyRelation config. |
| backends-velox/src/test/scala/org/apache/gluten/execution/VeloxEmptyRelationSuite.scala | New suite validating correctness + Spark 4.0+ plan-shape behavior and config disablement. |
| gluten-ut/spark41/src/test/scala/org/apache/spark/sql/execution/adaptive/velox/VeloxAdaptiveQueryExecSuite.scala | Adds Gluten-specific SPARK-35585 assertion accepting either raw or transformer node. |
| gluten-ut/spark41/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala | Adjusts included AQE test selection to account for new Gluten-specific coverage. |
| gluten-ut/spark40/src/test/scala/org/apache/spark/sql/execution/adaptive/velox/VeloxAdaptiveQueryExecSuite.scala | Same SPARK-35585 Gluten-specific assertion for Spark 4.0 test module. |
| gluten-ut/spark40/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala | Adjusts included AQE test selection for Spark 4.0 module. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
… docs The EmptyRelationExecTransformer is a JVM-side columnar leaf that returns an empty RDD[ColumnarBatch]; it does not invoke native execution. Reword the config doc string and scaladocs accordingly, and regenerate the Configuration.md row so it matches the config doc() string (fixes AllGlutenConfiguration check).
|
Run Gluten Clickhouse CI on x86 |
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.
Suppressed comments (1)
gluten-substrait/src/main/scala/org/apache/spark/sql/execution/EmptyRelationExecTransformer.scala:52
withNewChildrenInternalcurrently ignoresnewChildrenunconditionally. For a leaf node, it’s safer to enforce the invariant that no children are provided (otherwise a caller attempting a tree rewrite could silently fail to apply updates). Add arequire(newChildren.isEmpty, ...)guard (or otherwise validate) before returningthis.
override protected def withNewChildrenInternal(
newChildren: IndexedSeq[SparkPlan]): SparkPlan = this
SharedSparkSession mixes in Spark's SQLTestUtilsBase, which overrides withSQLConf to return Unit rather than the block value. Capturing the collected rows via the block return type therefore inferred Unit and failed to compile (isEmpty / checkAnswer on Unit). Assign the vanilla result to a var inside the block instead.
|
Run Gluten Clickhouse CI on x86 |
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.
Suppressed comments (1)
gluten-substrait/src/main/scala/org/apache/spark/sql/execution/EmptyRelationExecTransformer.scala:53
- EmptyRelationExecTransformer is intended as a semantic replacement for Spark's EmptyRelationExec, but
doExecute()currently throws. If this node ever ends up on a row-based path (e.g., under a non-offloaded parent, or if a transition is missed), the query would fail instead of producing an empty result. Consider returning an emptyRDD[InternalRow]here. Also, since this is a leaf exec, extendingLeafExecNodelets Spark handlechildren/withNewChildrenInternalinvariants and avoids silently ignoring non-emptynewChildren.
override protected def doExecute(): RDD[InternalRow] =
throw new UnsupportedOperationException(
"EmptyRelationExecTransformer does not support row execution.")
override protected def doExecuteColumnar(): RDD[ColumnarBatch] =
sparkContext.emptyRDD[ColumnarBatch]
override def children: Seq[SparkPlan] = Seq.empty
override protected def withNewChildrenInternal(
newChildren: IndexedSeq[SparkPlan]): SparkPlan = this
…RelationExec The plan-shape tests queried 'WHERE 1 = 0', which the logical optimizer folds to an empty LocalRelation (physical LocalTableScan) before physical planning, so no EmptyRelationExec is ever produced and the offload assertion failed (0 transformers; plan was LocalTableScan <empty>). EmptyRelationExec is an AQE-runtime node created by Propagate Empty Relations when a materialized query stage is empty at runtime. Drive it through an AQE INTERSECT whose left side (l_orderkey < 0) is empty only at runtime, mirroring the upstream SPARK-35585 trigger, and traverse with collectWithSubqueries. The config-gate test now exercises the same real EmptyRelationExec instead of passing vacuously.
|
Run Gluten Clickhouse CI on x86 |
Extend LeafExecNode with ValidatablePlan (matching ColumnarRangeBaseExec) instead of manually overriding children and withNewChildrenInternal. LeafExecNode/LeafLike supplies an empty children list and a withNewChildrenInternal that enforces the no-children invariant, rather than silently ignoring newChildren and returning this.
|
Run Gluten Clickhouse CI on x86 |
minni31
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
This PR offloads
EmptyRelationExecto the Velox backend so that empty relations are executed columnarly instead of forcing a fallback to vanilla row execution.EmptyRelationExecis a leaf node that AQE's Propagate Empty Relations optimization creates on Spark 4.0+ when a materialized query stage turns out to be empty at runtime (e.g. anINTERSECTor join whose input is empty only at runtime, or an aggregation over an empty runtime stage). A statically-empty predicate such asWHERE 1 = 0does not reach this path — the logical optimizer folds it to an emptyLocalRelation(aLocalTableScan) before physical planning. Because its defaultsupportsColumnarisfalse, Gluten currently wraps it inColumnarToRow/RowToColumnartransitions even though it returns zero rows. This PR addsEmptyRelationExecTransformer, a columnar leaf that returns an emptyRDD[ColumnarBatch], eliminating those transitions.EmptyRelationExeconly exists on Spark 4.0+ (SPARK-47217), so the node is never referenced from version-agnostic modules. Detection is routed through a newSparkShims.isEmptyRelationExec, which defaults tofalseand is overridden only in the Spark 4.0 and 4.1 shims. The sharedOffloadOthersrule and theSparkPlanExecApitrait therefore continue to compile unchanged against Spark 3.3–3.5.The offload is gated by a new config
spark.gluten.sql.columnar.emptyRelation(defaulttrue). The Velox backend implementsisSupportEmptyRelationExec; other backends inherit the trait default and keep vanilla execution.How was this patch tested?
VeloxEmptyRelationSuite: empty-result correctness across various schemas, emptyUNION ALL, AQE propagation through joins/aggregations, and side-by-side parity with vanilla Spark (asserted on all supported Spark versions). Plan-shape assertions (transformer present, no residualEmptyRelationExec) and the config-disabled negative case are gated to Spark 4.0+, where the node exists.SPARK-35585AQE test on Spark 4.0/4.1 with a Gluten-aware assertion that accepts eitherEmptyRelationExecorEmptyRelationExecTransformer.Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot (Claude Opus 4.8)