Skip to content

fix: rebase lambda params after projection schema changes - #24406

Open
shinzoxD wants to merge 1 commit into
apache:mainfrom
shinzoxD:fix/lambda-variable-schema-rebase
Open

fix: rebase lambda params after projection schema changes#24406
shinzoxD wants to merge 1 commit into
apache:mainfrom
shinzoxD:fix/lambda-variable-schema-rebase

Conversation

@shinzoxD

Copy link
Copy Markdown

Which issue does this PR close?

Closes #24372

Rationale for this change

Physical LambdaVariable bindings are planned in a combined input_schema ++ params index space. LambdaExpr then built its outer-batch capture projection from every Column and LambdaVariable index in the body.

That is safe only while the input width stays the same as at plan time. After a later rewrite moves the expression across a schema boundary (collapsing consecutive ProjectionExecs, or pushing a join filter below NestedLoopJoinExec), a stale param index such as x@1 can become a real input column. Evaluation then fails:

Field of physical LambdaVariable with index 0 doesn't match batch field
during evaluation Field { x: nullable Int64 } != Field { column2: Int64 }

The two issue reproducers were:

SELECT array_transform(arr, x -> x)
FROM (
  SELECT arr
  FROM (VALUES ([1, 2], 7)) AS t(arr, padding)
) AS q;

and the array_filter join-filter pushdown case.

What changes are included in this PR?

  • Own lambda parameters are no longer added to the outer-batch capture projection.
  • They are rebound by name into the captures ++ used_params layout that LambdaArgument::new already builds.
  • Nested-lambda parameters stay shadowed and are remapped by the inner LambdaExpr.

Are these changes tested?

Are there any user-facing changes?

No public API change. Queries that previously failed after a projection/join-filter rewrite now return the correct result.

LambdaExpr mixed own-parameter indexes into the outer-batch capture
projection. After a projection rewrite grew the input schema, a stale
param index such as x@1 became a real input column and evaluation
failed with a Field mismatch.

Bind own params by name into the captures++params layout instead, and
leave nested-lambda params for the inner LambdaExpr to remap.

Closes apache#24372
Copilot AI lite review requested due to automatic review settings August 16, 2026 00:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-expr Changes to the physical-expr crates sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Projection rewrites do not rebase physical LambdaVariable bindings across schema boundaries

2 participants