Skip to content

[AURON #2193] Implement native support for inner residual join conditions on SMJ/SHJ#2197

Draft
weimingdiit wants to merge 3 commits intoapache:masterfrom
weimingdiit:feat/native_support_inner_join_conditions
Draft

[AURON #2193] Implement native support for inner residual join conditions on SMJ/SHJ#2197
weimingdiit wants to merge 3 commits intoapache:masterfrom
weimingdiit:feat/native_support_inner_join_conditions

Conversation

@weimingdiit
Copy link
Copy Markdown
Contributor

@weimingdiit weimingdiit commented Apr 13, 2026

Which issue does this PR close?

Closes #2193

Rationale for this change

Auron currently only converts SMJ/SHJ joins when the join condition is empty. As a result, inner joins that contain both equi-join keys and a residual predicate fall back to Spark even though the equi-join part is already native-compatible.

The native join plan only models equi-join keys today, so this change keeps the native join focused on the equi-join portion and evaluates the residual predicate with a native filter above the join output.

What changes are included in this PR?

  • Allow native conversion of SortMergeJoinExec and ShuffledHashJoinExec when an InnerLike join has a residual condition.
  • Keep native join conversion based on equi-join keys only.
  • Apply the residual predicate as a native filter on top of the native join output.
  • Continue to reject residual join conditions for non-inner join types.
  • Add query tests for:
    • native SMJ with an inner residual condition
    • native SHJ with an inner residual condition in force-SHJ mode
  • Add a small test helper for Auron configs that are read from SparkEnv/SparkContext.

Are there any user-facing changes?

Yes. Inner joins with equi-join keys plus a residual predicate can now remain on the native SMJ/SHJ path instead of falling back entirely to Spark.

How was this patch tested?

CI.

@github-actions github-actions bot added the spark label Apr 13, 2026
Signed-off-by: weimingdiit <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement native support for inner residual join conditions on SMJ/SHJ

1 participant