[SPARK-58756][BUILD][CORE][4.3] Revert OIDC credential propagation from branch-4.3 (moved to 4.4.0) - #57985
Closed
sarutak wants to merge 9 commits into
Closed
[SPARK-58756][BUILD][CORE][4.3] Revert OIDC credential propagation from branch-4.3 (moved to 4.4.0)#57985sarutak wants to merge 9 commits into
branch-4.3 (moved to 4.4.0)#57985sarutak wants to merge 9 commits into
Conversation
…ct rotation by content" This reverts commit b7d480b.
…ialProvider` SPI" This reverts commit 1623ebd.
…module with CI/release integration" This reverts commit e9b8b1b.
This reverts commit 057b607.
…spark.security.credentials` to `spark.security.oidc`" This reverts commit 72c284f.
…Ingestor implementation" This reverts commit b02704a.
…afety docs in `CredentialProviderLoader`" This reverts commit e1eed92.
…der discovery" This reverts commit 54af016.
…erviceCredential, UserCredentials)" This reverts commit 177b6ce.
uros-b
approved these changes
Aug 13, 2026
Member
|
Seems like a well-executed, exhaustively documented revert, thank you @sarutak! cc @cloud-fan |
sarutak
added a commit
that referenced
this pull request
Aug 14, 2026
…om `branch-4.3` (moved to `4.4.0`) ### What changes were proposed in this pull request? This PR reverts all OIDC credential propagation commits from `branch-4.3`. The feature will ship in `4.4.0` instead. **Reverted commits (newest first):** | Commit | Description | |--------|-------------| | `b7d480b53e4` | [SPARK-58504] Simplify FileTokenIngestor cache and detect rotation by content | | `1623ebd0932` | [SPARK-58472] Add `AutoCloseable` lifecycle to `CredentialProvider` SPI | | `e9b8b1b1301` | [SPARK-57897] Create `connector/credential-aws` module with CI/release integration | | `057b6079689` | [SPARK-57893] Implement `UserCredentialManager` | | `72c284fb1c2` | [SPARK-58364] Rename the configuration namespace from `spark.security.credentials` to `spark.security.oidc` | | `b02704ad2ce` | [SPARK-57892] Add TokenIngestor interface and FileTokenIngestor implementation | | `e1eed92c6bc` | [SPARK-57891][FOLLOWUP] Fix contradictory thread-safety docs in `CredentialProviderLoader` | | `54af016ec8a` | [SPARK-57891] Add CredentialProvider SPI and ServiceLoader discovery | | `177b6cec95f` | [SPARK-57890] Add core credential types (UserContext, ServiceCredential, UserCredentials) | **Not reverted:** | Commit | Reason | |--------|--------| | `825f76a7e47` ([SPARK-57897][4.X][BUILD][FOLLOWUP]) | Contains a fix to `sql/connect/client/jdbc/pom.xml` (gRPC API version) unrelated to OIDC. The OIDC-related portion (`connector/credential-aws/pom.xml`) is removed by the revert of `e9b8b1b1301`. | **Conflict resolutions:** 1. **`connector/credential-aws/pom.xml`** (during revert of `e9b8b1b1301`): - Conflict type: modify/delete. The file was added by `e9b8b1b1301` and later modified by `825f76a7e47` (parent version fix). Since the entire module is being removed, resolved by `git rm`. 2. **`core/src/main/scala/org/apache/spark/internal/config/package.scala`** (during revert of `057b6079689`): - Conflict type: content. The OIDC config keys (`SECURITY_OIDC_*`) were added by `057b6079689`, and `DIRECT_CREDENTIAL_PROVIDERS_ENABLED` was added by a separate commit (`bcbe9e0fdd8`, SPARK-38954) in the same region. - Resolution: Removed the four `SECURITY_OIDC_*` config keys while **retaining** `DIRECT_CREDENTIAL_PROVIDERS_ENABLED`, which is used by SPARK-38954 (independent of OIDC). ### Why are the changes needed? The OIDC credential propagation feature was backported to `branch-4.3` during development, but the release target has been moved to 4.4.0. Version annotations on master have already been updated to 4.4.0. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? `build/sbt "core/compile"` passes after all reverts. ### Was this patch authored or co-authored using generative AI tooling? Kiro CLI / Claude Closes #57985 from sarutak/revert-oidc-from-4.3. Authored-by: Kousuke Saruta <sarutak@apache.org> Signed-off-by: Kousuke Saruta <sarutak@apache.org>
Member
Author
|
Merge Summary:
Posted by |
Member
Author
|
Merged. Thank you @uros-b for reviewing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR reverts all OIDC credential propagation commits from
branch-4.3. The feature will ship in4.4.0instead.Reverted commits (newest first):
b7d480b53e41623ebd0932AutoCloseablelifecycle toCredentialProviderSPIe9b8b1b1301connector/credential-awsmodule with CI/release integration057b6079689UserCredentialManager72c284fb1c2spark.security.credentialstospark.security.oidcb02704ad2cee1eed92c6bcCredentialProviderLoader54af016ec8a177b6cec95fNot reverted:
825f76a7e47([SPARK-57897][4.X][BUILD][FOLLOWUP])sql/connect/client/jdbc/pom.xml(gRPC API version) unrelated to OIDC. The OIDC-related portion (connector/credential-aws/pom.xml) is removed by the revert ofe9b8b1b1301.Conflict resolutions:
connector/credential-aws/pom.xml(during revert ofe9b8b1b1301):e9b8b1b1301and later modified by825f76a7e47(parent version fix). Since the entire module is being removed, resolved bygit rm.core/src/main/scala/org/apache/spark/internal/config/package.scala(during revert of057b6079689):SECURITY_OIDC_*) were added by057b6079689, andDIRECT_CREDENTIAL_PROVIDERS_ENABLEDwas added by a separate commit (bcbe9e0fdd8, SPARK-38954) in the same region.SECURITY_OIDC_*config keys while retainingDIRECT_CREDENTIAL_PROVIDERS_ENABLED, which is used by SPARK-38954 (independent of OIDC).Why are the changes needed?
The OIDC credential propagation feature was backported to
branch-4.3during development, but the release target has been moved to 4.4.0. Version annotations on master have already been updated to 4.4.0.Does this PR introduce any user-facing change?
No.
How was this patch tested?
build/sbt "core/compile"passes after all reverts.Was this patch authored or co-authored using generative AI tooling?
Kiro CLI / Claude