Skip to content

xds: Add ExtAuthzClientCall - #12897

Merged
sauravzg merged 4 commits into
masterfrom
dev/sauravzg/client-interceptor
Sep 10, 2026
Merged

sauravzg merged 4 commits into
masterfrom
dev/sauravzg/client-interceptor

Conversation

@sauravzg

@sauravzg sauravzg commented Jul 7, 2026 •

Copy link
Copy Markdown
Contributor

Introduces ExtAuthzClientCall, which buffers outgoing RPC frames while performing async ext_authz checks. Because the authorization call is asynchronous, outgoing sendMessage() and halfClose() frames are buffered in ExtAuthzClientCall until a decision arrives. On allow, buffered operations are replayed with any header mutations applied. On deny, the call is terminated immediately.

Key classes:

  • AuthzCallbackObserver — async listener on the authz gRPC stream that signals the buffering call on completion.
  • ExtAuthzClientCall — buffers outgoing frames while authz is pending; drains or cancels based on the decision.
  • MutatingClientCall — wraps the real call to inject header mutations from the authz response.
  • FailingClientCall / FailingCallWithTrailerMutations — immediate termination paths for denied or disabled-filter scenarios.

@sauravzg
sauravzg requested a review from kannanjgithub July 7, 2026 13:03
Comment thread xds/src/main/java/io/grpc/xds/internal/extauthz/AuthzCallbackObserver.java Outdated

@Override
public void onNext(CheckResponse value) {
// Note on exception safety: handleResponse() internally catches

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uncaught exception from onNext causes the stub to cancel the stream and route directly into onError(), which closes authzContext. Its closure does not rely on GC of the stream observer, and it cannot, because it has a direct reference to it from ExtAuthzClientCall.

In a unary rpc, onNext() delivers an inbound response payload; it does not signify that the RPC is complete. A unary RPC completes only when onClose / onCompleted() / onError() is invoked.

The comment needs to be corrected on both the above fronts.

Is relying on this mechanism "Valid Code"?

Mechanically, yes, gRPC will catch the exception and route to onError(). If an unhandled RuntimeException escapes onNext(), ClientCallImpl cancels the stream and delivers onError(), which drains delayedCall and closes authzContext.

However, intentionally relying on this as a design pattern is fragile and carries real risks:

A. Security Risk: Accidentally Failing Open on a DENY

This is the biggest risk:

// in onNext():
AuthzResponse authzResponse = responseHandler.handleResponse(value);
if (authzResponse.decision() == AuthzResponse.Decision.ALLOW) { ... }
else {
  // Suppose decision is DENY, but building FailingCallWithTrailerMutations throws an exception
  ...
}

If the external authorization server returned an explicit DENY response, but an unexpected RuntimeException is thrown while preparing the rejection:

  1. The exception escapes onNext().
  2. The stub cancels the stream and invokes onError(t).
  3. In onError():
    if (config.failureModeAllow()) {
      // FAILS OPEN: creates next.newCall() and forwards request to backend!
    }

If failure_mode_allow: true is configured (which is only intended for network/server outages reaching the authz service), a request that the authz server explicitly denied would be forwarded to the backend!

B. The "Throwing to Trigger Another Callback on Yourself" Anti-Pattern

Relying on throwing an unhandled exception out of onNext() so that the underlying framework cancels the transport stream and calls back into your own onError() is an indirect, surprising control flow:

  • It generates unnecessary transport RST_STREAM frames and Failed to read message error logs from ClientCallImpl exception handling.
  • It makes debugging difficult because the root cause in onNext gets swallowed into a transport Status.CANCELLED exception passed to onError.

C. Partial Failure State

If an exception occurs after delayedCall.setCall(call) was executed (for example, if callExecutor.execute(drain) throws a RejectedExecutionException):

  • When onError() subsequently runs, its call to delayedCall.setCall(...) returns null because realCall != null was already set. The failure call cannot be attached, leaving the call in an inconsistent state.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was interesting.
So, practically it largely doesn't matter right now, because the current code is exception safe. So, a runtime exception shouldn't happen.

But given the security considerations, maybe we should be defensive, I'll have to see if other implementations in flight are as defensive and make changes. Keeping his comment open for now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree the code is exception safe. We can just remove the comment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't removed the code but rephrased it. I have added that the code is exception safe, but added a todo to revisit this if this changes sometime in future just as a defensive callout.

Comment thread xds/src/test/java/io/grpc/xds/internal/extauthz/ExtAuthzClientCallTest.java Outdated
@sauravzg sauravzg changed the title xds: Add ExtAuthzClientInterceptor and call buffering xds: Add ExtAuthzClientCall Sep 4, 2026
@sauravzg
sauravzg force-pushed the dev/sauravzg/client-interceptor branch from 08fa030 to ac83094 Compare September 7, 2026 06:00
@sauravzg
sauravzg force-pushed the dev/sauravzg/client-interceptor branch from 5f3cb3f to 8a87048 Compare September 7, 2026 17:15
@sauravzg
sauravzg force-pushed the dev/sauravzg/client-interceptor branch from 8a87048 to 350a7bb Compare September 7, 2026 18:57
@sauravzg sauravzg added the kokoro:force-run Add this label to a PR to tell Kokoro to re-run all tests. Not generally necessary label Sep 9, 2026
@sauravzg
sauravzg force-pushed the dev/sauravzg/client-interceptor branch from 90946cc to 03e0356 Compare September 9, 2026 14:29
Base automatically changed from dev/sauravzg/response-handling to master September 9, 2026 15:35
Part 3 of the client-side ext_authz filter. Sits on top of the
response handling PR.

Introduces the ClientInterceptor that performs async ext_authz
checks on outgoing RPCs. Because the authorization call is
asynchronous, outgoing sendMessage() and halfClose() frames are
buffered in ExtAuthzClientCall until a decision arrives. On allow,
buffered operations are replayed with any header mutations applied.
On deny, the call is terminated immediately.

Key classes:
- AuthzCallbackObserver — async listener on the authz gRPC stream
  that signals the buffering call on completion.
- ExtAuthzClientCall — buffers outgoing frames while authz is
  pending; drains or cancels based on the decision.
- MutatingClientCall — wraps the real call to inject header
  mutations from the authz response.
- FailingClientCall / FailingCallWithTrailerMutations — immediate
  termination paths for denied or disabled-filter scenarios.
@sauravzg
sauravzg force-pushed the dev/sauravzg/client-interceptor branch from 03e0356 to 0724222 Compare September 9, 2026 15:35
@grpc-kokoro grpc-kokoro removed the kokoro:force-run Add this label to a PR to tell Kokoro to re-run all tests. Not generally necessary label Sep 9, 2026
@sauravzg
sauravzg merged commit 70e184a into master Sep 10, 2026
23 of 25 checks passed
@sauravzg
sauravzg deleted the dev/sauravzg/client-interceptor branch September 10, 2026 07:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants