[jencymaryjoseph]

Motivation and Context

AsyncRequestBody.split and splitCloseable use SplittingPublisher to divide one streaming request body into logical parts, such as the parts used by multipart uploads. The data for each logical part arrives as one or more ByteBuffer objects.
This change fixes two issues in that flow:

1. Logical part sizes larger than an int failed

SplittingPublisher calculated the number of bytes remaining in the current logical part as an int:

currentBody.maxLength() - currentBody.receivedBytesLength()

Both values are long, and a logical multipart part can be larger than Integer.MAX_VALUE. Converting the result to an int caused an ArithmeticException for part sizes such as 5 GiB, even though each individual ByteBuffer was small enough to use normal int-based buffer positions and limits.

2. Terminal error handling could call the upstream subscription

Splitter failures were handled through several separate paths. While processing upstream onError or onComplete, propagating a terminal signal could synchronously trigger another call to Subscription.cancel() or Subscription.request(...). Reactive Streams rule 2.3 prohibits calling methods on the upstream subscription from these terminal callbacks.
Failures could also escape directly instead of being consistently delivered to both the active split request body and the outer publisher through onError.

Modifications

• Keep amountRemainingInChunk as a long while comparing and tracking logical part sizes.
• Convert the remaining count to an int only when it is used to select a position inside one ByteBuffer. At that point, the preceding comparisons guarantee that the value is smaller than byteBuffer.remaining() and therefore fits in an int.
• Centralize failure handling in an idempotent failStream path that:
  • signals the active split request body;
  • signals the outer publisher;
  • cancels the upstream subscription only when the failure occurs before an upstream terminal callback.
• Track when upstream onError or onComplete has started, and prevent direct or consumption-triggered upstream requests after that point.
• Enforce the Reactive Streams null-signal requirements for onSubscribe, onNext, and onError.
• Correct the content-length mismatch message so the buffered and expected values are reported in the correct order.
• Add focused regression tests and a PublisherVerification TCK suite for SplittingPublisher.

Testing

The regression coverage includes:

• chunk sizes at Integer.MAX_VALUE, Integer.MAX_VALUE + 1, and 5 GiB;
• both split and splitCloseable;
• upstream request failures and content-length mismatches;
• error propagation to both the active body and outer publisher;
• ensuring upstream request and cancel are not invoked from terminal callbacks;
• standard Reactive Streams publisher behavior through the TCK.

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist

• I have read the CONTRIBUTING document
• Local run of mvn install succeeds
• My code follows the code style of this project
• My change requires a change to the Javadoc documentation
• I have updated the Javadoc documentation accordingly
• I have added tests to cover my changes
• All new and existing tests passed
• I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
• My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

• I confirm that this pull request can be released under the Apache 2 license

[dagnir]

Can we make this message more customer facing? i.e. this means that if customers try to upload with min part sizes of 2GiB or more through the multipart client or TM, it should work now right?

[jencymaryjoseph]

Yes, multipart uploads through the S3 multipart client or Transfer Manager can now use minimum part sizes of 2 GiB or larger, up to S3’s 5GB maximum. Updated the change log message