[Yunnglin]

Summary

Enable client-orchestrated, text-only multi-turn GRPO over HTTP: run tool-using episodes with the shared rollout core, consume complete groups through StreamingDriver, train the remote model, and publish immutable policy snapshots for subsequent episodes.

Changes

• Extend ClientMultiTurnRollout with admit/poll/collect/stop and scoped streaming cleanup while retaining synchronous and asynchronous batch calls. Preserve sampled token IDs, labels, masks and logprobs without re-encoding the conversation; reject unsupported media or divergent engine encodings.
• Track independent Env/harness borrows and immutable admission metadata. Cooperative cancellation waits for issued work; batch draining and interruptions after executor enqueue retain ownership until settlement. Unknown remote execution is reported and quarantined rather than treated as released.
• Add RolloutResources and scoped operation protection so examples call model methods directly without implementing cleanup or interrupt bookkeeping. Keep sampling concurrent with training, and do not automatically replay training operations.
• Add training_groups for complete-group membership, policy and training-data checks. Keep reward selection, GRPO advantages, backward, optimizer steps and publication/commit explicit in the cookbook.
• Provide Blackjack examples using local OpenEnv or one official E2B sandbox per trajectory. Both use the same training loop and task contract; the E2B path runs trusted game tools without a generated-code verifier. Add the optional [e2b] dependency group and bilingual setup/extension guidance; retain the existing batch multi-turn example.
• Fix supporting training contracts: completed-backward gradient-accumulation boundaries, exact LoRA slot-name matching in both training backends, missing Megatron text positions, optional media fields, template/harness metadata preservation, AgentEnv initialization failures and batched EnvTool reward/terminal-state recording. Correct task-head initialization and metric reporting in the existing classification/reranking examples.

Existing DataPlane/DataRef APIs and the paper TQ runtime are retained. This adds no server-side rollout service or HTTP/Wire protocol.

Validation

• Current implementation: 402 local tests passed, 0 failed, 0 skipped (407 → 402 after consolidating redundant coverage; 166 net test lines removed). Covers the complete client suite plus shared multi-turn, tools, AgentEnv initialization, token utilities, templates and input processing. Includes controlled concurrent-worker regressions and real local-HTTP subprocess tests with repeated SIGINT. The reviewed source snapshot matches the current PR head.
• New SDK modules pass explicit Ruff checks; existing changed SDK/agentic sources have no additional diagnostics relative to the base. Edited documentation links and git diff --check pass. This is not a claim that the entire repository is lint-clean.
• Earlier hardware snapshot, before the final SDK refactor: Transformers DP=2 + vLLM eager + official E2B completed real Blackjack training, two optimizer updates, policy publication and subsequent sampling with new policy URIs. Changed adapter weights and release of all 35 test sandboxes were verified. These results are historical evidence, not current-head hardware acceptance or evidence of learning improvement.

Remaining validation and boundaries

• Rerun real PPU/E2B acceptance after the final SDK lifecycle refactor; complete both training backends and strict sampler-versus-trainer logprob comparisons.
• Validate two independent tenants against one server, including external harness state isolation, cancelling one tenant while the other continues, and per-tenant resource release.
• Cancellation is cooperative; immediate remote generation/command termination requires additional server/provider support.
• Independent evaluation of training benefit remains outstanding. Successful updates, finite loss and changed weights alone do not establish improved task performance.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.