fix: support shrunk client payloads on v1 - #568
Conversation
client_payload arrives as plain JSON, base64(gzip), or a reference to a server-stashed payload. v1 resolved fields with fromJSON(fromJSON(github.event.inputs.client_payload)) in five places, which throws on the last two shapes before the step can run. Resolve those fields in a step instead, following the existing script.js idiom on this branch - a root-level module exported as a function of core, required with github.action_path. Only the five fields v1's action.yml actually consumes are emitted; v1 has no cm org checkout. CLIENT_PAYLOAD is still handed to the docker rules engine untouched. The engine resolves the envelope itself (gitstream/rules-engine:latest ships gitstream-core 2.1.301, which has that support), and passing the inflated payload through the runner env would risk E2BIG on large payloads. The deprecation notice, docker cache/pull/load steps and the engine invocation are untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Secrets | View in Orca |
There was a problem hiding this comment.
✨ PR Review
The PR cleanly solves the multi-shape payload problem and the origin-pinning, gzip-bomb cap, and token-masking hardening are all well thought out. Two focused correctness issues are worth addressing before merge.
2 issues detected:
🐞 Bug - A JSON parse exception thrown by `parsePayload` inside `readStashReference` escapes uncaught and aborts the entire resolution, preventing the gzip path from being tried. 🛠️
Details: readStashReference does a cheap substring pre-check and, when it matches, unconditionally calls parsePayload(raw). If a base64-gzip payload's text representation happens to contain the substring 'oversized-payload-reference' (statistically rare but possible), JSON.parse throws and the exception propagates straight out of resolvePayload without ever reaching the gzip branch. The step then fails with a JSON parse error instead of correctly inflating the payload.
File: resolve-payload-fields.js (56-57)
🛠️ A suggested code correction is included in the review comments.
🐞 Bug - Template-literal interpolation of `undefined` produces the string `"undefined"` rather than surfacing the missing field as an early, descriptive error. 🛠️
Details: In toStepOutputs, cm_repository is built as `${payload.owner}/${payload.cmRepo}` only when hasCmRepo is true, but neither payload.owner nor payload.cmRepo is validated. If either field is absent, the output becomes "undefined/undefined" or "owner/undefined", which is passed directly to actions/checkout@v4 and causes a confusing "repository not found" failure far from the real defect.
File: resolve-payload-fields.js (127-127)
🛠️ A suggested code correction is included in the review comments.
Generated by LinearB AI and added by gitStream.
AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Review using Guidelines Learn how
| const parsed = parsePayload(raw); | ||
| return parsed && parsed.type === OVERSIZED_PAYLOAD_REFERENCE ? parsed : null; |
There was a problem hiding this comment.
🐞 Bug - Spurious Parse Failure: Wrap the parsePayload call in readStashReference in a try/catch and return null on any parse error, so the caller can continue to the gzip path:
function readStashReference(raw) {
if (!raw.includes(OVERSIZED_PAYLOAD_REFERENCE)) {
return null;
}
try {
const parsed = parsePayload(raw);
return parsed && parsed.type === OVERSIZED_PAYLOAD_REFERENCE ? parsed : null;
} catch {
return null;
}
}| const parsed = parsePayload(raw); | |
| return parsed && parsed.type === OVERSIZED_PAYLOAD_REFERENCE ? parsed : null; | |
| try { | |
| const parsed = parsePayload(raw); | |
| return parsed && parsed.type === OVERSIZED_PAYLOAD_REFERENCE ? parsed : null; | |
| } catch { | |
| return null; | |
| } |
Is this review accurate? Use 👍 or 👎 to rate it
If you want to tell us more, use /gs feedback e.g. /gs feedback this review doesn't make sense, I disagree, and it keeps repeating over and over
| github_token: payload.githubToken || '', | ||
| url: payload.headHttpUrl || payload.repoUrl || '', | ||
| has_cm_repo: String(hasCmRepo), | ||
| cm_repository: hasCmRepo ? `${payload.owner}/${payload.cmRepo}` : '', |
There was a problem hiding this comment.
🐞 Bug - Undefined Repository Fields: Guard the construction and fall back to an empty string if either field is missing:
cm_repository: hasCmRepo && payload.owner && payload.cmRepo
? `${payload.owner}/${payload.cmRepo}`
: '',| cm_repository: hasCmRepo ? `${payload.owner}/${payload.cmRepo}` : '', | |
| cm_repository: hasCmRepo && payload.owner && payload.cmRepo | |
| ? `${payload.owner}/${payload.cmRepo}` | |
| : '', |
Is this review accurate? Use 👍 or 👎 to rate it
If you want to tell us more, use /gs feedback e.g. /gs feedback this review doesn't make sense, I disagree, and it keeps repeating over and over
Same resolver change as develop and v2-lite: parse first and switch on the value of `type`, so the double-encoded compressed-payload envelope the GitHub trigger will send is inflated instead of being mistaken for a raw payload and silently resolving to empty fields. Both compressed forms stay supported permanently - Bitbucket has no run-name to protect and keeps sending the bare base64(gzip) form. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
✨ PR Review
The PR correctly replaces five inline fromJSON(fromJSON(...)) expressions with a dedicated resolver step, following the existing script.js idiom. The gzip-bomb cap, origin-pinning, and token-masking hardening are well-placed. One previous issue is resolved and one persists; one new issue was identified in the stash-fetch path.
2 issues detected:
🔒 Security - `core.setSecret` is called with a potentially `undefined` value, so the credential is never masked and an opaque auth failure is surfaced instead of a clear configuration error.
Details: In fetchStashedPayload, reference.resolverToken is used without first verifying it is a non-empty string. If the stash reference object lacks a resolverToken field, core.setSecret(undefined) is a no-op (or masks the literal string "undefined"), and the actual secret is never registered for redaction. The fetch then proceeds with Authorization: Bearer undefined, which will fail, but any logging between core.setSecret and the failure may expose the raw reference object — including a token that arrived as a different field name — unmasked.
File: resolve-payload-fields.js (94-96)
🐞 Bug - Missing fields silently produce an `"undefined/..."` repository string that is passed to checkout without any diagnostic.
Details: toStepOutputs constructs cm_repository as \${payload.owner}/${payload.cmRepo}`wheneverhasCmRepoistrue, but neither payload.ownernorpayload.cmRepois validated. If either is absent from the payload, the output becomes"undefined/undefined"or"owner/undefined", which is then passed directly to actions/checkout@v4` as the repository, causing an opaque "repository not found" failure far from the actual defect.
File: resolve-payload-fields.js (151-151)
Generated by LinearB AI and added by gitStream.
AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Review using Guidelines Learn how
Same diagnostic as develop and v2-lite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Closing — not needed. Shrink-payload support is landing on |
* fix: support shrunk client payloads on v1 client_payload arrives as plain JSON, base64(gzip), or a reference to a server-stashed payload. v1 resolved fields with fromJSON(fromJSON(github.event.inputs.client_payload)) in five places, which throws on the last two shapes before the step can run. Resolve those fields in a step instead, following the existing script.js idiom on this branch - a root-level module exported as a function of core, required with github.action_path. Only the five fields v1's action.yml actually consumes are emitted; v1 has no cm org checkout. CLIENT_PAYLOAD is still handed to the docker rules engine untouched. The engine resolves the envelope itself (gitstream/rules-engine:latest ships gitstream-core 2.1.301, which has that support), and passing the inflated payload through the runner env would risk E2BIG on large payloads. The deprecation notice, docker cache/pull/load steps and the engine invocation are untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: read the wrapped compressed-payload envelope Same resolver change as develop and v2-lite: parse first and switch on the value of `type`, so the double-encoded compressed-payload envelope the GitHub trigger will send is inflated instead of being mistaken for a raw payload and silently resolving to empty fields. Both compressed forms stay supported permanently - Bitbucket has no run-name to protect and keeps sending the bare base64(gzip) form. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: name the offending URL when payloadUrl is not absolute Same diagnostic as develop and v2-lite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: hand the engine the payload shape it resolves #568's port resolved the YAML field expressions but passed CLIENT_PAYLOAD to the docker engine untouched. The trigger double-encodes every tier so the workflow's run-name can parse it, and gitstream-core does a single JSON.parse on a reference - so a double-encoded envelope resolves to a string, the reference is never followed, and the engine treats the envelope as the payload. Stripping the outer layer here lets both consumers read the same dispatch, without changing core or the trigger. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: update github-script action version and clean up comments in payload resolution * fix: reattach the toStepOutputs doc to its function Inserting normalizeForEngine above it left the JSDoc describing the wrong function, and toStepOutputs with none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Problem
client_payloadarrives in one of three shapes — plain JSON,base64(gzip(JSON)), or a small reference to a payload stashed on the resolver. v1 read fields out of it withfromJSON(fromJSON(github.event.inputs.client_payload))in five places, which throws on the last two shapes before the step can run.Change
Resolve those fields once, in a step, and read the outputs downstream — following this branch's existing
script.jsidiom (root-level module exported as a function ofcore, required viagithub.action_path), not v2'sscripts/layout.Only the five fields v1's
action.ymlactually consumes are emitted (github_token,url,has_cm_repo,cm_repository,cm_repo_ref) — v1 has no cm-org checkout, so there's nohas_cm_org/cm_org_ref.CLIENT_PAYLOADis still handed to the docker rules engine untouched. The engine resolves the envelope itself, and passing the inflated payload (~1.4MB+) through the runner env would riskE2BIGand defeat the compression.Engine support confirmed
Verified directly against the image rather than assumed:
Same core version as
develop. Since the tag is:latestand pulled fresh each run, v1's engine is current.Untouched
Deprecation notice, docker cache/pull/load steps, the engine invocation,
actions/checkout@v4,actions/github-script@v7, andscript.js.Verification
action.ymlparses (13 steps; the resolve step sits after the deprecation notice and before every consumer). The five outputs consumed match the five emitted, exactly. Exercised end to end:mode=plain, all 5 outputs correctbase64(gzip)mode=compressed, all 5 outputs correctCarries the same hardening as v2: gzip-bomb cap, resolver-origin pinning for stashed fetches, and token masking.
Note
v1
action.ymlstill warns it "is deprecated and will be disabled soon". This is the minimum to keep it working for shrunk payloads, not an investment in the branch.🤖 Generated with Claude Code
✨ PR Description
Purpose: Enable v1 action to handle compressed and oversized client payloads by extracting payload resolution logic into a dedicated module.
Main changes:
Generated by LinearB AI and added by gitStream.
AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Description using Guidelines Learn how