ENG-1635: Decrypt and install encrypted private artifacts in the runtime setup path#3816
Merged
Merged
Conversation
…path Detect encrypted private-ingredient payloads by envelope magic during the unpack stage, decrypt them under the org key supplied via the options seam, and extract the inner tar.gz archive in place before the artifact is committed to the depot. Decrypted artifacts are marked private so they survive LRU eviction. Extraction routes through the shared untrusted-source sanitizer. A wrong key or corrupt payload fails closed with an error naming the artifact. A missing key (no key service configured, or the key service unreachable) fails open: the artifact is skipped, not committed to the depot, and the runtime hash is left unsaved so the next update retries once a key is available. Skipped artifacts fire ArtifactInstallSkipped so the install progress bar still completes. ENG-1635 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the organization key is unavailable, encrypted private ingredients are skipped during runtime setup. pkg/runtime cannot output to the user and the skip set lives in a transient setup object, so the skip is reported by way of the existing event stream: ArtifactInstallSkipped now carries the artifact name, a small collector handler in runbits accumulates them, and the caller prints a single warning naming the skipped ingredients after the update completes. This also covers the case where no key service is configured at all, which was previously skipped silently. ENG-1635 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR adds support for consuming encrypted private-ingredient artifacts during runtime setup: the runtime unpacks the outer artifact as usual, detects an encrypted payload by envelope magic, decrypts/extracts it using an org key (when available), and ensures decrypted artifacts are protected from cache eviction. When the org key is unavailable, only the affected artifacts are skipped (with user-facing reporting) and the runtime hash isn’t saved so the next run retries automatically.
Changes:
- Decrypt/extract encrypted payloads during
pkg/runtimeartifact unpack, and track skipped artifacts so installs can continue. - Introduce a
Privatecache flag to exempt decrypted artifacts from depot eviction; skip saving runtime hash when any artifacts were skipped. - Add an
ArtifactInstallSkippedevent and wire it into runtime progress + final user warning; fetch org key viainternal/runbits/orgkey.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| pkg/runtime/setup.go | Detects encrypted payloads during unpack, decrypts/extracts when key is present, or records skips when key is absent. |
| pkg/runtime/runtime.go | Avoids saving the runtime hash when any artifacts were skipped, forcing a retry on next update. |
| pkg/runtime/events/events.go | Adds ArtifactInstallSkipped event to represent unpack-time skips in the install stage. |
| pkg/runtime/depot.go | Adds Private flag + MarkPrivate to prevent eviction of decrypted private artifacts. |
| pkg/runtime/decrypt_test.go | Adds unit tests for payload detection/decryption and private artifact eviction behavior. |
| internal/runbits/runtime/runtime.go | Fetches org key (if configured), passes it into runtime setup, and reports skipped private ingredients. |
| internal/runbits/runtime/progress/progress.go | Advances install progress when an artifact is skipped (ensures bar completes). |
| internal/runbits/runtime/progress/progress_skip_test.go | Tests install progress completion behavior for skipped artifacts. |
| internal/artifactcrypto/artifactcrypto.go | Adds lightweight IsEncrypted probe for envelope detection. |
| internal/artifactcrypto/artifactcrypto_test.go | Adds tests for IsEncrypted. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Collaborator
Author
|
Test failures are known and/or sporadic, and unrelated to this PR. |
MDrakos
requested changes
Jun 23, 2026
| key, keyID, err := orgKeyProvider.Key(context.Background()) | ||
| if err != nil { | ||
| prime.Output().Notice(locale.Tl("warn_orgkey_unavailable", | ||
| "[WARNING]Warning:[/RESET] Could not fetch the organization key: {{.V0}}. Encrypted private ingredients will be skipped and retried once the key service is reachable.", |
Member
There was a problem hiding this comment.
I think the user will have to try a command that updates the runtime in order for these to be retired, so it's not just restarting their service.
Collaborator
Author
There was a problem hiding this comment.
You're right. I've updated both this message and the one below to have the user run state refresh once the key is available in order to install the private artifacts.
MDrakos
approved these changes
Jun 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ENG-1635: Decrypt and install encrypted private artifacts in the runtime setup path
The consume side of the private ingredient work (ENG-1563). When a project uses an encrypted private ingredient, the State Tool now decrypts it with the organization's key while setting up the runtime, so it installs like any other package. If the key is missing or the key service is unreachable, everything else still installs and the user is told which ingredients were skipped — they're picked up automatically on the next run once the key is available.
The outer artifact downloads and unpacks as usual. Before it's committed to the cache, an encrypted payload (detected by its envelope, not its filename) is decrypted with the org key fetched in
runbits(ENG-1632) and extracted through the untrusted-source sanitizer (ENG-1640). Decrypted artifacts are kept out of cache eviction. A wrong key fails the install with a clear error naming the artifact; a missing key skips just that ingredient and never blocks the rest of the runtime.Deviations: the inner payload is a tar.gz rather than a wheel/zip, so it can carry symlinks and permissions and matches the existing artifact format; the cache flag is named
Privaterather than the ticket'sPinned. Architecture/import tests and the end-to-endstate installtest are deferred (ENG-1642 and a follow-up).Base branch: targets
mitchell/eng-1640(in review) so the diff is only this change; GitHub will retarget it toversion/0-48-1-RC2once the upstream PRs land.Tested for detection, decrypt, wrong-key and missing-key handling, cache-eviction survival, and progress/skip reporting.
🤖 Generated with Claude Code