Skip to content

feat(bundle): run --dry-run, another bundle's entities by @local id, bare depends_on ids - #22

Merged
earakely-scale merged 8 commits into
mainfrom
edgararakelyan/plan-checks-dry-run
Sep 30, 2026
Merged

earakely-scale merged 8 commits into
mainfrom
edgararakelyan/plan-checks-dry-run

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Three changes to how a bundle is checked before anything is written, one commit each:

  1. depends_on takes a bare step id. ["load"] means [{"task_step_id": "load"}].
  2. agent-env run --dry-run makes every check the run makes before its first task and writes nothing.
  3. A bundle can name another bundle's entities by their @local/… id.

A bare step id in depends_on

"depends_on": ["load", {"task_step_id": "box"}]
  • TaskStepDependency.from_dict takes either form, and reads the object form exactly as before, so every stored task that loaded still loads; only a bare entry must be a string. The new dependencies() reads a whole depends_on, and refuses a lone string, which would otherwise be read as a list of its characters.

  • Steps are still stored in the object form, so every stored task and every reader of to_dict() is unchanged.

  • Task._validate_dag now separates the two ways a dependency can be wrong: a step that comes later (still "forward refs are not allowed"), and an id that names no step in the task.

  • The bundle resolver reads depends_on through the same parser. A malformed one used to show up only as "…which this step doesn't depend on"; now it is reported with its cause:

    tasks/t.json: step 'run': depends_on is a list of step ids, not 'cli'
    tasks/t.json: step 'run': depends_on 'clii' names no step in this task
    

agent-env run --dry-run

$ agent-env run triage --dry-run      # after a newer @local/~/other/solver-image was stored
Dry run: nothing is written or run.
agents/solver: v2 (artifact @local/~/other/solver-image has a new version in the store (v1 → v2))
tasks/t.json: v1, unchanged

Store refs:
  artifact @local/~/other/solver-image v2, the latest
  artifact @local/~/other/greeting v2, the latest
Would run:
  tasks/t.json v1
Dry run: nothing is written or run.
  • One flow. It is materialize(plan, dry_run=True): the same refusals, ledger checks and preflights as a run, without the lock, the writers or Task.put / Eval.put. Each write is reported at the version it would reuse or the store's next one. dry_run_bundle() returns the same to Python as a DryRun.
  • Needs are hashed from the walk. Ledger.check(write, versions) hashes the versions the walk passes in for what an env or agent needs, instead of reading the ledger as it stands. So a need that would be written anew rewrites its dependent in the prediction, as it does in the run. Versions are still hashed as strings, so rows a run already recorded stay reused, and no ledger scheme bump is needed.
  • Steps it can't preflight. A step that reads what the run would write first (for example run_code over a changed artifact) can't be preflighted against the store. The dry run lists those whose type has its own preflight, under Not preflighted. Steps over reused writes are preflighted as in a run.
  • Exit status. 0 for a valid plan, whatever it would write; 1 for a problem the run would stop at. --sandbox is checked; --model and --keep change nothing it shows.
  • Known limits. It takes no lock (as run --help says), so another run writing the same ids can change the answer first, and it doesn't see what another bundle's pending run would write. It reads the configured stores the way a run does, so a read can create an empty default local store.

Another bundle's entities by their @local id

# agents/solver/agent.toml
image = "@local/~/other/solver-image"
  • An explicit @local/… id outside the bundle is checked against the @local namespace's store, like any store id. Until now it was refused as not supported yet.
  • An unpinned one takes the latest version at plan time: an agent is written with that version, and written again, with the reason shown, once a newer version is stored under that id. Tasks and evals stay bare, as for any store ref.
  • An id under the bundle's own root that the bundle doesn't write is still refused without a read, since it is most likely left over from a rename.
  • Planning such a read with namespace routing off raises, as materializing already does.
  • An eval naming another bundle's task is refused as a store task, with the existing agent-env task run hint.

Docs

The README now points to the docs site and keeps only the plugin contract, so this PR changes no README text; the user-facing docs for bare depends_on, --dry-run and cross-bundle references go to the docs site separately.

Testing

  • Unit tier (make unit-test): 5,684 passed, 13 skipped, with main merged in. Each of the three feature commits also passes on its own: 5,649, 5,665 and 5,671.
  • tst/integration/cli/run_bundle_test.py, which runs agent-env run on hello: 7 passed.
  • End to end: one bundle's agent names another bundle's image, and its task names another bundle's artifact with a bare depends_on.
    • Every dry run, from Python and from the CLI, predicted exactly what the run that followed wrote, including the rewrite after a newer image version was stored under the other bundle's root.
    • The state directory was byte-identical after each dry run.
    • hello with bare depends_on scores 1, stores its steps in the object form, and a rerun reuses both writes.

🤖 Generated with Claude Code

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no new issue or outstanding earlier finding remains.

Fix All in CursorFindings

  1. P1 Bad dependency crashes bundle checks ▶
Fix with agent prompt
### Issue 1
src/agent_env/task_step/task_step.py:26-27
A step with `depends_on: [{"task_step_id": ["load"]}]` now gets past `TaskStepDependency.from_dict`. The resolver then tries to look up that list in a set and raises `TypeError`. Bundle checks and dry runs crash instead of explaining the bad dependency. Reject object-form IDs that are not strings.

```suggestion
        if isinstance(data, dict) and isinstance(data.get("task_step_id"), str):
            return cls(task_step_id=data["task_step_id"])
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Bundle runs can now be previewed with --dry-run, and bundles can read entities from another bundle using @local IDs. Task steps also accept bare IDs in depends_on, while keeping the stored format the same.

  • Dry runs show predicted writes and tasks without running or writing them.
  • Cross-bundle references are checked against the local namespace store.
  • Bare dependency IDs work alongside the existing object form.

Reviews (5) · Last reviewed commit: "refactor(bundle): simpler ledger walk an..."

earakely-scale and others added 3 commits September 30, 2026 09:58
TaskStepDependency.from_dict takes a step id or {"task_step_id": <id>}, and dependencies() reads a whole
depends_on: a lone string or any other shape is a ValueError rather than a list of characters. Steps are
still stored in the object form. _validate_dag now tells a step that comes later apart from an id that names
no step in the task.

The bundle resolver reads depends_on through the same parser, so a malformed one or an unknown id is
reported with its cause, not only as an output the step doesn't depend on.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
materialize(plan, dry_run=True) makes every check and preflight a run makes before its first task and
writes nothing: no entity, ledger row, lock or instance. Each write is reported at the version it would
reuse or the store's next one. dry_run_bundle() returns the same to Python as a DryRun.

Ledger.check(write, versions) hashes the versions the walk passes in for what an env or agent needs, so a
need that would be written anew rewrites its dependent in the prediction as in the run. Versions are
hashed as strings, as before, so the rows a run recorded stay reused.

A step that reads what the run would write first can't be preflighted against the store; the dry run
lists those whose type has a preflight.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An explicit @local/... id outside the bundle is checked against the @Local namespace's store like any
store id. An agent is written with the version the plan read, and again once that bundle writes a new
one; tasks and evals stay bare. An id under the bundle's own root that it doesn't write is still refused
without a store read. Planning such a read with namespace routing off raises.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale
earakely-scale requested review from a team and polakamtejas as code owners September 30, 2026 20:01
Comment thread src/agent_env/bundle/materialize.py Outdated
earakely-scale and others added 2 commits September 30, 2026 11:29
An artifact and an agent can share an id, so a dry run rewriting the agent skipped the preflight of a step
over the reused artifact, and exited 0 where the run then stopped. It now compares (store, id); a step
reading its own task's output is still matched by id, so the real run is unchanged.

- Deriving a step's output id reads the step without its depends_on, so a malformed one is reported once.
- dry_run_bundle's docstring says a failed preflight raises and that routing is on process-wide while it
  runs; run --dry-run's help says it takes no lock.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale
earakely-scale requested a review from a team as a code owner September 30, 2026 21:30
{"task_step_id": <id>} is read as it always was, whatever the id holds, so every stored task main loads
still loads; only the new bare form must be a string. A lone string and any other shape are still refused.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment on lines +26 to +27
if isinstance(data, dict) and "task_step_id" in data:
return cls(task_step_id=data["task_step_id"])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Bad dependency crashes bundle checks

A step with depends_on: [{"task_step_id": ["load"]}] now gets past TaskStepDependency.from_dict. The resolver then tries to look up that list in a set and raises TypeError. Bundle checks and dry runs crash instead of explaining the bad dependency. Reject object-form IDs that are not strings.

Suggested change
if isinstance(data, dict) and "task_step_id" in data:
return cls(task_step_id=data["task_step_id"])
if isinstance(data, dict) and isinstance(data.get("task_step_id"), str):
return cls(task_step_id=data["task_step_id"])
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/task_step/task_step.py
Line: 26-27

Comment:
**Bad dependency crashes bundle checks**

A step with `depends_on: [{"task_step_id": ["load"]}]` now gets past `TaskStepDependency.from_dict`. The resolver then tries to look up that list in a set and raises `TypeError`. Bundle checks and dry runs crash instead of explaining the bad dependency. Reject object-form IDs that are not strings.

```suggestion
        if isinstance(data, dict) and isinstance(data.get("task_step_id"), str):
            return cls(task_step_id=data["task_step_id"])
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Cursor Fix in Claude Code Fix in Codex

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the resolver instead (next commit): a bundle's depends_on that names a step by anything but its id is now reported as a problem, not a crash. The object form stays lenient in TaskStepDependency on purpose, so every stored task that loads today keeps loading.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That resolution is valid. Keeping TaskStepDependency.from_dict lenient preserves compatibility for already-stored tasks, while _depends_on now validates the bundle’s raw depends_on before dependency IDs are used. Malformed entries and non-string task_step_id values therefore become bundle problems instead of reaching an unhashable lookup. The added resolver tests cover the list-valued object form and confirm it reports the cause alongside the missing dependency edge, so no further change to TaskStepDependency.from_dict is needed.

earakely-scale and others added 2 commits September 30, 2026 11:50
…roblem

The object form reads any id, as before, so a bundle task could reach the resolver with
{"task_step_id": ["load"]} and crash on the lookup. The resolver now reports such an entry and
leaves it out of the step's edges.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- materialize checks each write in its own closure and gives the ledger only the versions of what the
  write needs, so Ledger.check(write, needs) matches digest(write, needs) and no versions map shadows done.
- The resolver keeps a dependency id as an edge only when it names a step, with one message otherwise, so
  _upstream no longer filters.
- _validate_dag raises one error with the clause that applies, and _preflight's docstring is rewrapped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit 67f1040 into main Sep 30, 2026
14 checks passed
@earakely-scale
earakely-scale deleted the edgararakelyan/plan-checks-dry-run branch September 30, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant