Skip to content

fix(python): lift the ~95-call cap and stop echoing the final value - #2832

Merged
chaliy merged 1 commit into
mainfrom
claude/project-thread-pxlkst-python-budget
Nov 5, 2026
Merged

chaliy merged 1 commit into
mainfrom
claude/project-thread-pxlkst-python-budget

Conversation

@chaliy

@chaliy chaliy commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Михайло · project thread

What changed

  • Embedded python3 (Monty) can be called as often as a script likes. Each call now pays the shared work budget for the heap it actually uses (one unit per 64 bytes past the run's high-water mark) instead of reserving max_memory / 64 units up front. Repeated large runs still share one budget (TM-DOS-096 holds: the regression test now uses two 12 MiB runs against a ~19 MiB budget).
  • The last expression is no longer echoed (breaking: python3 -c "2 + 2" prints nothing, like CPython for -c, files and stdin). Every program ends with a None statement so Monty's final value is always None.

Why

Found reviewing a downstream integration (harnx), reproduced on 0.20.3:

  • With defaults, the reservation was 1M of 100M units, so the 96th python3 -c pass in one script failed with work units exhausted.
  • Monty converts the module's final value to a host object outside its memory and time limits. x = [x, x] thirty times as the last line expands to 2^30 leaves (~500 MiB) and ignores the timeout. Bashkit can't bound that conversion, so it must never happen.

Before / After

Before:

$ bashkit -c 'for i in $(seq 1 100); do python3 -c pass || { echo "failed at $i"; break; }; done'
Error: resource limit exceeded: execution budget exhausted: work units (100663793 used, max 100000000)
$ timeout 20 bashkit -c "python3 -c 'x=[1]
for i in range(30): x=[x,x]
x'"; echo $?
124

After: 300 calls in one script succeed (many_small_invocations_fit_in_one_script); the shared-graph program exits 0 immediately (test_large_shared_final_value_finishes); a heavy-memory loop still exhausts a small budget (memory_used_is_charged_to_the_work_budget, verified failing with the charge removed).

Risk

  • Medium
  • Users relying on REPL-style echo get no output; docs (python.md, knowledge) now say to use print(). Code ending in a trailing backslash is left unmodified so it stays a SyntaxError.

Checklist

  • Tests added or updated
  • Backward compatibility considered

Generated by Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

@chaliy chaliy self-assigned this Nov 5, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit 739ac73 Commit Preview URL

Branch Preview URL
Nov 05 2026, 03:55 AM

@bashkit-pr-reviewer

bashkit-pr-reviewer Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Review verdict: approve-worthy, no findings

This PR fixes two real bugs in the Python builtin and the fix approach is sound. Charging actual Monty heap high-water (charge_memory_peak, 64 bytes/work unit) instead of reserving max_memory / 64 up front removes the ~95-call cap on trivial python3 -c 'pass' invocations while keeping TM-DOS-096 enforced — the updated tests prove both directions (300× pass succeeds; repeated 12–16 MiB heaps still exhaust the shared work budget). Appending \nNone\n so Monty's final value is always None closes the ~500 MiB shared-list host-expansion hole, and matches CPython (-c/file/stdin never echo the last expression); the trailing-backslash guard preserves the syntax-error path and the Complete(_) arm already ignores the result, so nothing user-visible is lost.

Findings: none — no P0/P1/P2. Ordering in charge_memory_peak (record high-water before consume_work, so a failed charge isn't double-counted) and the monotonic charged_memory (frees don't refund, regrowth to the same peak costs nothing) are both correct for a peak-based charge.

Context read beyond the diff: AGENTS.md, knowledge/runtimes/python-builtin.md, knowledge/log.md, crates/bashkit/docs/python.md, full run_python loop (budget entry, OsCall/FunctionCall/Complete arms), BudgetTracker::{consume_work, charge_memory_peak, check_vm, on_grow}, unit tests (test_inline_expression_is_not_echoed, test_large_shared_final_value_finishes, test_trailing_backslash_stays_a_syntax_error), integration tests (execution_budget_tests.rs, python_integration_tests.rs, python_security_tests.rs incl. class-escape module), and tests/spec_cases/python/python.test.sh. Verified the PR's claims against project rules (regression tests added, knowledge + docs updated in the same change, decisions recorded as comments).

Update on latest push (head 739ac73, rebased onto 2f35ec2): re-checked out and re-verified — same 8 files, same +143/−42 shape, same logic (high-water charge, None suffix + backslash guard, same regression tests). No new changes to review; earlier conclusions stand.

Not reviewed: nothing material. Diff reported truncated: false. As usual I could not build or run tests (no cargo in this shell) — I reasoned from the code, and LimitedTracker::current_memory lives in the external Monty crate so I took its byte-count semantics from the name and usage.

…ng the final value

Each embedded python3 call reserved max_memory/64 work units up front
(1M at the 64 MiB default) out of the 100M budget, so the 96th call in a
script failed even for `pass`. Charge Monty heap growth past the run's
high-water mark instead; repeated large runs still share one budget.

Monty converts the module's final value to a host object outside its
memory and time limits, so a shared list graph as the last expression
(x = [x, x] thirty times) expanded to 2^30 leaves and hung past the
timeout. End every program with a None statement and drop the REPL-style
echo, matching CPython for -c, files and stdin.
@chaliy
chaliy force-pushed the claude/project-thread-pxlkst-python-budget branch from a141233 to 739ac73 Compare November 5, 2026 03:54
@chaliy
chaliy merged commit 876421f into main Nov 5, 2026
57 checks passed
@chaliy
chaliy deleted the claude/project-thread-pxlkst-python-budget branch November 5, 2026 04:31
@chaliy chaliy mentioned this pull request Nov 10, 2026
2 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant