Add a Lua json global, bound its work, and rename the Plugin vocabulary by vinniefalco · Pull Request #155 · cppalliance/runvil

vinniefalco · 2026-10-18T18:26:13Z

This PR holds three changes.

The first adds a reserved json Engine global to every section VM and Plugin prelude, so section code can turn JSON text into Lua values and back without loading anything. It follows the plan in vibe/2026-10-18-1-json-global.md. Encoding is strict and gives the same text for the same data, decoding handles one document or a whitespace-separated stream, and a list can stay a list when it is empty. Fixes #146.

The second renames the Plugin, tool, and model vocabulary so "package" is free for the unit Plugins ship in, which the dynamic-library work adds next. It follows the plan in vibe/2026-10-18-2-plugin-registry-rename.md. It is a pure rename: no behavior change and no change to recorded runs.

The third removes debt that the first two added, found by reviewing the six commits against upstream/master. It follows the plan in vibe/2026-10-19-1-json-debt-removal.md. Frozen argv now keeps JSON nulls, so json.encode(argv) returns the stored data. json.encode, json.encode_lines, and var writes now stop with an error when a value would expand far past the Lua heap, where they used to be able to take down the Host.

The json global

What `json` holds

json.encode(value) writes compact JSON on one line with object keys sorted. It is strict by default and raises an error naming the path to the first value it cannot represent, as in value.items[3]: a function, a non-finite number, invalid UTF-8, a table that contains itself, or a table whose keys make neither a list nor an object. json.encode(value, { debug = true }) writes a placeholder such as [function] or [cycle] in place of each failure instead, so any value can be dumped. debug is the only option.
json.encode_lines(list) writes each item followed by a newline, for JSON Lines, and names a failing item under the list path, as in value[2].name.
json.decode(text) parses one document, and json.decode_lines(text) parses a stream of values separated by any whitespace and returns them as a list. Integers in the i64 range become Lua integers, and every other number becomes a float.
json.null is one shared null value. Every decoded JSON null is this value, and it is truthy.
json.array(t) marks a table as a list, so it encodes as [] when empty where a plain empty table encodes as {}. Decoded lists already carry the mark.
The encoder reads through the guarded var table, frozen argv, sealed Plugin globals, and fanout results to the data behind them. It never calls back into Lua, so no __tostring, __pairs, or __eq runs, and output never contains a memory address.
Argument, size, and parse errors from these functions raise as kind lua and name the function.

Changes callers will notice

Empty lists in var and argv. An empty list stored in var or frozen argv now reads back as [] instead of {}. This includes prose substitution, so {{ var.items }} for an empty list renders [].
Reserved name. json joins the Engine's reserved names, so a Plugin prelude can no longer define a global called json.
Limits. json.decode and json.decode_lines refuse input over 8,388,608 bytes before parsing starts. Encode and decode both allow at most 127 levels of nesting and fail at 128.
Facade. The json global adds nothing to the public API of the runvil facade.

The Plugin vocabulary rename

What is renamed

PackageRegistry is now PluginRegistry, in harness::plugin.
Package is now PluginInfo, in runvil_plugin, and its name field is now fqn. A Plugin's vendor/name, such as runvil/web, is its fully qualified name (FQN), and the one-segment name a Host installs it under, such as web, is its Plugin id. Each plugin-* crate exports PLUGIN instead of PACKAGE.
ToolDescriptor is now ToolInfo, and ModelDescriptor is now ModelInfo, in the runvil facade and runvil_plugin. ToolBinding::from_descriptor is now from_info.
PluginKind::Native is now PluginKind::Builtin, so "native" means only compiled code.
The InstallError variants are now InvalidFqn, DuplicatePlugin, UnknownPlugin, ReservedName, and NameTaken.
The AGENTS.md Plugin entry now defines FQN, Plugin id, Plugin registry, and package. It also says a Host installs each Plugin once, with plugin-mcp as the one exception until its redesign. The code does not enforce that rule yet.

Changes callers will notice

Names. Hosts and Plugin crates must use the new type, constant, variant, and field names. Workshop is updated in this PR. Paperweight uses PackageRegistry through its vendored copy of runvil and updates when it next takes the vendored code.
Messages. The five InstallError messages use the new vocabulary. When plugins.get or plugins.offer refuses an argument, it now says "takes a Plugin id" instead of "takes a Plugin name". No other model-facing text changes.
Logs and Debug output. Workshop's add-failure warning now reads "the Plugin was not added to the registry". The construct-failure warning names the FQN in a field called fqn. Debug output prints the new type and field names.
Facade. crates/runvil/public-api.txt changes only its 14 ToolDescriptor lines and 17 ModelDescriptor lines.
Recorded runs. No serialized field or format changes, so recorded runs replay unchanged. ToolInfo keeps every serde field, and ModelInfo is not serialized.

The json debt removal

What is fixed

Frozen argv nulls. Outside the H1 body, a JSON null nested in the argument string used to read as nil, which left a hole in a list. For {"ids":[1,null,3]}, json.encode(argv) raised value.ids is missing list item [2], and for [1,null] or {"a":null} it dropped items without an error. A nested null now reads as json.null, the same value the H1 body and var already give, so json.encode(argv) returns the data. A whole argument string of null still makes argv nil.
Shared tables in json.encode. The encoder walked a table once for every place it appeared and copied every string occurrence, with no size limit, inside one Rust call that the cancel hook cannot interrupt. local t = {} for i = 1, 40 do t = { t, t } end return json.encode(t) asked for about 2^41 values on the Rust heap. Each call now spends a budget of one eighth of the Lua memory limit in values and the limit itself in bytes of text, which is 8,388,608 values and 67,108,864 bytes by default, and it raises json.encode: value is too large to encode when the budget runs out. A value that fits the Lua heap once is never refused for its size.
Long lists and big objects. The encoder held a reference for every entry of a table at once, and mlua panics when a Lua stack runs out of reference slots, which lands in the Host. A list of 1,200,000 strings reached it. The encoder now reads each value as its turn comes and allows at most 100,000 string or other non-number keys on one path.
var writes. var.x = t converted a shared table once per occurrence before any Lua allocation. A write now spends the same budget first and raises var.x is too large to store.
View chains. The encoder checks a chain of sealed views for a loop with a set instead of a linear scan.

Changes callers will notice

Frozen argv. argv.a for {"a":null} and argv.ids[2] for [1,null,3] now read json.null, which is truthy, instead of nil, and pairs and # see those slots. A prompt that tested a null argument with if argv.a then needs argv.a == json.null.
New errors. json.encode: value is too large to encode, also with the json.encode_lines: prefix and with no path, and {path} is too large to store. Debug mode raises the first one too.
Facade. Nothing changes in the public API of the runvil facade.

Commits

4453bc29a Add a Lua json global and keep empty var lists as arrays. Installs json with null, array, decode, and decode_lines, reserves the name, shares the table with Plugin preludes, and keeps the list mark on lists stored in var.
600a83428 Add json.encode and json.encode_lines to the json global. Adds the strict and debug encoder, moves the decoder under json/, and keeps the list mark on lists in frozen argv.
fd91a35b1 Close plan: json-global
0988c99f8 Rename tool and model descriptors to ToolInfo and ModelInfo. Renames the two types and the file that defines ToolInfo, updates the facade re-exports and every helper, test, and doc that named them, and re-blesses public-api.txt.
29051b256 Free the word package from the Plugin vocabulary. Renames Package, PACKAGE, PackageRegistry, PluginKind::Native, and the InstallError variants, rewords the messages, moves packages.rs to registry.rs, and writes the new AGENTS.md Plugin entry.
51c4b234e Close plan: plugin-registry-rename
268ef5cbc Read nested JSON nulls in frozen argv as json.null. Stores json.null for nested nulls in frozen argv and keeps a top-level null as nil.
41c3e38d3 Bound JSON encoding work by the Lua heap size. Adds the value and byte budget, sized from the VM's memory limit, to json.encode and json.encode_lines.
6c3d44718 Limit held keys and read encoder values lazily. Caps the keys held on one path and reads each value when its turn comes.
c5d883452 Refuse oversized var writes before converting them. Runs the same budget over a value before var converts it.
fc4a1c0ed Close plan: json-debt-removal

Each commit message lists its design changes and repairs.

Testing

For the json global, every gate passed locally on Windows: cargo fmt --all --check, both clippy runs with warnings denied, the rustdoc builds with -D warnings, the full nextest suites (5205, 310, and 18 tests passing), cargo test -p build-xtask, the docs-claims wording scan, cargo +nightly-2026-09-05 xtask api --check, and the 1.95 build at the rust-version floor.

For the rename, the same gates passed again at 29051b256 on Windows: formatting, both clippy runs, the rustdoc builds, the full nextest suites (5205 and 310 tests passing), cargo test -p build-xtask, the docs-claims scan, and xtask api --check. Searches for every old name also come back empty outside vibe/ plan records. The pre-push hook passed, including cargo deny check.

For the debt removal, the full verification passed on Windows at the final code state: the gateway build, the format check, both clippy runs with warnings denied, the headless gateway check, the rustdoc builds, the full nextest suites (5222 and 310 tests passing), cargo test -p build-xtask, the docs-claims scan, and the facade surface check, which reports no change. The new tests include a 1,200,000-item list that made mlua panic before the change. The pre-push hook passed, including cargo deny check.

CI passed on the first six commits. The debt-removal commits get their first Linux and macOS run on this push.

Section code can now decode JSON text into Lua values through a reserved built-in JSON table, one document at a time or as a whitespace-separated stream, and Plugin preludes can read the same table. Decoded nulls become one shared null value and decoded lists carry a list marker that authors can also put on their own tables, so an empty list is sent as a JSON array rather than an object. Values stored in the section's variable table now keep that marker, so an empty list there reads back as an empty array instead of an empty object. - `crate::json::install` runs last in the persistent Engine-global setup, after the store table, so every section VM has `json` beside `log` and `store`. - `install` builds `json` as a plain table, like `string`. Null and list detection use the values themselves, never fields of `json`, so a reassigned field cannot break them. - `array` marks a list with mlua's array metatable, and every conversion through mlua's `from_value` already encodes that as a JSON array, so `json.array()` sends `[]` where a plain empty table sends `{}`. It returns a marked table unchanged and refuses a table with any other metatable, so guarded and sealed tables keep theirs. - `decode` and `decode_lines` parse with `serde_json` into a tree on the Rust heap, outside the Lua memory limit, then convert it with mlua's `to_value`. `INPUT_LIMIT` refuses input over 8,388,608 bytes before parsing starts. - `VISIBLE_GLOBALS` gains `json`, and each Plugin prelude captures it from `_G` when the prelude installs, so preludes and section code share one mutable table. - `RESERVED_NAMES` lists `json` as an Engine global, so a Plugin prelude can no longer define a global with that name. - `guarded_json_value` puts the array metatable on the data table of each JSON array stored in `var`, and `materialize_guarded_value` copies it to the plain copy, so an empty list reads back as `[]`, not `{}`. - `json.null` is mlua's null light userdata. `type` reports `userdata`, the value is truthy, and every decoded JSON null is this value. - `decode` gives a Lua integer for an integer in the i64 range and a float for every other number. It accepts 127 nested levels and raises the `serde_json` recursion-limit error at 128. - `decode_lines` splits values on any whitespace, so several values can share a line and one value can span lines. It returns a marked list, empty for blank text, and its error positions count lines across the whole text. - `refusal` raises each argument, size, and parser error from `json.array`, `json.decode`, and `json.decode_lines` as kind `lua` and names the function, but a memory-limit error from `to_value` passes through unchanged. - `tasks.fanout` over `{ json.null }` runs one arm whose `item` is `json.null`. A read of `prose` whose placeholder names a global holding `json.null` raises the `NullValue` failure, and an H1 `argv` field set to `json.null` passes the freeze and reads as nil. - `decode` does not wrap strings in the untrusted envelope again, so a body cut from an envelope decodes with the envelope's `<` escapes still in place. - `json` holds only `null`, `array`, `decode`, and `decode_lines`, and has no function that encodes Lua values as JSON text. Design: new surface-growth @ crates/runvil-internal/lua/src/json.rs::install deps: Lua boundary: pub Design: new pure-function @ crates/runvil-internal/lua/src/json.rs::refusal deps: String Design: new pure-function @ crates/runvil-internal/lua/src/json-decode.rs::input deps: Value,str Design: extends surface-growth @ crates/runvil-internal/lua/src/globals.rs::RESERVED_NAMES boundary: pub Design: extends shared-mutable-state @ crates/runvil-internal/lua/src/prelude.rs::VISIBLE_GLOBALS Design: extends temporal-coupling @ crates/runvil-internal/lua/src/prelude.rs::VISIBLE_GLOBALS Repairs: var round-trips an empty JSON array @ crates/runvil-internal/lua/src/sys.rs::guarded_json_value - an empty list seeded into var read back as {} Plan: vibe/2026-10-18-1-json-global.md

Section code and Plugin preludes can now turn Lua values into compact JSON text, one value at a time or one value per line, with object keys sorted so the same data always gives the same text. Encoding is strict by default and raises an error naming the path to the first value it cannot represent, such as a function, a non-finite number, invalid UTF-8, a table that contains itself, or a table whose keys make neither a list nor an object. An opt-in debug mode writes a placeholder string in place of each failure, so any value can be dumped. The encoder reads through the guarded variable table, frozen arguments, sealed Plugin globals, and fanout results to the data behind them, and frozen arguments now keep the list marker, so an empty list there encodes as an empty array. - `Encoder` builds a `serde_json` tree on the Rust heap and writes it in one pass, so object keys come out sorted. It keeps the tables on the current path, compared by pointer, for cycle detection, and the keys on that path for error messages, which start at `value`, as in `value.items[3]`. - `resolve` reads through any table with no entries of its own whose metatable has both `__index` and `__metatable`, relying on the proxy layout that `var`, frozen `argv`, sealed prelude globals, and fanout results share rather than on a declared interface. A table `__index` is encoded in the view's place, and a function `__index`, as on `sys` and a prelude's `var` view, raises a cannot-read error telling the author to copy the fields into a plain table. - `crates/runvil-internal/lua/src/json/keys.rs` owns key order and table shape. Keys are visited booleans first, then numbers by value, then strings bytewise, then other key types by type name, with non-UTF-8 strings placed by their bytes and infinities by value, so one table reports the same error however it was built. - `crates/runvil-internal/lua/src/json/decode.rs` is the decoder moved unchanged into standard module layout under `json/`, replacing its `#[path]` attribute. - `Encoder::new` takes the `debug` flag, which swaps each failure for a placeholder string such as `[function]`, `[cycle]`, `[too deep]`, or `[cannot read: sys is sealed]`, and turns a table that breaks the key rules into an object with text keys. Option errors and `json.encode_lines` argument errors still raise in debug mode. - `json.encode` checks its options before the value and writes compact JSON on one line. A whole float from -2^63 up to but not including 2^63 encodes as an integer, so `2.0` is `2` and `-0.0` is `0`, while `2^63` stays a float. - `shape` checks a table's keys before any of its values, reporting in order a bad key type, a mix of list items and named keys, then the first missing list item. An unmarked table is `{}` when empty, a list when its keys are exactly 1 to n, and an object when every key is a string. - `DEPTH_LIMIT` allows 127 nested tables, the depth `json.decode` accepts, and reading through a sealed view adds no level. The cycle check runs first, so a loop of 127 tables reports the cycle, while one table shared in two places encodes normally. - `json.encode_lines` writes each item followed by a newline, so an empty list gives an empty string. It refuses a non-table and an unmarked table with no integer keys, and names a failing item under the list path, as in `value[2].name`. - `frozen_json_value` puts mlua's array metatable on each list's data table, so an empty list in frozen `argv` encodes as `[]`. - `Encoder` never calls back into Lua, so no `__tostring`, `__pairs`, or `__eq` runs, a userdata with `__tostring` still shows as `[userdata]` in debug mode, and output never contains a memory address. - `debug_option` accepts no key but `debug`, so there is no pretty-print mode and `{ pretty = true }` raises `unknown option pretty`. Design: extends surface-growth @ crates/runvil-internal/lua/src/json.rs::install deps: Lua boundary: pub Design: new flag-parameter @ crates/runvil-internal/lua/src/json/encode.rs::Encoder::new Design: new hidden-dependency @ crates/runvil-internal/lua/src/json/encode.rs::resolve deps: Table Design: new pure-function @ crates/runvil-internal/lua/src/json/encode.rs::finite deps: f64 Design: new pure-function @ crates/runvil-internal/lua/src/json/encode.rs::non_finite_text deps: f64 Design: new pure-function @ crates/runvil-internal/lua/src/json/keys.rs::shape deps: Value,bool Design: new pure-function @ crates/runvil-internal/lua/src/json/keys.rs::missing_item deps: Value,usize Design: new pure-function @ crates/runvil-internal/lua/src/json/keys.rs::position deps: Value Design: new pure-function @ crates/runvil-internal/lua/src/json/keys.rs::compare deps: Position Plan: vibe/2026-10-18-1-json-global.md

fixes cppalliance#146 Plan: vibe/2026-10-18-1-json-global.md

The types that hold a tool's or a model's static data are now called tool info and model info instead of descriptors, so each name says what the value is. The functions, fields, test helpers, test names, locals, and comments that named them follow. The serialized tool fields are unchanged and the model type is still never serialized, so recorded runs read as before. The public API listing is regenerated to match, and the model type's documentation now points out the two gateway types that share its new name. - `ToolInfo` replaces `ToolDescriptor`, and its file moves from `tools/descriptor.rs` to `tools/info.rs`. The `runvil::tools` facade, the `runvil_plugin` re-exports, `ToolCatalog`, and `Plugin::tools` now name only the new type. - `ModelInfo` replaces `ModelDescriptor` in `runvil::model`, `ModelCatalog`, `ModelBindings`, and `RunContext::model` and `current_model`. Its doc gains a paragraph naming `gateway_api_types::ModelInfo` and `gateway_local::gguf::ModelInfo` as different types, written as plain code spans rather than doc links. - `ToolBinding::from_info` replaces `from_descriptor`, `FnTool::info` replaces `FnTool::descriptor`, and the `Listed` field `descriptor` in `plugin-mcp` becomes `info`. Test helpers, test names, locals, and comments that said descriptor now say tool info or model info. - `crates/runvil/public-api.txt` is regenerated: its 31 entries that named the old types now name the new ones in sorted position, and no other entry changes. - `ToolInfo` keeps every field and its serde derives, `#[serde(default)]` on `survives_stop` included, and `ModelInfo` still derives no serde traits, so serialized tool data reads as before. - `Debug` output derived for `ToolInfo`, `ModelInfo`, and the `plugin-mcp` `Listed` struct prints the new type and field names. - `#[test]` functions are renamed and their messages reworded, but no assertion condition is added, removed, or changed. Design: new shotgun-surgery @ crates/runvil-internal/types/src/tools/info.rs::ToolInfo boundary: pub Design: new shotgun-surgery @ crates/runvil-internal/types/src/models.rs::ModelInfo boundary: pub Design: replaces pure-function @ crates/runvil/tests/suite/support.rs::tool_info deps: Value,str was: crates/runvil/tests/suite/support.rs::descriptor Design: replaces pure-function @ crates/runvil-internal/lua/src/handles-tests.rs::tool_info deps: bool was: crates/runvil-internal/lua/src/handles-tests.rs::descriptor Design: replaces pure-function @ crates/runvil-internal/engine/src/execute/bindings.rs::tests::model_info deps: str was: crates/runvil-internal/engine/src/execute/bindings.rs::tests::descriptor Design: replaces pure-function @ crates/runvil-internal/types/src/tools/tests.rs::inspect_info was: crates/runvil-internal/types/src/tools/tests.rs::inspect_descriptor Design: replaces pure-function @ crates/runvil-internal/types/src/tools/tests.rs::catalog_info deps: str was: crates/runvil-internal/types/src/tools/tests.rs::catalog_descriptor Plan: vibe/2026-10-18-2-plugin-registry-rename.md

The registry of installable Plugins, the static data each Plugin crate exports, and the constant that holds it were all named for packages, which left no word for the unit Plugins ship in. They now take Plugin names: the registry is the Plugin registry, the static data is Plugin info, and a Plugin's vendor and name pair is its fully qualified name. The kind for Plugins the Engine will build is now built-in rather than native, so native keeps its meaning of compiled code. Text that meant the one-segment install name now says Plugin id. Nothing installs or runs differently; only names, error and warning text, and debug output change. - `PluginRegistry` replaces `PackageRegistry` and moves from `packages.rs` to `registry.rs`. It still holds the built-in `runvil/subagents` from the start, and `default_id` replaces `package_name` for deriving a default Plugin id from the FQN's second segment. - `PluginInfo` replaces `Package` in `runvil-plugin`, and its `name` field becomes `fqn`. Each `plugin-*` crate exports it as `PLUGIN` instead of `PACKAGE`. - `PluginKind::Builtin` replaces `PluginKind::Native`, and the private `NATIVE`, `Native`, and `native` become `BUILTIN`, `Builtin`, and `builtin`. Renaming the public variant breaks any Host that names it. - `InstallError` renames `InvalidPackage`, `DuplicatePackage`, and `UnknownPackage` to `InvalidFqn`, `DuplicatePlugin`, and `UnknownPlugin`. Every variant that carried a `package` field now carries `fqn`. - `AGENTS.md` redefines a Plugin by its FQN and Plugin id. It reserves package for what Plugins ship in, never a Plugin. - `InstallError` messages now say FQN, built-in, and Plugin id. `NameTaken` reads "Plugin id {name} is taken by the installed {existing}; install it under another Plugin id". - `plugins.get` and `plugins.offer` argument errors say "Plugin id" where they said "Plugin name". The "no Plugin named" refusal keeps its words. - `fqn = info.fqn` replaces the `package` field on the construct-failure warning. Workshop's add failure now warns "the Plugin was not added to the registry". - `HostContext` debug output names its registry `plugins`, and `PluginInfo` debug output prints its own type name and `fqn`. - `HostContext::install` still accepts one FQN more than once, because `plugin-mcp` is installed once per MCP server. - `PluginInfo` and `PluginRegistry` derive no serde traits, so the rename touches no recorded format. Design: new shotgun-surgery @ crates/runvil-plugin/src/plugin.rs::PluginInfo boundary: pub Design: new shotgun-surgery @ crates/harness-internal/runner/src/registry.rs::PluginRegistry boundary: pub Design: replaces registry @ crates/harness-internal/runner/src/registry.rs::PluginRegistry boundary: pub was: crates/harness-internal/runner/src/packages.rs::PackageRegistry Design: new shotgun-surgery @ crates/plugin-user-input/src/lib.rs::PLUGIN boundary: pub Design: replaces pure-function @ crates/harness-internal/runner/src/registry.rs::default_id deps: str was: crates/harness-internal/runner/src/packages.rs::package_name Design: replaces pure-function @ crates/harness-internal/runner/src/registry.rs::builtin deps: HostServices,PluginId,Value was: crates/harness-internal/runner/src/packages.rs::native Design: replaces pure-function @ crates/harness-internal/runner/src/host-tests.rs::info deps: str was: crates/harness-internal/runner/src/host-tests.rs::package Design: replaces pure-function @ crates/harness-internal/runner/tests/it/prepare.rs::echo_plugin deps: Option,str was: crates/harness-internal/runner/tests/it/prepare.rs::echo_package Design: replaces pure-function @ crates/harness-internal/runner/tests/it/registry.rs::name deps: str was: crates/harness-internal/runner/tests/it/packages.rs::name Design: replaces pure-function @ crates/runvil-plugin/tests/it/plugin_info.rs::construct deps: HostServices,PluginId,Value was: crates/runvil-plugin/tests/it/package.rs::construct Design: replaces pure-function @ crates/runvil-plugin/tests/it/plugin_info.rs::host_wide was: crates/runvil-plugin/tests/it/package.rs::host_wide Design: replaces pure-function @ crates/runvil-plugin/tests/it/plugin_info.rs::installed_as deps: str was: crates/runvil-plugin/tests/it/package.rs::installed_as Plan: vibe/2026-10-18-2-plugin-registry-rename.md

Plan: vibe/2026-10-18-2-plugin-registry-rename.md

Frozen run arguments turned every nested JSON null into a missing value. Scripts read those fields as nil, and encoding the arguments failed on a list with a null in the middle. Nested nulls now read as the same null value that the writable repair-pass copy holds. Every section sees the same data, and encoding returns what was stored. A null or absent value at the top level still reads nil, so the malformed-input check keeps working. - `install_frozen` installs nil for an absent or top-level null value before it builds any table. The top-level malformed-input check stays as it was. - `frozen_json_value` takes `&Json` and has no nil arm. A nested null now reaches the scalar arm and converts the way `install_writable` converts it. - `argv.a` reads the null value instead of nil once frozen, and `argv.ids[2]` does the same for a list slot. The null value is truthy, so a script that tests a nested field for nil now takes the other branch. - `an_h1_argv_holding_json_null_passes_the_freeze_and_reads_the_field_as_json_null` flips from a nil read to a null value read. It pins that a null field set in the repair pass keeps its null in later sections. Repairs: frozen argv reads a nested null as json.null, as writable argv does @ crates/runvil-internal/lua/src/argv.rs::frozen_json_value - nested null fields and list slots read nil Repairs: json.encode(argv) returns the stored data when argv holds nested nulls @ crates/runvil-internal/lua/src/argv.rs::frozen_json_value - json.encode raised a missing list item error for a list holding null Plan: vibe/2026-10-19-1-json-debt-removal.md

Encoding a Lua value to JSON text now has a work budget. Before this change a table reachable from many places was walked once per reference, and a repeated string was copied once per occurrence. A small value in the Lua heap could ask for a result far larger than the heap, inside one native call that the instruction hook never interrupts. The walk now counts every value it visits and every byte of text it reads, sized from the heap limit, and refuses with a clear error once either count runs out. The refusal holds in debug mode too, and following a long chain of sealed views now costs time in proportion to its length. - `WalkBudget` counts the values a walk visits and the bytes of text it reads. It starts with one value per 8 bytes of the Lua heap limit and one byte of text per byte of that limit, and each count stops at zero instead of wrapping. - `HeapCeiling` records the VM's heap limit as Lua app data, because mlua has no getter for it. `vm.rs` sets it beside the startup memory limit and again in `apply_lua_limits`, and `for_vm` falls back to `DEFAULT_LUA_MEMORY_BYTES` when none is recorded. - `resolve` is now a method on `Encoder`, and each hop to another view costs one value of the budget. The views already passed sit in a `HashSet` instead of a `Vec`, so a long chain costs time in proportion to its length. - `json.encode` and `json.encode_lines` raise `json.encode: value is too large to encode`, or the same text under the `json.encode_lines` name, once a walk runs out of values or bytes of text. Shared tables and repeated strings count once per occurrence. - `too_large` builds that refusal with no path, and debug mode raises it instead of writing a placeholder. One placeholder per remaining value would not bound the work. - `value` charges one value for each value it visits, scalars included, and the byte length of each string. Object key text is charged by its byte length too. - `WalkBudget` has only the encoder as a caller in the touched files. Design: new newtype @ crates/runvil-internal/lua/src/walk_budget.rs::HeapCeiling Design: new hidden-dependency @ crates/runvil-internal/lua/src/walk_budget.rs::WalkBudget::for_vm deps: Lua Design: new surface-growth @ crates/runvil-internal/lua/src/json/encode.rs::Encoder::too_large boundary: pub Repairs: encoding work stays bounded by the Lua heap @ crates/runvil-internal/lua/src/json/encode.rs::Encoder::value - a table shared at every level of a 19-deep doubling encoded about 1 million values from 20 tables instead of raising Repairs: encoded text stays within the Lua heap's byte budget @ crates/runvil-internal/lua/src/json/encode.rs::Encoder::value - 32 references to one 512 KiB string encoded 16 MiB of text against a 4 MiB heap instead of raising Repairs: each hop through a sealed view counts against the budget @ crates/runvil-internal/lua/src/json/encode.rs::Encoder::resolve - 1,024 reads of a 1,000-view chain, about 1 million hops, encoded instead of raising Plan: vibe/2026-10-19-1-json-debt-removal.md

The JSON encoder used to collect every key and value of a table before encoding any of it, so each string or table in a large table held a Lua reference at the same moment. Now it collects only the keys, drops each value as soon as its pair is read, and fetches a value from the table when its turn comes. It also counts the keys that hold Lua references along the path being encoded and refuses the value once that count passes 100,000. The aim is to keep live references well under the Lua stack cap. Lists are exempt because their integer keys hold no reference. - `entries` returns the sorted keys and a count of reference-holding keys instead of key and value pairs, and gives `None` once that count passes the room it is given. Callers read each value with `raw_get` when its turn comes. - `held` tracks the keys that the tables on the current path hold as references. A table adds its count before encoding its children and subtracts it afterward, so finished siblings never count against each other. - `too_large` is now a free function so `debug_option` can raise the same refusal before the value is read. - `HELD_KEY_LIMIT` caps one path at 100,000 reference-holding keys, a tenth of the 1,000,000-slot Lua stack cap. An object with 150,000 string keys now raises the too-large error where it used to encode, and an options table with 100,001 keys raises it in place of the unknown-option error. - `keys_of` raises the refusal in debug mode too, because a table with that many keys cannot become a placeholder. - `items` adds nothing to `held`, since only integer keys reach its list and integer keys hold no reference. Design: new pure-function @ crates/runvil-internal/lua/src/json/encode.rs::too_large deps: str Design: new shared-parameter-cluster @ crates/runvil-internal/lua/src/json/encode.rs::Encoder::list Design: new shared-parameter-cluster @ crates/runvil-internal/lua/src/json/encode.rs::Encoder::object Design: new shared-parameter-cluster @ crates/runvil-internal/lua/src/json/encode.rs::Encoder::fallback Design: new pure-function @ crates/runvil-internal/lua/src/json/keys.rs::holds_reference deps: Value Plan: vibe/2026-10-19-1-json-debt-removal.md

Writing a value into a script's guarded variable table used to convert the whole value on the Rust heap before anything checked its size, so a table shared many times over could expand far past the Lua heap. The write now walks the value first, counts every value and every byte of text the conversion would read, and refuses with a too-large error once the count passes a budget sized from the heap limit. The walk leaves a table that contains itself, and tables nested very deep, to the conversion's own handling. A write of a cyclic value still reports the recursion error rather than the size refusal. Tests cover a shared table, a repeated string, a repeated key, and a cyclic table. - `check_value` walks a value one table entry at a time and charges every value it visits, repeats of a shared table included, plus the bytes of each string value and string key. It stops at the first charge past the budget. - `check_at` keeps the tables on its current path and returns without descending into one it already holds or one at a level past `CHECK_DEPTH_LIMIT`. Such a table still costs one value, and its entries are not read. - `guarded_table` raises a Lua runtime error `{target} is too large to store` when `check_value` fails. The check runs after the function, userdata, and thread rejection and before `from_value`. - `WalkBudget` is built fresh for each write from the VM's heap ceiling, so the limit applies to one write and not to several writes together. - `a_var_write_of_a_table_that_contains_itself_keeps_the_recursion_error` pins that a table holding itself twice still raises the conversion's recursive-table error and is not reported as too large. - `CHECK_DEPTH_LIMIT` leaves the entries of a table at level 201 or deeper unmeasured, so a value nested that deep reaches the conversion without a size check on them. - `map_while(Result::ok)` ends the walk of a table quietly when an entry cannot be read, and the conversion that runs next reads the same value. Design: new surface-growth @ crates/runvil-internal/lua/src/sys.rs::guarded_table deps: Lua,Table,Table,str boundary: pub Deferred: bounding the other from_value call sites outside sys.rs Repairs: var write work stays within the Lua heap's value budget @ crates/runvil-internal/lua/src/sys.rs::guarded_table - a table shared at every level of a 19-deep doubling converted about 1 million values on the Rust heap and then failed on a Lua memory error instead of raising the too-large refusal Repairs: var write text stays within the Lua heap's byte budget @ crates/runvil-internal/lua/src/sys.rs::guarded_table - 32 references to one 512 KiB string value or one 512 KiB key were copied 32 times, 16 MiB against a 4 MiB heap, and then failed on a Lua memory error instead of raising the too-large refusal Plan: vibe/2026-10-19-1-json-debt-removal.md

Plan: vibe/2026-10-19-1-json-debt-removal.md

vinniefalco added 6 commits October 18, 2026 10:10

Close plan: json-global

fd91a35

fixes cppalliance#146 Plan: vibe/2026-10-18-1-json-global.md

Close plan: plugin-registry-rename

51c4b23

Plan: vibe/2026-10-18-2-plugin-registry-rename.md

vinniefalco changed the title ~~Add a Lua json global~~ Add a Lua json global and rename the Plugin vocabulary Oct 19, 2026

vinniefalco added 5 commits October 19, 2026 06:50

Close plan: json-debt-removal

fc4a1c0

Plan: vibe/2026-10-19-1-json-debt-removal.md

vinniefalco changed the title ~~Add a Lua json global and rename the Plugin vocabulary~~ Add a Lua json global, bound its work, and rename the Plugin vocabulary Oct 19, 2026

vinniefalco merged commit fc4a1c0 into cppalliance:master Oct 19, 2026
19 checks passed

vinniefalco deployed to github-pages October 19, 2026 15:25 — with GitHub Actions

View deployment

vinniefalco deployed to github-pages October 20, 2026 07:37 — with GitHub Actions

View deployment

Navigation Menu

Repository navigation

Add a Lua json global, bound its work, and rename the Plugin vocabulary - #155

Add a Lua json global, bound its work, and rename the Plugin vocabulary#155
vinniefalco merged 11 commits into
cppalliance:masterfrom
vinniefalco:master

vinniefalco commented Oct 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

vinniefalco commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The json global

What json holds

Changes callers will notice

The Plugin vocabulary rename

What is renamed

Changes callers will notice

The json debt removal

What is fixed

Changes callers will notice

Commits

Testing

Related

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vinniefalco commented Oct 18, 2026 •

edited

Loading

What `json` holds