Skip to content

spec: graphics substrate + img element - #140

Draft
natemoo-re wants to merge 4 commits into
mainfrom
spec/graphics
Draft

natemoo-re wants to merge 4 commits into
mainfrom
spec/graphics

Conversation

@natemoo-re

Copy link
Copy Markdown
Member

Focused feature specification for the graphics substrate and its first element, fulfilling renderer-spec §7.8's deferred "Kitty graphics emission" slot. Builds on the terminfo foundation stack (#131–#133) — doc-only PR, no code, no behavior change.

Contents

specs/graphics-spec.md (new, renamed during drafting from the original img-element framing):

  • Substrate (§4, §5) — a per-Term registry of caller-owned pixel surfaces (decoded RGBA8 via term.setImage/term.removeImage), plus the engine that downsamples surfaces through a fidelity ladder. Memory is normative no-malloc: fixed-capacity entry table (256), compaction-only pixel pool (imagePoolBytes option, 4 MiB default), double-buffered placements tables (64), O(1) per-frame scratch, closed RangeError failures. setImage/removeImage MUST NOT grow linear memory (output-view safety).
  • Tier engine (§9) — deterministic resolution: kitty → ascii → alt, evidence-gated on RuntimeCapabilities.kittyGraphics (probe query 9), per-directive variant override gated the same way. Kitty tier: lazy chunked RGBA transmission (≤4096 base64 bytes, multiples of 4, m=1/m=0, q=2 everywhere), delete-before-re-transmit per the protocol, cursor-placed letterbox-free placements via same-pair (image id, placement id) replacement, scoped lowercase d=i,…,p=… deletions for stale placements, data-freeing d=I removal. Placement ids are a pinned FNV-1a derivation with same-frame collision demotion. Ascii tier: shape-vector glyph selection per Alex Harri's ASCII rendering — six staggered internal sampling circles, ten external, directional-then-global contrast enhancement, 95-character nearest-match set, build-time baked shape vectors, golden-test-locked calibration. Immediate cleanup bytes from update() on capability denial and resize (TINV-5); line mode capped at ascii.
  • Layering (§4.5) — the substrate is element-agnostic and the renderer never grows a drawing API: video (per-frame revision bumps; kitty animation frames deferred as the fast path), svg/3D (TS-side rasterizer producers), and canvas (next focused spec: directive + pure-TS 2D context over zero-copy pool views, sound because INV-I8 fixes the carve) are all consumers of this substrate, not renderer features.
  • Appendix A records the amendments applied; Open Decisions holds the nine deliberately unresolved calls (candidate-set curation, calibration constants, alpha weighting, pool policy, APC passthrough/mux, terminal-reset desync, …).

Amendments applied (Appendix A): renderer-spec §4.3 retained-state wording, §7.6 cursor carve-out, new §8.3.5 img(), §7.8 reference, §8.1 imagePoolBytes, §7.7 update-bytes note; terminfo-spec §10.2 restatement. Current-surface notes (A5/A6: opcode + error types in §12) phase in with the implementation PR so §12 never describes unbuilt state.

Two corrections found while preparing the implementation

  1. Clip containment gate (§8.5, §9.1) — a partially clipped placement would letterbox inside its c=/r= box and claim cells its pixels do not fill (INV-I4 violation). The kitty tier now requires the effective footprint to be fully contained in the effective clip region; failure falls through to ascii (same footprint, no re-measurement), which also resolves scroll containers.
  2. Placements record double-buffering (§5.2, §5.4) — explicit, like the cell buffers: stale detection diffs front/back tables.

Not here

No C/TS changes, no tests, no changeset — the implementation stack follows: spec (this PR) → implementation (C/WASM substrate) → img (TS surface + test suite + changeset).

Review guidance: the load-bearing protocol facts (re-transmission deletes placements; C=1 semantics; d= deletion forms; letterboxing; id 0 illegal; chunk multiples of 4) were verified against the kitty graphics protocol spec and are quoted or paraphrased in §9.2.

@bombshell-cooper

Copy link
Copy Markdown
Contributor

No changeset needed

This pull request is classified as feature: no published package

Focused feature spec for the void img() directive: a term-level
image registry (setImage/removeImage), a fidelity ladder of kitty
graphics, shape-aware ascii art, and alt text, resolved
deterministically from capability evidence.

Kitty tier: lazy chunked RGBA transmission with delete-before-
re-transmit, cursor-placed letterbox-free placements via same-pair
replacement, scoped lowercase deletions for stale placements,
data-freeing uppercase removal, immediate cleanup bytes from
update() on capability denial and resize, and transmission-state
resets grounded in the protocol's storage-quota rules.

Ascii tier: per-character shape vectors over six staggered internal
sampling circles with ten external circles, per-component max
normalization, directional-then-global contrast enhancement, and
95-character nearest-match glyph selection, all baked at build
time and golden-locked.

Memory: no-malloc arena discipline — fixed-capacity entry table,
compaction-only pixel pool, placements table, and O(1) per-frame
scratch, carved at Term creation with closed RangeError failures.

Includes amendments appendix (A1-A10), testing obligations, and
open decisions. Synthesized via a three-candidate design arena;
protocol facts verified against the kitty graphics protocol
documentation.
Rename to specs/graphics-spec.md and reframe: the pixel-surface
registry plus the tier engine is the substrate (§4.5), img is its
first consumer, and video/svg/3d/canvas are producers or later
focused specs on the same substrate - the renderer stays a
transcoder and never grows a drawing API. Canvas's zero-copy
surface views lean on INV-I8's fixed carve; kitty animation frames
are the deferred fast path for video. No normative changes to the
arena-synthesized contract.
Apply the approved amendments to renderer-spec (retained state,
cursor carve-out, img constructor, capability-consumption
reference, imagePoolBytes option, update-bytes note) and
terminfo-spec (createTerm restatement); the appendix becomes an
applied-record, with current-surface notes (A5/A6) phased to the
implementation PR.

Two spec corrections found while preparing the implementation:
the kitty tier gains a containment gate (a partially clipped
placement would letterbox inside its box and claim cells its
pixels do not fill, violating INV-I4 - falls through to ascii
instead, which also resolves scroll containers), and the
placements record is explicitly double-buffered like the cell
buffers (stale detection diffs the two). Test file renamed to
test/graphics.test.ts with the spec.
@pkg-pr-new

pkg-pr-new Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@bomb.sh/tty@140

commit: 5b92fb3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant