diff --git a/.gitignore b/.gitignore index 4196662..3271b5a 100644 --- a/.gitignore +++ b/.gitignore @@ -10,6 +10,8 @@ mojoh2 mojoh2tls mojoget mojo_http_server +snapshot/snapshot +screen/screen # editor/OS .DS_Store __pycache__/ diff --git a/CHAT_UI_TODO.md b/CHAT_UI_TODO.md index d71ddab..5b87d38 100644 --- a/CHAT_UI_TODO.md +++ b/CHAT_UI_TODO.md @@ -13,6 +13,7 @@ else. Not started; this captures the plan + the verified pieces it builds on. | `rusqlite` conversations + messages | **MOJO-libs `sqlite/`** — write + read + SELECT, verified vs real `sqlite3` (`integrity_check: ok`) | HAVE | | `serde_json` | **MOJO-libs `json/`** | HAVE | | `reqwest` → LLM (streaming) | **MOJO-libs `http/client.mojo`** — DNS, chunked, gzip/deflate, **https/TLS**, redirects | HAVE (SSE token-stream parse = small add) | +| clipboard copy/paste | **MOJO-libs `clipboard/`** — Wayland/X11 text clipboard + explicit OSC52 fallback | HAVE (Linux text clipboard; xsel round-trip verified) | | LLM backend | see "LLM endpoint" below | DECISION | ## Architecture @@ -21,6 +22,9 @@ state via `store_user_state`/`retrieve_user_state`): - **left panel** — conversation list (from `sqlite`: `conversations` table). - **center** — scrollable message history (`scroll_area` over `messages`) + a `text_area` input + Send button. +- **clipboard** — copy message text / generated paths via `clipboard.write_text`; + paste external prompt text via `clipboard.read_text` where the UI exposes a + paste action. - **on Send** — append user msg → `sqlite`; POST to the LLM endpoint via `http.client` (json body); stream/collect the reply; append assistant msg → `sqlite`; re-render. diff --git a/README.md b/README.md index 35ed2d9..0a3477d 100644 --- a/README.md +++ b/README.md @@ -6,16 +6,19 @@ TLS + WebSockets, JSON (with a high-performance tape parser), an async executor, (PNG/JPEG/WebP decode+encode, resize/filters, 16-bit/ICC/EXIF/CMYK), fast memory allocators (arena/pool/slab/ring), a SQLite-format database engine (read + SELECT + write, no FFI), and a full PDF 1.7 writer + reader -(embedded/subset fonts, encryption, digital signatures) — built from the ground up -on libc/OpenSSL/nghttp2/zlib/brotli via FFI. - -Mojo's standard library has no sockets, TLS, HTTP, JSON, or PDF. These libraries -fill that gap. The guiding principle is **"Mojo for everything; C only for the gaps -Mojo genuinely can't reach"** — and those gaps are small (C-ABI callbacks for -nghttp2 and OpenSSL ALPN; zlib's `z_stream`). Everything else — the event loop, -sockets, the HTTP/1.1 and WebSocket protocols, the JSON parser and validator, the -DEFLATE codec (both directions), and the PDF crypto stack (**MD5, SHA-1/256/384/512, -RC4, AES, RSA, ECDSA, big-integer math** — all pure Mojo) — is implemented here. +(embedded/subset fonts, encryption, digital signatures), plus Linux desktop +clipboard integration for Mojo apps — built from the ground up on +libc/OpenSSL/nghttp2/zlib/brotli via FFI. + +Mojo's standard library has no sockets, TLS, HTTP, JSON, PDF, or desktop +clipboard API. These libraries fill that gap. The guiding principle is **"Mojo +for everything; C only for the gaps Mojo genuinely can't reach"** — and those +gaps are small (C-ABI callbacks for nghttp2 and OpenSSL ALPN; zlib's `z_stream`; +desktop clipboard ownership delegated to the user's Wayland/X11 provider). Everything +else — the event loop, sockets, the HTTP/1.1 and WebSocket protocols, the JSON +parser and validator, the DEFLATE codec (both directions), and the PDF crypto +stack (**MD5, SHA-1/256/384/512, RC4, AES, RSA, ECDSA, big-integer math** — all +pure Mojo) — is implemented here. Toolchain: **Mojo 1.0.0b1 / MAX 26.3** (via [pixi](https://pixi.sh)). @@ -148,6 +151,17 @@ frame sequence (+ audio) into an mp4 — e.g. generated frames + generated audio (LTX2/NAVA). Verified: an mp3 decodes to samples and a frames+wav mux produces a valid **h264+aac** mp4 (independent `ffprobe` oracle). Requires `ffmpeg` on PATH. +### [`clipboard/`](clipboard/) — desktop clipboard for Mojo apps +Linux desktop clipboard helpers for app workflows like copying generated image +paths, prompts, logs, and user-selected text. Runtime provider detection covers +Wayland `wl-copy`/`wl-paste`, X11 `xclip`, and X11 `xsel`; OSC52 terminal +clipboard writes are available only when explicitly requested. Payload bytes +travel over provider stdin/stdout, not through shell-interpolated command strings. +The public API is intentionally small (`write_text`, `read_text`, `clear`, +`detect_backend`, `availability_report`) so apps can import one stable module +while backend support grows. Compile-safe tests always run; real clipboard +round-trip is opt-in because it mutates the user's clipboard. + ### [`svg/`](svg/) — pure-Mojo SVG icon loader (subset → raster) Parse an **SVG icon** and rasterize it to a `graphics.Canvas` (RGBA, transparent bg) → PNG or GPU texture. Pure Mojo on top of the `graphics` vector engine: full @@ -207,8 +221,9 @@ timeout, write backpressure, **streamed responses**, **prefork multi-core**, and ## Building -Everything builds from the repo root with `-I .` so the packages (`json`, `net`, -`http`, `async`) resolve. C shims compile to `.o` and link via `-Xlinker`. +Everything builds from the repo root with `-I .` so top-level packages (`json`, +`net`, `http`, `async`, `clipboard`, etc.) resolve. C shims compile to `.o` and +link via `-Xlinker`. ```bash # pure-Mojo libs need nothing extra: @@ -255,6 +270,9 @@ live local TLS server. A sampling of what is *measured*: confirming TOML-invalid input + non-ASCII strings are handled correctly. - **mem**: arena/pool/slab/ring allocators, 170 assertions, with microbenchmarks (pool 2.9×, slab 2.5×, arena bump+reset ~5.3× vs raw `alloc`/`free`). +- **clipboard**: compile-safe helper tests cover provider detection/reporting, + OSC52 sequence generation, and Base64 vectors; real Wayland/X11 round-trip is + opt-in with `CLIPBOARD_TEST_REAL=1` because it changes the user's clipboard. Known limits are documented per-lib and are version/scope choices, not design walls — e.g. the JSON codec's per-field *value* binding is one line per field diff --git a/clipboard/README.md b/clipboard/README.md new file mode 100644 index 0000000..c331e1a --- /dev/null +++ b/clipboard/README.md @@ -0,0 +1,104 @@ +# clipboard + +Desktop clipboard helpers for Mojo apps. + +The module is Linux-first and dependency-light: it links only libc from Mojo and +uses the desktop provider already present at runtime: + +- Wayland: `wl-copy` and `wl-paste` from `wl-clipboard` +- X11: `xclip` or `xsel` +- Terminal fallback: OSC52 write sequence, explicit only + +Payload text is sent over provider stdin/stdout. It is not interpolated into a +shell command, so paths, prompts, and generated text do not become shell input. + +## Modules + +| Module | What it is | +|---|---| +| `clipboard.mojo` | Public text clipboard API: `write_text`, `read_text`, `clear`, `detect_backend`, `backend_available`, `availability_report`, `osc52_sequence`, and `write_text_osc52`. Uses libc `popen`/`fread`/`fwrite` to stream payload bytes to trusted provider commands. | +| `tests/clipboard_test.mojo` | Compile-safe checks for backend names, selection names, provider reporting, Base64 vectors, and OSC52 sequences. With `CLIPBOARD_TEST_REAL=1`, runs a real Wayland/X11 round-trip and restores prior non-empty clipboard text. | + +## API + +```mojo +from clipboard.clipboard import ( + read_text, write_text, clear, detect_backend, availability_report, + SELECTION_CLIPBOARD, SELECTION_PRIMARY, +) + +def main() raises: + print(availability_report()) + write_text(String("/tmp/generated/image.png")) + var pasted = read_text() + print("clipboard:", pasted) + + write_text(String("primary selection"), SELECTION_PRIMARY) +``` + +Backends are selected automatically by `detect_backend()`: + +1. Wayland when `WAYLAND_DISPLAY` is set and `wl-copy`/`wl-paste` exist. +2. X11 `xclip` when `DISPLAY` is set and `xclip` exists. +3. X11 `xsel` when `DISPLAY` is set and `xsel` exists. + +Explicit backends are available through `BACKEND_WAYLAND`, `BACKEND_XCLIP`, +`BACKEND_XSEL`, and `BACKEND_OSC52`. + +## Backend Behavior + +| Backend | Read | Write | Selection support | Runtime requirement | +|---|---:|---:|---|---| +| Wayland | yes | yes | clipboard + primary | `WAYLAND_DISPLAY`, `wl-copy`, `wl-paste` | +| X11 `xclip` | yes | yes | clipboard + primary | `DISPLAY`, `xclip` | +| X11 `xsel` | yes | yes | clipboard + primary | `DISPLAY`, `xsel` | +| OSC52 | no | yes | clipboard + primary target codes | terminal that accepts OSC52 | + +`BACKEND_AUTO` chooses the first full read/write provider in that order. It does +not auto-select OSC52 because OSC52 is write-only and visibly emits terminal +escape sequences. + +## OSC52 + +OSC52 is write-only and terminal-dependent, so it is never selected +automatically: + +```mojo +from clipboard.clipboard import write_text_osc52 + +def main() raises: + write_text_osc52(String("copied through the terminal")) +``` + +Use it only for terminal apps where emitting an escape sequence to stdout or a +terminal writer is expected behavior. `write_text_osc52()` prints a convenience +sequence to stdout; use `osc52_sequence()` when your app needs exact byte control. + +## Limits + +- Text API only. UTF-8 is preserved byte-for-byte through the provider pipe. +- Clipboard persistence is owned by the provider. On some X11 setups, the + provider process may need a running X server until ownership is transferred. +- `read_text(max_bytes=...)` defaults to 64 MiB to protect apps from accidental + unbounded reads. +- No macOS/Windows backend yet. Add native providers behind the same public API + rather than changing app code. + +## Tests + +Compile-safe tests: + +```bash +pixi run mojo run -I . clipboard/tests/clipboard_test.mojo +``` + +Real clipboard round-trip is opt-in because it mutates the user's clipboard: + +```bash +CLIPBOARD_TEST_REAL=1 pixi run mojo run -I . clipboard/tests/clipboard_test.mojo +``` + +Observed validation on the development machine: + +- `pixi run --manifest-path /home/alex/mojodiffusion/pixi.toml mojo run -I . -I /home/alex/MOJO-libs /home/alex/MOJO-libs/clipboard/tests/clipboard_test.mojo` → `14 passed, 0 failed`. +- `CLIPBOARD_TEST_REAL=1 pixi run --manifest-path /home/alex/mojodiffusion/pixi.toml mojo run -I . -I /home/alex/MOJO-libs /home/alex/MOJO-libs/clipboard/tests/clipboard_test.mojo` → `xsel`, `16 passed, 0 failed`. diff --git a/clipboard/__init__.mojo b/clipboard/__init__.mojo new file mode 100644 index 0000000..7ea4179 --- /dev/null +++ b/clipboard/__init__.mojo @@ -0,0 +1,11 @@ +# clipboard - desktop clipboard helpers for Mojo apps. +# +# Public surface: +# from clipboard.clipboard import ( +# BACKEND_AUTO, BACKEND_WAYLAND, BACKEND_XCLIP, BACKEND_XSEL, +# BACKEND_OSC52, SELECTION_CLIPBOARD, SELECTION_PRIMARY, +# detect_backend, backend_available, backend_name, availability_report, +# read_text, write_text, clear, osc52_sequence, write_text_osc52, +# ) +# +# See clipboard/README.md for backend requirements and limits. diff --git a/clipboard/clipboard.mojo b/clipboard/clipboard.mojo new file mode 100644 index 0000000..4467daa --- /dev/null +++ b/clipboard/clipboard.mojo @@ -0,0 +1,421 @@ +# clipboard.clipboard - Linux desktop clipboard helpers for Mojo apps. +# +# The library intentionally keeps payload bytes out of shell command strings: +# desktop provider commands are static, while clipboard data flows over stdin or +# stdout through libc popen/fread/fwrite. This avoids shell injection from app +# content and keeps the Mojo side dependency-light. Runtime providers: +# * Wayland: wl-copy / wl-paste +# * X11: xclip or xsel +# * OSC52: explicit write-only terminal escape fallback + +from std.ffi import external_call +from std.memory import alloc, UnsafePointer +from std.builtin.type_aliases import MutExternalOrigin + +comptime BytePtr = UnsafePointer[UInt8, MutExternalOrigin] + +comptime BACKEND_AUTO = 0 +comptime BACKEND_WAYLAND = 1 +comptime BACKEND_XCLIP = 2 +comptime BACKEND_XSEL = 3 +comptime BACKEND_OSC52 = 4 + +comptime SELECTION_CLIPBOARD = 0 +comptime SELECTION_PRIMARY = 1 + +comptime PIPE_CHUNK = 65536 +comptime DEFAULT_MAX_READ_BYTES = 64 * 1024 * 1024 +comptime B64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/" + + +def _cbuf(s: String) -> BytePtr: + """NUL-terminated copy of `s` for libc calls.""" + var n = s.byte_length() + var b = alloc[UInt8](n + 1) + var src = s.as_bytes() + for i in range(n): + b[i] = src[i] + b[n] = 0 + return BytePtr(unsafe_from_address=Int(b)) + + +def _bytes_to_string(data: List[UInt8]) -> String: + var n = len(data) + if n == 0: + return String("") + var buf = alloc[UInt8](n) + for i in range(n): + buf[i] = data[i] + var out = String(StringSlice(ptr=BytePtr(unsafe_from_address=Int(buf)), length=n)) + buf.free() + return out^ + + +def _string_to_bytes(s: String) -> List[UInt8]: + var out = List[UInt8]() + var b = s.as_bytes() + for i in range(s.byte_length()): + out.append(b[i]) + return out^ + + +def env_nonempty(name: String) -> Bool: + """True when an environment variable exists and is not empty.""" + var np = _cbuf(name) + var p = external_call["getenv", BytePtr](np) + np.free() + if Int(p) == 0: + return False + return p[0] != 0 + + +def _run_status(cmd: String) -> Int: + var cp = _cbuf(cmd) + var rc = Int(external_call["system", Int32](cp)) + cp.free() + return rc + + +def _safe_command_name(name: String) -> Bool: + if name.byte_length() == 0: + return False + var b = name.as_bytes() + for i in range(name.byte_length()): + var c = Int(b[i]) + var alpha = (c >= 65 and c <= 90) or (c >= 97 and c <= 122) + var digit = c >= 48 and c <= 57 + var punct = c == 45 or c == 46 or c == 95 + if not (alpha or digit or punct): + return False + return True + + +def command_exists(name: String) -> Bool: + """PATH probe for simple command names. + + Rejects shell metacharacters before calling `command -v`. + """ + if not _safe_command_name(name): + return False + return _run_status(String("command -v ") + name + String(" >/dev/null 2>&1")) == 0 + + +def _popen(cmd: String, mode: String) -> Int: + var cp = _cbuf(cmd) + var mp = _cbuf(mode) + var fp = external_call["popen", BytePtr](cp, mp) + cp.free() + mp.free() + return Int(fp) + + +def _pclose(fp_addr: Int) -> Int: + if fp_addr == 0: + return -1 + return Int(external_call["pclose", Int32](BytePtr(unsafe_from_address=fp_addr))) + + +def _read_cmd_bytes(cmd: String, max_bytes: Int = DEFAULT_MAX_READ_BYTES) raises -> List[UInt8]: + var fp_addr = _popen(cmd, String("r")) + if fp_addr == 0: + raise Error("clipboard: popen failed for read provider") + var fp = BytePtr(unsafe_from_address=fp_addr) + var buf = alloc[UInt8](PIPE_CHUNK) + var bp = BytePtr(unsafe_from_address=Int(buf)) + var out = List[UInt8]() + while True: + var n = external_call["fread", Int](bp, Int(1), Int(PIPE_CHUNK), fp) + if n <= 0: + break + if len(out) + n > max_bytes: + buf.free() + _ = _pclose(fp_addr) + raise Error( + "clipboard: provider output exceeded max_bytes=" + + String(max_bytes) + ) + for i in range(n): + out.append(buf[i]) + buf.free() + var rc = _pclose(fp_addr) + if rc != 0: + raise Error("clipboard: read provider exited with status " + String(rc)) + return out^ + + +def _write_cmd_bytes(cmd: String, data: List[UInt8]) raises: + var fp_addr = _popen(cmd, String("w")) + if fp_addr == 0: + raise Error("clipboard: popen failed for write provider") + var fp = BytePtr(unsafe_from_address=fp_addr) + var n = len(data) + var total = 0 + if n > 0: + var src = BytePtr(unsafe_from_address=Int(data.unsafe_ptr())) + while total < n: + var wrote = external_call["fwrite", Int]( + src + total, Int(1), Int(n - total), fp + ) + if wrote <= 0: + _ = _pclose(fp_addr) + raise Error("clipboard: provider stdin write failed") + total += wrote + var rc = _pclose(fp_addr) + if rc != 0: + raise Error( + "clipboard: write provider failed (status=" + String(rc) + ")" + ) + + +def _validate_selection(selection: Int) raises: + if selection != SELECTION_CLIPBOARD and selection != SELECTION_PRIMARY: + raise Error("clipboard: invalid selection " + String(selection)) + + +def backend_name(backend: Int) -> String: + if backend == BACKEND_AUTO: + return String("auto") + if backend == BACKEND_WAYLAND: + return String("wayland") + if backend == BACKEND_XCLIP: + return String("xclip") + if backend == BACKEND_XSEL: + return String("xsel") + if backend == BACKEND_OSC52: + return String("osc52") + return String("unknown") + + +def selection_name(selection: Int) -> String: + if selection == SELECTION_CLIPBOARD: + return String("clipboard") + if selection == SELECTION_PRIMARY: + return String("primary") + return String("unknown") + + +def backend_available(backend: Int) -> Bool: + """True when the named backend is usable in the current environment. + + OSC52 is write-only. It reports available when TERM is set; read_text() will + still reject it because terminal OSC52 has no portable read path. + """ + if backend == BACKEND_WAYLAND: + return ( + env_nonempty(String("WAYLAND_DISPLAY")) + and command_exists(String("wl-copy")) + and command_exists(String("wl-paste")) + ) + if backend == BACKEND_XCLIP: + return ( + env_nonempty(String("DISPLAY")) + and command_exists(String("xclip")) + ) + if backend == BACKEND_XSEL: + return ( + env_nonempty(String("DISPLAY")) + and command_exists(String("xsel")) + ) + if backend == BACKEND_OSC52: + return env_nonempty(String("TERM")) + return False + + +def detect_backend(selection: Int = SELECTION_CLIPBOARD) raises -> Int: + """Select the best full read/write provider for this process. + + Prefers Wayland when WAYLAND_DISPLAY is present, then X11 providers. OSC52 is + not auto-selected because it is write-only and visibly emits terminal escape + sequences. + """ + _validate_selection(selection) + if backend_available(BACKEND_WAYLAND): + return BACKEND_WAYLAND + if backend_available(BACKEND_XCLIP): + return BACKEND_XCLIP + if backend_available(BACKEND_XSEL): + return BACKEND_XSEL + return -1 + + +def availability_report() -> String: + var out = String("clipboard backends:") + out += String(" wayland=") + String(backend_available(BACKEND_WAYLAND)) + out += String(" xclip=") + String(backend_available(BACKEND_XCLIP)) + out += String(" xsel=") + String(backend_available(BACKEND_XSEL)) + out += String(" osc52=") + String(backend_available(BACKEND_OSC52)) + if not env_nonempty(String("WAYLAND_DISPLAY")) and not env_nonempty(String("DISPLAY")): + out += String(" (no WAYLAND_DISPLAY or DISPLAY)") + return out^ + + +def _resolve_backend(backend: Int, selection: Int) raises -> Int: + _validate_selection(selection) + if backend == BACKEND_AUTO: + var detected = detect_backend(selection) + if detected < 0: + raise Error( + "clipboard: no full read/write backend available; install " + + "wl-clipboard, xclip, or xsel and ensure WAYLAND_DISPLAY or " + + "DISPLAY is set. " + availability_report() + ) + return detected + var valid = ( + backend == BACKEND_WAYLAND + or backend == BACKEND_XCLIP + or backend == BACKEND_XSEL + or backend == BACKEND_OSC52 + ) + if not valid: + raise Error("clipboard: invalid backend " + String(backend)) + if not backend_available(backend): + raise Error( + "clipboard: backend " + backend_name(backend) + + " is not available. " + availability_report() + ) + return backend + + +def _write_cmd(backend: Int, selection: Int) raises -> String: + _validate_selection(selection) + if backend == BACKEND_WAYLAND: + var cmd = String("wl-copy --type text/plain") + if selection == SELECTION_PRIMARY: + cmd += String(" --primary") + return cmd^ + if backend == BACKEND_XCLIP: + if selection == SELECTION_PRIMARY: + return String("xclip -selection primary -in") + return String("xclip -selection clipboard -in") + if backend == BACKEND_XSEL: + if selection == SELECTION_PRIMARY: + return String("xsel --primary --input") + return String("xsel --clipboard --input") + raise Error("clipboard: backend " + backend_name(backend) + " is not a pipe writer") + + +def _read_cmd(backend: Int, selection: Int) raises -> String: + _validate_selection(selection) + if backend == BACKEND_WAYLAND: + var cmd = String("wl-paste --no-newline --type text/plain") + if selection == SELECTION_PRIMARY: + cmd += String(" --primary") + return cmd^ + if backend == BACKEND_XCLIP: + if selection == SELECTION_PRIMARY: + return String("xclip -selection primary -out") + return String("xclip -selection clipboard -out") + if backend == BACKEND_XSEL: + if selection == SELECTION_PRIMARY: + return String("xsel --primary --output") + return String("xsel --clipboard --output") + raise Error("clipboard: backend " + backend_name(backend) + " is not a pipe reader") + + +def write_text( + text: String, + selection: Int = SELECTION_CLIPBOARD, + backend: Int = BACKEND_AUTO, +) raises: + """Write UTF-8 text to the desktop clipboard. + + The payload is sent to the provider over stdin, never interpolated into a + shell command. + """ + var b = _resolve_backend(backend, selection) + if b == BACKEND_OSC52: + write_text_osc52(text, selection) + return + var data = _string_to_bytes(text) + _write_cmd_bytes(_write_cmd(b, selection), data) + + +def read_text( + selection: Int = SELECTION_CLIPBOARD, + backend: Int = BACKEND_AUTO, + max_bytes: Int = DEFAULT_MAX_READ_BYTES, +) raises -> String: + """Read UTF-8 text from the desktop clipboard.""" + var b = _resolve_backend(backend, selection) + if b == BACKEND_OSC52: + raise Error("clipboard: OSC52 is write-only; read_text is unsupported") + return _bytes_to_string(_read_cmd_bytes(_read_cmd(b, selection), max_bytes)) + + +def clear(selection: Int = SELECTION_CLIPBOARD, backend: Int = BACKEND_AUTO) raises: + """Clear the clipboard by making the selected backend own an empty string.""" + write_text(String(""), selection, backend) + + +def base64_encode(data: List[UInt8]) -> String: + """Base64 encoder used by OSC52. Kept local to avoid HTTP package coupling.""" + var b64s = String(B64) + var alpha = b64s.as_bytes() + var out = List[UInt8]() + var n = len(data) + var i = 0 + while i + 3 <= n: + var b0 = Int(data[i]) + var b1 = Int(data[i + 1]) + var b2 = Int(data[i + 2]) + out.append(alpha[(b0 >> 2) & 0x3F]) + out.append(alpha[((b0 & 3) << 4) | (b1 >> 4)]) + out.append(alpha[((b1 & 15) << 2) | (b2 >> 6)]) + out.append(alpha[b2 & 0x3F]) + i += 3 + var rem = n - i + if rem == 1: + var b0 = Int(data[i]) + out.append(alpha[(b0 >> 2) & 0x3F]) + out.append(alpha[(b0 & 3) << 4]) + out.append(UInt8(61)) + out.append(UInt8(61)) + elif rem == 2: + var b0 = Int(data[i]) + var b1 = Int(data[i + 1]) + out.append(alpha[(b0 >> 2) & 0x3F]) + out.append(alpha[((b0 & 3) << 4) | (b1 >> 4)]) + out.append(alpha[(b1 & 15) << 2]) + out.append(UInt8(61)) + return _bytes_to_string(out) + + +def base64_text(text: String) -> String: + return base64_encode(_string_to_bytes(text)) + + +def osc52_sequence(text: String, selection: Int = SELECTION_CLIPBOARD) raises -> String: + """Return an OSC52 escape sequence for terminals that support clipboard set. + + Clipboard selection uses target 'c'; primary selection uses target 'p'. + """ + _validate_selection(selection) + var target = String("c") + if selection == SELECTION_PRIMARY: + target = String("p") + return ( + String(chr(0x1B)) + String("]52;") + target + String(";") + + base64_text(text) + String(chr(0x07)) + ) + + +def _write_fd(fd: Int32, data: String) raises: + if fd != 1: + raise Error( + "clipboard: custom OSC52 fd writes are not available in this build; " + + "use osc52_sequence() with the app's own terminal writer" + ) + print(data) + + +def write_text_osc52( + text: String, + selection: Int = SELECTION_CLIPBOARD, + fd: Int32 = 1, +) raises: + """Emit an OSC52 clipboard write sequence to stdout by default. + + For exact byte control or non-stdout destinations, call osc52_sequence() and + write the returned string through the app's terminal layer. + """ + _write_fd(fd, osc52_sequence(text, selection)) diff --git a/clipboard/tests/__init__.mojo b/clipboard/tests/__init__.mojo new file mode 100644 index 0000000..3817b7b --- /dev/null +++ b/clipboard/tests/__init__.mojo @@ -0,0 +1 @@ +# clipboard tests package marker. diff --git a/clipboard/tests/clipboard_test.mojo b/clipboard/tests/clipboard_test.mojo new file mode 100644 index 0000000..111316b --- /dev/null +++ b/clipboard/tests/clipboard_test.mojo @@ -0,0 +1,103 @@ +from clipboard.clipboard import ( + BACKEND_AUTO, + BACKEND_OSC52, + BACKEND_WAYLAND, + BACKEND_XCLIP, + BACKEND_XSEL, + SELECTION_CLIPBOARD, + SELECTION_PRIMARY, + availability_report, + backend_available, + backend_name, + base64_text, + clear, + detect_backend, + env_nonempty, + osc52_sequence, + read_text, + selection_name, + write_text, +) + + +struct Tally(Movable): + var p: Int + var f: Int + + def __init__(out self): + self.p = 0 + self.f = 0 + + +def chk(mut t: Tally, cond: Bool, label: String): + if cond: + t.p += 1 + else: + t.f += 1 + print(" FAIL", label) + + +def _read_text_or_empty() -> String: + try: + return read_text() + except: + return String("") + + +def main() raises: + var t = Tally() + + chk(t, backend_name(BACKEND_AUTO) == "auto", "backend auto name") + chk(t, backend_name(BACKEND_WAYLAND) == "wayland", "backend wayland name") + chk(t, backend_name(BACKEND_XCLIP) == "xclip", "backend xclip name") + chk(t, backend_name(BACKEND_XSEL) == "xsel", "backend xsel name") + chk(t, backend_name(BACKEND_OSC52) == "osc52", "backend osc52 name") + chk(t, selection_name(SELECTION_CLIPBOARD) == "clipboard", "clipboard selection name") + chk(t, selection_name(SELECTION_PRIMARY) == "primary", "primary selection name") + + chk(t, base64_text(String("")) == "", "base64 empty") + chk(t, base64_text(String("M")) == "TQ==", "base64 one byte") + chk(t, base64_text(String("Ma")) == "TWE=", "base64 two bytes") + chk(t, base64_text(String("Man")) == "TWFu", "base64 three bytes") + + var seq = osc52_sequence(String("hello"), SELECTION_CLIPBOARD) + var expected = ( + String(chr(0x1B)) + String("]52;c;aGVsbG8=") + String(chr(0x07)) + ) + chk(t, seq == expected, "OSC52 clipboard sequence") + + var primary_seq = osc52_sequence(String("hi"), SELECTION_PRIMARY) + var expected_primary = ( + String(chr(0x1B)) + String("]52;p;aGk=") + String(chr(0x07)) + ) + chk(t, primary_seq == expected_primary, "OSC52 primary sequence") + + var report = availability_report() + chk(t, report.byte_length() > 0, "availability report nonempty") + print(report) + print("wayland", backend_available(BACKEND_WAYLAND), + "xclip", backend_available(BACKEND_XCLIP), + "xsel", backend_available(BACKEND_XSEL)) + + if env_nonempty(String("CLIPBOARD_TEST_REAL")): + var backend = detect_backend() + if backend < 0: + print("SKIP: no real clipboard backend available") + else: + print("real backend:", backend_name(backend)) + var before = _read_text_or_empty() + var payload = String("mojo-clipboard-roundtrip-2026-06-12\nline2") + write_text(payload) + var got = read_text() + chk(t, got == payload, "real clipboard round-trip") + + clear() + var empty = read_text() + chk(t, empty.byte_length() == 0, "real clipboard clear") + if before.byte_length() > 0: + write_text(before) + + print("") + print("clipboard:", t.p, "passed,", t.f, "failed") + if t.f != 0: + raise Error("clipboard tests FAILED") diff --git a/screen/README.md b/screen/README.md new file mode 100644 index 0000000..73fcd68 --- /dev/null +++ b/screen/README.md @@ -0,0 +1,98 @@ +# screen — Mojo screen snapshots + +Versatile screen capture for Mojo, with three sources: + +| backend | captures | output | status | +|---|---|---|---| +| **x11** | the X11 desktop (root window or a rect) | RGB → PNG | ✅ verified live (xwininfo + PIL) | +| **fb** | the Linux framebuffer `/dev/fb0` | RGB → PNG | decode unit-tested; live read needs `video` group | +| **vcsa** | a virtual-console text grid `/dev/vcsaN` | text | decode unit-tested; live read needs `tty` group | + +PNG output reuses the repo's `image` library; nothing else is third-party. + +## X11 (pixels) — the primary path + +Xlib's `Display*` is opaque and segfaults when passed back across Mojo FFI, so +the handful of Xlib calls live in a tiny C floor (`cshim/screen_shim.c`) — the +repo's "C for the gaps, Mojo for the rest" pattern. Build it once: + +```sh +gcc -shared -fPIC -O2 screen/cshim/screen_shim.c -o screen/cshim/screen_shim.so -lX11 +``` + +Then, from the repo root (so the default relative `.so` path resolves — or set +`SCREEN_SHIM_PATH=/abs/path/screen_shim.so`): + +```sh +mojo build -I . screen/cli.mojo -o screen/screen + +./screen/screen size # -> 4096x2160 +./screen/screen x11 -o shot.png # full screen +./screen/screen x11 --rect 100 100 800 600 -o w.png # a region +``` + +Library API: + +```mojo +from screen import x11_size, grab_x11, capture_x11_png + +var s = x11_size() # Size{w, h} +var img = grab_x11(0, 0, 0, 0) # image.Image (RGB); 0,0,0,0 = full screen +capture_x11_png("shot.png", 0, 0, 0, 0) # grab + write PNG +``` + +## Framebuffer (pixels, no X needed) + +```sh +./screen/screen fb -o console.png # reads /dev/fb0 (+ sysfs geometry) +``` + +Geometry is read from `/sys/class/graphics/fb0/{virtual_size,bits_per_pixel,stride}`. +32-bpp is decoded as BGRX, 24-bpp as BGR (the usual little-endian fbdev layout). +`/dev/fb0` is `root:video 0660`, so reading it needs the `video` group (or root); +otherwise `grab_fb()` fails loud. Under X the framebuffer may not reflect the live +desktop (X renders via the GPU) — `fb` is for console/headless capture. + +```mojo +from screen import grab_fb, fb_decode, fb_info, FbInfo +var img = grab_fb() # /dev/fb0 -> Image +var img2 = fb_decode(raw_bytes, FbInfo(w, h, 32, stride)) # decode a buffer +``` + +## Console text (vcsa) + +Snapshots the **text** on a Linux virtual console (the tty cell grid) — the text +complement to the framebuffer. + +```sh +./screen/screen vcsa # active console -> stdout +./screen/screen vcsa 1 -o tty1.txt +``` + +`/dev/vcsaN` format: `[rows, cols, cursor_col, cursor_row]` then `rows*cols` +cells of `[char, attr]`; the char byte is kept, attributes dropped, NUL → space. +It is `root:tty 0660` (needs the `tty` group) and only exists for the real Linux +console — not for X terminals, tmux, or SSH sessions. + +```mojo +from screen import grab_vcsa, vcsa_decode, render_text +var ts = grab_vcsa(0) # TextScreen{rows, cols, cursor, lines} +print(render_text(ts)) +``` + +## Tests + +```sh +mojo run -I . screen/tests/screen_test.mojo +``` + +4 unit tests for the framebuffer (BGRX→RGB, stride padding) and vcsa (cell grid → +text, short-buffer error) decode paths. The X11 path is verified live (a captured +PNG matches `xwininfo` dimensions and decodes in PIL as correct-color desktop). + +## FFI gotcha (1.0.0b1) + +`OwnedDLHandle` is destroyed at its last use (ASAP), which `dlclose`s the `.so` +out from under any function pointers obtained from it — so the *second* FFI call +segfaults. Every function here ends with `_ = lib^` to hold the handle alive past +all its calls. diff --git a/screen/__init__.mojo b/screen/__init__.mojo new file mode 100644 index 0000000..47b894e --- /dev/null +++ b/screen/__init__.mojo @@ -0,0 +1,14 @@ +# screen — pure-Mojo screen snapshots (X11 pixels, framebuffer, console text). +# +# from screen import grab_x11, capture_x11_png, x11_size +# from screen import grab_fb, fb_decode, grab_vcsa, vcsa_decode + +from screen.x11 import grab_x11, capture_x11_png, x11_size, shim_path, Size +from screen.framebuffer import ( + grab_fb, + capture_fb_png, + fb_decode, + fb_info, + FbInfo, +) +from screen.vcsa import grab_vcsa, vcsa_decode, render_text, TextScreen diff --git a/screen/cli.mojo b/screen/cli.mojo new file mode 100644 index 0000000..69c4c52 --- /dev/null +++ b/screen/cli.mojo @@ -0,0 +1,118 @@ +# screen.cli — runnable entrypoint for screen snapshots. +# +# mojo run -I . screen/cli.mojo size +# mojo run -I . screen/cli.mojo x11 [-o out.png] [--rect X Y W H] +# mojo run -I . screen/cli.mojo fb [-o out.png] +# mojo run -I . screen/cli.mojo vcsa [N] [-o out.txt] +# +# Build the X11 shim first and run from the repo root (default .so path is +# relative), or set SCREEN_SHIM_PATH=/abs/path/screen_shim.so: +# gcc -shared -fPIC -O2 screen/cshim/screen_shim.c -o screen/cshim/screen_shim.so -lX11 + +from std.sys import argv + +from screen.x11 import x11_size, capture_x11_png +from screen.framebuffer import capture_fb_png +from screen.vcsa import grab_vcsa, render_text + + +def _atoi(s: String) -> Int: + var bs = s.as_bytes() + var i = 0 + var neg = False + if len(bs) > 0 and Int(bs[0]) == ord("-"): + neg = True + i = 1 + var v = 0 + while i < len(bs): + var c = Int(bs[i]) + if c < ord("0") or c > ord("9"): + break + v = v * 10 + (c - ord("0")) + i += 1 + return -v if neg else v + + +def _usage(): + print("usage:") + print(" screen size") + print(" screen x11 [-o out.png] [--rect X Y W H]") + print(" screen fb [-o out.png]") + print(" screen vcsa [N] [-o out.txt]") + + +def main() raises: + var raw = argv() + var args = List[String]() + for i in range(len(raw)): + args.append(String(raw[i])) + + if len(args) < 2: + _usage() + return + + var cmd = args[1] + + if cmd == "size": + var s = x11_size() + print(String(s.w) + "x" + String(s.h)) + return + + if cmd == "x11": + var out = String("screen.png") + var x = 0 + var y = 0 + var w = 0 + var h = 0 + var i = 2 + while i < len(args): + var a = args[i] + if (a == "-o" or a == "--out") and i + 1 < len(args): + i += 1 + out = args[i] + elif a == "--rect" and i + 4 < len(args): + x = _atoi(args[i + 1]) + y = _atoi(args[i + 2]) + w = _atoi(args[i + 3]) + h = _atoi(args[i + 4]) + i += 4 + i += 1 + capture_x11_png(out, x, y, w, h) + print("wrote " + out) + return + + if cmd == "fb": + var out = String("screen.png") + var i = 2 + while i < len(args): + if (args[i] == "-o" or args[i] == "--out") and i + 1 < len(args): + i += 1 + out = args[i] + i += 1 + capture_fb_png(out) + print("wrote " + out) + return + + if cmd == "vcsa": + var n = 0 + var out = String("") + var i = 2 + while i < len(args): + var a = args[i] + if (a == "-o" or a == "--out") and i + 1 < len(args): + i += 1 + out = args[i] + elif _atoi(a) > 0 or a == "0": + n = _atoi(a) + i += 1 + var ts = grab_vcsa(n) + var text = render_text(ts) + if out.byte_length() == 0: + print(text) + else: + with open(out, "w") as f: + f.write(text) + print("wrote " + out + " (" + String(ts.rows) + "x" + String(ts.cols) + ")") + return + + _usage() diff --git a/screen/cshim/screen_shim.c b/screen/cshim/screen_shim.c new file mode 100644 index 0000000..b3a7b34 --- /dev/null +++ b/screen/cshim/screen_shim.c @@ -0,0 +1,80 @@ +/* screen_shim.c — minimal C floor for X11 screen capture. + * + * The repo's "C for the gaps, Mojo for the rest" pattern (cf. http/cshim, + * net/cshim): Xlib's Display* is opaque and its struct/macros don't pass + * cleanly across Mojo FFI, so the few Xlib calls live here. Mojo loads this + * .so, asks for an RGB24 buffer, encodes the PNG, and frees the buffer. + * + * Build: + * gcc -shared -fPIC -O2 screen_shim.c -o screen_shim.so -lX11 + * + * ABI (all C, thin): + * int screen_x11_size(int* w, int* h); // 0 ok, -1 no display + * unsigned char* screen_x11_grab(int x,int y,int w,int h,int* ow,int* oh); + * // RGB24, row-major, caller frees with screen_free; NULL on failure. + * // w<=0 || h<=0 => full screen. Rect is clamped to the screen. + * void screen_free(unsigned char* p); + */ + +#include +#include +#include + +int screen_x11_size(int *w, int *h) { + Display *d = XOpenDisplay(NULL); + if (!d) return -1; + int s = DefaultScreen(d); + *w = DisplayWidth(d, s); + *h = DisplayHeight(d, s); + XCloseDisplay(d); + return 0; +} + +static int mask_shift(unsigned long m) { + int sh = 0; + if (!m) return 0; + while (!(m & 1UL)) { m >>= 1; sh++; } + return sh; +} + +unsigned char *screen_x11_grab(int x, int y, int w, int h, int *ow, int *oh) { + Display *d = XOpenDisplay(NULL); + if (!d) return NULL; + int s = DefaultScreen(d); + Window root = RootWindow(d, s); + int sw = DisplayWidth(d, s), sh = DisplayHeight(d, s); + + if (w <= 0 || h <= 0) { x = 0; y = 0; w = sw; h = sh; } + if (x < 0) x = 0; + if (y < 0) y = 0; + if (x + w > sw) w = sw - x; + if (y + h > sh) h = sh - y; + if (w <= 0 || h <= 0) { XCloseDisplay(d); return NULL; } + + XImage *img = XGetImage(d, root, x, y, w, h, AllPlanes, ZPixmap); + if (!img) { XCloseDisplay(d); return NULL; } + + unsigned char *out = (unsigned char *)malloc((size_t)w * (size_t)h * 3); + if (!out) { XDestroyImage(img); XCloseDisplay(d); return NULL; } + + unsigned long rm = img->red_mask, gm = img->green_mask, bm = img->blue_mask; + int rsh = mask_shift(rm), gsh = mask_shift(gm), bsh = mask_shift(bm); + + for (int j = 0; j < h; j++) { + for (int i = 0; i < w; i++) { + unsigned long px = XGetPixel(img, i, j); + size_t o = ((size_t)j * (size_t)w + (size_t)i) * 3; + out[o + 0] = (unsigned char)((px & rm) >> rsh); + out[o + 1] = (unsigned char)((px & gm) >> gsh); + out[o + 2] = (unsigned char)((px & bm) >> bsh); + } + } + + *ow = w; + *oh = h; + XDestroyImage(img); + XCloseDisplay(d); + return out; +} + +void screen_free(unsigned char *p) { free(p); } diff --git a/screen/framebuffer.mojo b/screen/framebuffer.mojo new file mode 100644 index 0000000..da59d7d --- /dev/null +++ b/screen/framebuffer.mojo @@ -0,0 +1,103 @@ +# screen.framebuffer — Linux framebuffer capture (/dev/fb0), pure Mojo. +# +# Geometry comes from sysfs (no ioctl needed): /sys/class/graphics/fb0/ +# virtual_size -> "W,H" +# bits_per_pixel -> e.g. "32" +# stride -> bytes per scanline (may exceed W*bpp/8) +# Pixels come straight from /dev/fb0. 32-bpp is assumed BGRX, 24-bpp BGR — the +# usual fbdev/efifb layout on little-endian x86 (sysfs does not expose channel +# offsets; use ioctl FBIOGET_VSCREENINFO if you need exotic layouts). +# +# NOTE: /dev/fb0 is typically root:video 0660 — reading it requires membership +# in the `video` group (or root). grab_fb() fails loud otherwise. + +from image.buffer import Image +from image.png import encode_png + + +def _read_text(path: String) raises -> String: + var f = open(path, "r") + var s = f.read() + f.close() + return s + + +def _read_bytes(path: String) raises -> List[UInt8]: + var f = open(path, "r") + var s = f.read() + f.close() + var bs = s.as_bytes() + var out = List[UInt8]() + for i in range(len(bs)): + out.append(bs[i]) + return out^ + + +def _atoi(s: String) -> Int: + var bs = s.as_bytes() + var v = 0 + for i in range(len(bs)): + var c = Int(bs[i]) + if c < ord("0") or c > ord("9"): + break + v = v * 10 + (c - ord("0")) + return v + + +@fieldwise_init +struct FbInfo(ImplicitlyCopyable, Movable): + var width: Int + var height: Int + var bpp: Int + var stride: Int + + +def fb_info(sysfs: String) raises -> FbInfo: + """Read framebuffer geometry from a sysfs dir (e.g. /sys/class/graphics/fb0).""" + var vs = _read_text(String(sysfs + "/virtual_size")) # "W,H\n" + var comma = vs.split(",") + if len(comma) < 2: + raise Error("fb: bad virtual_size: " + vs) + var w = _atoi(String(comma[0])) + var h = _atoi(String(comma[1])) + var bpp = _atoi(_read_text(String(sysfs + "/bits_per_pixel"))) + var stride = _atoi(_read_text(String(sysfs + "/stride"))) + if stride <= 0: + stride = w * (bpp // 8) + return FbInfo(w, h, bpp, stride) + + +def fb_decode(raw: List[UInt8], info: FbInfo) raises -> Image: + """Decode raw framebuffer bytes (BGRX/BGR) into an RGB Image.""" + if info.width <= 0 or info.height <= 0: + raise Error("fb: bad geometry") + var bytespp = info.bpp // 8 + if bytespp < 3: + raise Error("fb: unsupported bpp " + String(info.bpp)) + var img = Image.new(info.width, info.height, 3) + for y in range(info.height): + var row = y * info.stride + for x in range(info.width): + var p = row + x * bytespp + if p + 2 >= len(raw): + continue + var b = raw[p] + var g = raw[p + 1] + var r = raw[p + 2] + var o = (y * info.width + x) * 3 + img.data[o] = r + img.data[o + 1] = g + img.data[o + 2] = b + return img^ + + +def grab_fb(device: String = "/dev/fb0", sysfs: String = "/sys/class/graphics/fb0") raises -> Image: + """Capture the framebuffer into an RGB Image. Needs read access to `device`.""" + var info = fb_info(sysfs) + var raw = _read_bytes(device) + return fb_decode(raw, info) + + +def capture_fb_png(path: String, device: String = "/dev/fb0", sysfs: String = "/sys/class/graphics/fb0") raises: + var img = grab_fb(device, sysfs) + encode_png(img, path) diff --git a/screen/tests/screen_test.mojo b/screen/tests/screen_test.mojo new file mode 100644 index 0000000..5c8b4b6 --- /dev/null +++ b/screen/tests/screen_test.mojo @@ -0,0 +1,110 @@ +# screen tests — decode-logic units. +# pixi run --manifest-path /home/alex/rill/pixi.toml mojo run -I . \ +# screen/tests/screen_test.mojo +# +# The X11 path is verified live against xwininfo/PIL (see README); these unit +# tests cover the pure decode logic for framebuffer (BGRX->RGB) and vcsa +# (cell grid -> text), which can't read the real devices without video/tty group. + +from std.testing import assert_equal, assert_true, TestSuite + +from screen.framebuffer import fb_decode, FbInfo +from screen.vcsa import vcsa_decode, render_text + + +def test_fb_decode_bgrx() raises: + # 2x1 image, 32bpp BGRX, stride = 8 (=2px*4). pixel0 = red, pixel1 = green. + # BGRX byte order: B,G,R,X + var raw = List[UInt8]() + # px0 red: B=0 G=0 R=255 X=0 + raw.append(0) + raw.append(0) + raw.append(255) + raw.append(0) + # px1 green: B=0 G=255 R=0 X=0 + raw.append(0) + raw.append(255) + raw.append(0) + raw.append(0) + var img = fb_decode(raw, FbInfo(2, 1, 32, 8)) + assert_equal(img.width, 2) + assert_equal(img.height, 1) + # RGB out: px0 -> (255,0,0) + assert_equal(Int(img.data[0]), 255) + assert_equal(Int(img.data[1]), 0) + assert_equal(Int(img.data[2]), 0) + # px1 -> (0,255,0) + assert_equal(Int(img.data[3]), 0) + assert_equal(Int(img.data[4]), 255) + assert_equal(Int(img.data[5]), 0) + _ = img^ # keep alive: img's data.free() is ASAP at last use, else the last read is UB + + +def test_fb_decode_stride_padding() raises: + # 1x2 image, stride 8 but only 4 bytes/px used; row1 padded by 4 bytes. + var raw = List[UInt8]() + # row0 px0 = (R=10,G=20,B=30) -> BGRX 30,20,10,0 + raw.append(30) + raw.append(20) + raw.append(10) + raw.append(0) + raw.append(0) # 4 bytes padding to reach stride=8 + raw.append(0) + raw.append(0) + raw.append(0) + # row1 px0 = (R=40,G=50,B=60) -> BGRX 60,50,40,0 + raw.append(60) + raw.append(50) + raw.append(40) + raw.append(0) + var img = fb_decode(raw, FbInfo(1, 2, 32, 8)) + assert_equal(Int(img.data[0]), 10) # row0 R + assert_equal(Int(img.data[1]), 20) + assert_equal(Int(img.data[2]), 30) + assert_equal(Int(img.data[3]), 40) # row1 R (stride skipped the padding) + assert_equal(Int(img.data[4]), 50) + assert_equal(Int(img.data[5]), 60) + _ = img^ # keep alive past the last read (ASAP-destruction guard) + + +def test_vcsa_decode() raises: + # 2 rows x 3 cols. cursor at (1,0). cells: "ABC" / "D E" (with a NUL middle). + var raw = List[UInt8]() + raw.append(2) # rows + raw.append(3) # cols + raw.append(1) # cursor col + raw.append(0) # cursor row + # row0: A,B,C (char,attr pairs) + for ch in [ord("A"), ord("B"), ord("C")]: + raw.append(UInt8(ch)) + raw.append(7) # attr + # row1: D, NUL(->space), E + raw.append(UInt8(ord("D"))) + raw.append(7) + raw.append(0) # NUL char -> space + raw.append(7) + raw.append(UInt8(ord("E"))) + raw.append(7) + var ts = vcsa_decode(raw) + assert_equal(ts.rows, 2) + assert_equal(ts.cols, 3) + assert_equal(ts.cursor_col, 1) + assert_equal(len(ts.lines), 2) + assert_equal(ts.lines[0], "ABC") + assert_equal(ts.lines[1], "D E") + assert_equal(render_text(ts), "ABC\nD E\n") + + +def test_vcsa_too_small() raises: + var raw = List[UInt8]() + raw.append(1) + var threw = False + try: + _ = vcsa_decode(raw) + except e: + threw = True + assert_true(threw) + + +def main() raises: + TestSuite.discover_tests[__functions_in_module()]().run() diff --git a/screen/vcsa.mojo b/screen/vcsa.mojo new file mode 100644 index 0000000..cd8fcfd --- /dev/null +++ b/screen/vcsa.mojo @@ -0,0 +1,72 @@ +# screen.vcsa — Linux virtual-console TEXT snapshot (/dev/vcsaN), pure Mojo. +# +# This captures the *text* on a Linux virtual console (the tty grid), not pixels +# — the text complement to the framebuffer. /dev/vcsaN format: +# byte 0: rows byte 1: cols +# byte 2: cursor col byte 3: cursor row +# then rows*cols cells of 2 bytes each: [char, attribute] +# We keep the char byte and drop the attribute (color/blink). NUL cells render +# as spaces. +# +# NOTE: only meaningful on the Linux text console; inside X/Wayland terminals, +# xterm, tmux, or over SSH there is no vcsa for your session. /dev/vcsaN is +# root:tty 0660 — reading it needs the `tty` group (or root). grab_vcsa() fails +# loud otherwise. N=0 is the *active* console; N>=1 is tty1, tty2, … + + +def _read_bytes(path: String) raises -> List[UInt8]: + var f = open(path, "r") + var s = f.read() + f.close() + var bs = s.as_bytes() + var out = List[UInt8]() + for i in range(len(bs)): + out.append(bs[i]) + return out^ + + +@fieldwise_init +struct TextScreen(Copyable, Movable): + var rows: Int + var cols: Int + var cursor_col: Int + var cursor_row: Int + var lines: List[String] + + +def vcsa_decode(raw: List[UInt8]) raises -> TextScreen: + """Decode a /dev/vcsa buffer into a TextScreen (rows of text).""" + if len(raw) < 4: + raise Error("vcsa: buffer too small") + var rows = Int(raw[0]) + var cols = Int(raw[1]) + var ccol = Int(raw[2]) + var crow = Int(raw[3]) + var lines = List[String]() + var pos = 4 + for _r in range(rows): + var line = String("") + for _c in range(cols): + if pos + 1 >= len(raw): + break + var ch = Int(raw[pos]) + pos += 2 # skip the attribute byte + if ch == 0: + ch = ord(" ") + line += chr(ch) + lines.append(line) + return TextScreen(rows, cols, ccol, crow, lines^) + + +def render_text(ts: TextScreen) -> String: + var out = String("") + for i in range(len(ts.lines)): + out += ts.lines[i] + "\n" + return out + + +def grab_vcsa(n: Int = 0) raises -> TextScreen: + """Snapshot virtual console N (0 = active). Needs read access to /dev/vcsaN.""" + var dev = String("/dev/vcsa") if n == 0 else String("/dev/vcsa" + String(n)) + var raw = _read_bytes(dev) + return vcsa_decode(raw) diff --git a/screen/x11.mojo b/screen/x11.mojo new file mode 100644 index 0000000..b0c3c71 --- /dev/null +++ b/screen/x11.mojo @@ -0,0 +1,98 @@ +# screen.x11 — X11 screen capture via the screen_shim.so C floor. +# +# Xlib's Display* is opaque and crashes when passed back across Mojo FFI, so +# the Xlib calls live in cshim/screen_shim.c. This module loads that .so, asks +# for an RGB24 buffer, and hands back an `image.Image` (or writes a PNG). +# +# Build the shim first: +# gcc -shared -fPIC -O2 screen/cshim/screen_shim.c -o screen/cshim/screen_shim.so -lX11 +# Run from the repo root (so the default relative .so path resolves), or set +# SCREEN_SHIM_PATH to an absolute path. + +from std.ffi import OwnedDLHandle as DLHandle +from std.memory import alloc, UnsafePointer +from std.builtin.type_aliases import MutExternalOrigin +from std.os import getenv + +from image.buffer import Image +from image.png import encode_png + +comptime PtrI32 = UnsafePointer[Int32, MutExternalOrigin] +comptime PtrU8 = UnsafePointer[UInt8, MutExternalOrigin] +comptime DEFAULT_SHIM = "screen/cshim/screen_shim.so" + + +@fieldwise_init +struct Size(ImplicitlyCopyable, Movable): + var w: Int + var h: Int + + +def shim_path() -> String: + var p = getenv("SCREEN_SHIM_PATH") + if p.byte_length() > 0: + return p + return String(DEFAULT_SHIM) + + +def x11_size() raises -> Size: + """Size of the default-screen root window. Raises if no display.""" + var lib = DLHandle(shim_path()) + var f = lib.get_function[ + def(PtrI32, PtrI32) thin abi("C") -> Int32 + ]("screen_x11_size") + var wh = alloc[Int32](2) + wh[0] = 0 + wh[1] = 0 + var rc = f(wh, wh + 1) + var w = Int(wh[0]) + var h = Int(wh[1]) + wh.free() + # Keep the handle alive past the FFI call: OwnedDLHandle is destroyed at its + # last use (ASAP), which dlclose's the .so out from under the function + # pointer and segfaults the call. `_ = lib^` defers that to here. + _ = lib^ + if rc != 0: + raise Error("screen.x11: XOpenDisplay failed (no DISPLAY?)") + return Size(w, h) + + +def grab_x11(x: Int, y: Int, w: Int, h: Int) raises -> Image: + """Capture rect (x,y,w,h) of the root window into an RGB Image. + + w<=0 or h<=0 captures the whole screen. The rect is clamped to the screen. + """ + var lib = DLHandle(shim_path()) + var fgrab = lib.get_function[ + def(Int32, Int32, Int32, Int32, PtrI32, PtrI32) thin abi("C") -> PtrU8 + ]("screen_x11_grab") + var ffree = lib.get_function[def(PtrU8) thin abi("C") -> None]("screen_free") + + var wh = alloc[Int32](2) + wh[0] = 0 + wh[1] = 0 + var buf = fgrab(Int32(x), Int32(y), Int32(w), Int32(h), wh, wh + 1) + var ow = Int(wh[0]) + var oh = Int(wh[1]) + wh.free() + + # The shim sets ow/oh only on success and returns NULL on failure — gate on + # the dimensions so we never dereference a NULL buffer. + if ow <= 0 or oh <= 0: + raise Error("screen.x11: grab failed (no display or empty rect)") + + var img = Image.new(ow, oh, 3) + var n = ow * oh * 3 + for i in range(n): + img.data[i] = buf[i] + ffree(buf) + # Hold the handle alive through both FFI calls above (grab + free); see + # x11_size for the ASAP-destruction rationale. + _ = lib^ + return img^ + + +def capture_x11_png(path: String, x: Int, y: Int, w: Int, h: Int) raises: + """Grab the screen (or a rect) and write it to a PNG file.""" + var img = grab_x11(x, y, w, h) + encode_png(img, path) diff --git a/snapshot/README.md b/snapshot/README.md new file mode 100644 index 0000000..d22e1d0 --- /dev/null +++ b/snapshot/README.md @@ -0,0 +1,111 @@ +# snapshot — pure-Mojo project snapshots + +A compact, structured **index of a project directory** — built so an agent (or a +human) can grasp and track a codebase without re-reading every file, and can +diff the state of a tree over time. + +100% Mojo + libc. No third-party dependencies. Filesystem via `std.os`, current +time via libc `time(2)` (`std.ffi`); everything else is pure Mojo string work. + +## What a snapshot captures + +- **File tree** — every file (excluding `.git`, `.pixi`, `node_modules`, + `target`, `build`, `__pycache__`, … ), with byte size and line count. +- **Language breakdown** — files / lines / size per extension, sorted by size. +- **Symbol map** — top-level declarations per source file: + - Mojo: `struct` / `trait` / `def` / `fn` + - Python: `class` / `def` / `async def` + - Rust: `fn` / `struct` / `enum` / `trait` / `impl` / `macro_rules!` (+`pub`) + - (top-level only — indentation 0 — so it stays a clean module-level API map) +- **Content hash** — FNV-1a 64 per source file, so two snapshots can be diffed. + +Binary / non-source files (and any text file over `max_text_bytes`, default +2 MiB) are recorded with size only (no line count, no hash). + +## Usage + +Build the CLI once (from the repo root, with `-I .`): + +```sh +mojo build -I . snapshot/cli.mojo -o snapshot/snapshot +``` + +Or run it directly: + +```sh +# print a markdown map to stdout (lands straight in a tool result) +mojo run -I . snapshot/cli.mojo + +# brief = tree + language table only (skip the symbol map) +mojo run -I . snapshot/cli.mojo --brief + +# write PREFIX.md (human map) + PREFIX.tsv (machine manifest) +mojo run -I . snapshot/cli.mojo -o PREFIX + +# diff two manifests: added / removed / changed +mojo run -I . snapshot/cli.mojo --diff OLD.tsv NEW.tsv +``` + +Typical agent workflow: snapshot a project to `-o /tmp/proj`, do work, snapshot +again to `-o /tmp/proj2`, then `--diff /tmp/proj.tsv /tmp/proj2.tsv` to see +exactly what changed. + +## Output + +**Markdown** (stdout / `PREFIX.md`): + +``` +# Snapshot — json +_epoch 1781465771 · 20 files (20 source) · 4458 lines · 163.2K_ + +## languages +| ext | files | lines | size | +|---|--:|--:|--:| +| mojo | 19 | 4352 | 157.0K | +| md | 1 | 106 | 6.2K | + +## files +`README.md` 6.2K 106L +`parser.mojo` 9.9K 297L +... + +## symbols +### serialize.mojo +- def dumps +- def dumps_pretty +... +``` + +**TSV manifest** (`PREFIX.tsv`) — one line per file, easy to re-parse: + +``` +#snapshot +#path size lines hash kind +README.md 6325 106 8d1e822ded6785ba T +parser.mojo 10140 297 ... T +graphics/images/text.png 5841 -1 - B +``` + +## Library API + +```mojo +from snapshot import scan, default_config, render_markdown, render_tsv, diff + +var cfg = default_config() # max_text_bytes=2MiB, max_depth=64, brief=False +var snap = scan("myproject", cfg) # Snapshot{ root, epoch, files: List[FileEntry] } +print(render_markdown(snap, cfg)) +write_file("snap.tsv", render_tsv(snap)) +print(diff("old.tsv", "snap.tsv")) +``` + +Also exported: `FileEntry`, `Config`, `Snapshot`, `symbols_for`, `fnv1a_hex`, +`human_size`, `parse_manifest`. + +## Tests + +```sh +mojo run -I . snapshot/tests/snapshot_test.mojo +``` + +9 unit tests (hashing determinism, line counting, extension parsing, +classification, size formatting, symbol extraction for Mojo/Python). diff --git a/snapshot/__init__.mojo b/snapshot/__init__.mojo new file mode 100644 index 0000000..8300369 --- /dev/null +++ b/snapshot/__init__.mojo @@ -0,0 +1,19 @@ +# snapshot — pure-Mojo project snapshots (tree + languages + symbols + hashes). +# +# from snapshot import scan, render_markdown, render_tsv, diff + +from snapshot.snapshot import ( + Snapshot, + FileEntry, + Config, + LangStat, + scan, + default_config, + render_markdown, + render_tsv, + write_file, + human_size, + symbols_for, + fnv1a_hex, +) +from snapshot.diff import diff, parse_manifest diff --git a/snapshot/cli.mojo b/snapshot/cli.mojo new file mode 100644 index 0000000..7c255cb --- /dev/null +++ b/snapshot/cli.mojo @@ -0,0 +1,64 @@ +# snapshot.cli — runnable entrypoint. +# +# mojo run -I . snapshot/cli.mojo [-o PREFIX] [--brief] +# mojo run -I . snapshot/cli.mojo --diff OLD.tsv NEW.tsv +# +# With no -o, the markdown map is printed to stdout (lands straight in a tool +# result). With -o PREFIX it writes PREFIX.md (human map) + PREFIX.tsv +# (machine manifest for `--diff`). + +from std.sys import argv + +from snapshot.snapshot import scan, default_config, render_markdown, render_tsv, write_file +from snapshot.diff import diff + + +def _usage(): + print("usage:") + print(" snapshot [-o PREFIX] [--brief]") + print(" snapshot --diff OLD.tsv NEW.tsv") + + +def main() raises: + var raw = argv() + var args = List[String]() + for i in range(len(raw)): + args.append(String(raw[i])) + + if len(args) < 2: + _usage() + return + + if args[1] == "--diff": + if len(args) < 4: + _usage() + return + print(diff(args[2], args[3])) + return + + if args[1] == "-h" or args[1] == "--help": + _usage() + return + + var root = args[1] + var cfg = default_config() + var out_prefix = String("") + + var i = 2 + while i < len(args): + var a = args[i] + if a == "-o" or a == "--out": + if i + 1 < len(args): + i += 1 + out_prefix = args[i] + elif a == "--brief": + cfg.brief = True + i += 1 + + var snap = scan(root, cfg) + if out_prefix.byte_length() == 0: + print(render_markdown(snap, cfg)) + else: + write_file(String(out_prefix + ".md"), render_markdown(snap, cfg)) + write_file(String(out_prefix + ".tsv"), render_tsv(snap)) + print("wrote " + out_prefix + ".md + " + out_prefix + ".tsv (" + String(len(snap.files)) + " files)") diff --git a/snapshot/diff.mojo b/snapshot/diff.mojo new file mode 100644 index 0000000..947d7ca --- /dev/null +++ b/snapshot/diff.mojo @@ -0,0 +1,133 @@ +# snapshot.diff — compare two TSV manifests produced by `render_tsv`. +# +# Reports files added / removed / changed between an OLD and a NEW snapshot. +# "changed" = content hash differs (text files) or size differs (binary/other). + +from snapshot.snapshot import FileEntry, human_size + + +def _atoi(s: String) -> Int: + var bs = s.as_bytes() + var i = 0 + var neg = False + if len(bs) > 0 and Int(bs[0]) == ord("-"): + neg = True + i = 1 + var v = 0 + while i < len(bs): + var c = Int(bs[i]) + if c < ord("0") or c > ord("9"): + break + v = v * 10 + (c - ord("0")) + i += 1 + return -v if neg else v + + +def parse_manifest(path: String) raises -> Dict[String, FileEntry]: + var m = Dict[String, FileEntry]() + var f = open(path, "r") + var data = f.read() + f.close() + var lines = data.split("\n") + for i in range(len(lines)): + var line = lines[i] + if line.byte_length() == 0: + continue + if Int(line.as_bytes()[0]) == ord("#"): + continue + var c = line.split("\t") + if len(c) < 5: + continue + var path_ = String(c[0]) + var is_text = c[4] == "T" + m[path_] = FileEntry( + path_, _atoi(String(c[1])), _atoi(String(c[2])), String(c[3]), is_text + ) + return m^ + + +def _sort_strs(mut xs: List[String]): + for i in range(1, len(xs)): + var j = i + while j > 0 and xs[j - 1] > xs[j]: + var tmp = xs[j] + xs[j] = xs[j - 1] + xs[j - 1] = tmp + j -= 1 + + +def diff(old_path: String, new_path: String) raises -> String: + var oldm = parse_manifest(old_path) + var newm = parse_manifest(new_path) + + var added = List[String]() + var removed = List[String]() + var changed = List[String]() + + for entry in newm.items(): + if entry.key not in oldm: + added.append(entry.key) + else: + var o = oldm[entry.key] + var n = entry.value + var differ = (n.is_text and o.hash != n.hash) or ( + (not n.is_text) and o.size != n.size + ) + if differ: + changed.append(entry.key) + for entry in oldm.items(): + if entry.key not in newm: + removed.append(entry.key) + + _sort_strs(added) + _sort_strs(removed) + _sort_strs(changed) + + var out = String("") + out += "# Snapshot diff\n" + out += "old: " + old_path + "\nnew: " + new_path + "\n" + out += ( + "+" + + String(len(added)) + + " added · -" + + String(len(removed)) + + " removed · ~" + + String(len(changed)) + + " changed\n\n" + ) + + if len(added) > 0: + out += "## added\n" + for i in range(len(added)): + var n = newm[added[i]] + out += "+ " + added[i] + " " + human_size(n.size) + if n.is_text: + out += " " + String(n.lines) + "L" + out += "\n" + out += "\n" + + if len(removed) > 0: + out += "## removed\n" + for i in range(len(removed)): + out += "- " + removed[i] + "\n" + out += "\n" + + if len(changed) > 0: + out += "## changed\n" + for i in range(len(changed)): + var k = changed[i] + var o = oldm[k] + var n = newm[k] + out += "~ " + k + if n.is_text: + var dl = n.lines - o.lines + var sign = String("+") if dl >= 0 else String("") + out += " " + String(o.lines) + "→" + String( + n.lines + ) + "L (" + sign + String(dl) + ")" + else: + out += " " + human_size(o.size) + "→" + human_size(n.size) + out += "\n" + out += "\n" + + return out diff --git a/snapshot/snapshot.mojo b/snapshot/snapshot.mojo new file mode 100644 index 0000000..a58727a --- /dev/null +++ b/snapshot/snapshot.mojo @@ -0,0 +1,430 @@ +# snapshot.snapshot — pure-Mojo project snapshots. +# +# A "snapshot" is a compact, structured index of a project directory: +# * the file tree (sizes + line counts) +# * a per-extension language breakdown +# * a top-level symbol map (struct/trait/def/fn for Mojo, class/def for +# Python, fn/struct/enum/trait/impl for Rust) +# * a content hash per source file (FNV-1a 64) so two snapshots can be diffed +# +# Designed to be RUN, not just imported: `cli.mojo` prints a markdown map to +# stdout (so the snapshot lands directly in a tool result) and can write a TSV +# manifest for change-detection across time. +# +# 100% Mojo + libc (no third-party deps). Filesystem via std.os; current time +# via libc time(2) (std.ffi); everything else is pure Mojo string work. + +from std.os import listdir +from std.os.path import isdir, getsize +from std.ffi import external_call + +# ── FNV-1a 64-bit (content hashing) ────────────────────────────────────────── +comptime FNV_OFFSET: UInt64 = 14695981039346656037 +comptime FNV_PRIME: UInt64 = 1099511628211 + +# ── language ids ────────────────────────────────────────────────────────────── +comptime LANG_NONE = 0 +comptime LANG_MOJO = 1 +comptime LANG_PY = 2 +comptime LANG_RUST = 3 + + +@fieldwise_init +struct FileEntry(ImplicitlyCopyable, Movable): + var path: String # relative to scan root + var size: Int # bytes + var lines: Int # -1 when not counted (non-text / too big) + var hash: String # 16-hex FNV-1a, or "-" when not hashed + var is_text: Bool + + +@fieldwise_init +struct Config(ImplicitlyCopyable, Movable): + var max_text_bytes: Int # files larger than this are recorded as "other" + var max_depth: Int + var brief: Bool # skip the symbol map + + +@fieldwise_init +struct LangStat(ImplicitlyCopyable, Movable): + var files: Int + var lines: Int + var bytes: Int + + +@fieldwise_init +struct Snapshot(Copyable, Movable): + var root: String + var epoch: Int + var files: List[FileEntry] + + +def default_config() -> Config: + return Config(2 * 1024 * 1024, 64, False) + + +def now_epoch() -> Int: + return external_call["time", Int](Int(0)) + + +# ── small string helpers ───────────────────────────────────────────────────── +def to_hex16(v: UInt64) -> String: + var chars = String("0123456789abcdef") + var res = String("") + var shift = 60 + while shift >= 0: + var nib = Int((v >> UInt64(shift)) & UInt64(15)) + res += chars[byte=nib] + shift -= 4 + return res + + +def fnv1a_hex(data: String) -> String: + var bs = data.as_bytes() + var h = FNV_OFFSET + for i in range(len(bs)): + h = (h ^ UInt64(bs[i])) * FNV_PRIME + return to_hex16(h) + + +def count_lines(data: String) -> Int: + return data.count("\n") + + +def ext_of(name: String) -> String: + # lowercase extension after the final '.', else "". + var parts = name.split(".") + if len(parts) < 2: + return String("") + return parts[len(parts) - 1].lower() + + +def is_text_ext(ext: String) -> Bool: + var t = [ + "mojo", "py", "pyi", "rs", "md", "txt", "rst", "toml", "json", "jsonl", + "ndjson", "yaml", "yml", "ini", "cfg", "conf", "sh", "bash", "zsh", "c", + "h", "cc", "cpp", "cxx", "hpp", "hxx", "js", "mjs", "ts", "tsx", "jsx", + "html", "htm", "css", "scss", "go", "java", "kt", "rb", "lua", "sql", + "csv", "tsv", "xml", "svg", "make", "mk", "cmake", "gradle", "proto", + "graphql", "vue", "env", "gitignore", "dockerignore", "lock", + ] + for i in range(len(t)): + if ext == t[i]: + return True + return False + + +def is_text_name(name: String) -> Bool: + var t = [ + "README", "LICENSE", "Makefile", "Dockerfile", "CHANGELOG", "TODO", + "NOTICE", "AUTHORS", "COPYING", "MANIFEST", + ] + for i in range(len(t)): + if name == t[i]: + return True + return False + + +def is_excluded_dir(name: String) -> Bool: + var t = [ + ".git", ".pixi", ".magic", ".venv", "venv", "node_modules", "target", + "build", "dist", "__pycache__", ".mypy_cache", ".pytest_cache", + ".ruff_cache", ".cache", ".idea", ".gradle", ".next", ".turbo", + "site-packages", ".eggs", + ] + for i in range(len(t)): + if name == t[i]: + return True + return False + + +def lang_of(ext: String) -> Int: + if ext == "mojo": + return LANG_MOJO + if ext == "py" or ext == "pyi": + return LANG_PY + if ext == "rs": + return LANG_RUST + return LANG_NONE + + +# ── top-level symbol extraction (cheap line-prefix scan, no real parser) ────── +def _is_ident_byte(c: UInt8) -> Bool: + var i = Int(c) + return ( + (i >= ord("a") and i <= ord("z")) + or (i >= ord("A") and i <= ord("Z")) + or (i >= ord("0") and i <= ord("9")) + or i == ord("_") + ) + + +def _ident_after(line: String, start: Int) -> String: + var bs = line.as_bytes() + var n = len(bs) + var i = start + while i < n and Int(bs[i]) == ord(" "): + i += 1 + var s = i + while i < n and _is_ident_byte(bs[i]): + i += 1 + if i == s: + return String("") + return String(line[byte=s:i]) + + +def _match_kw(line: String, kws: List[String]) -> Int: + # returns the length of the matched keyword, else -1. kws longest-first. + for i in range(len(kws)): + if line.startswith(kws[i]): + return kws[i].byte_length() + return -1 + + +def _kws_for(lang: Int) -> List[String]: + if lang == LANG_MOJO: + return ["struct ", "trait ", "def ", "fn "] + if lang == LANG_PY: + return ["async def ", "class ", "def "] + if lang == LANG_RUST: + return [ + "pub struct ", "pub enum ", "pub trait ", "pub fn ", "pub async fn ", + "struct ", "enum ", "trait ", "impl ", "fn ", "async fn ", + "macro_rules! ", + ] + return List[String]() + + +def symbols_for(data: String, ext: String) -> List[String]: + var syms = List[String]() + var lang = lang_of(ext) + if lang == LANG_NONE: + return syms^ + var kws = _kws_for(lang) + var lines = data.split("\n") + for li in range(len(lines)): + var line = String(lines[li]) + if line.byte_length() == 0: + continue + var b0 = Int(line.as_bytes()[0]) + if b0 == ord(" ") or b0 == ord("\t"): + continue # top-level declarations only + var kwlen = _match_kw(line, kws) + if kwlen < 0: + continue + var name = _ident_after(line, kwlen) + if name.byte_length() == 0: + continue + # label = the keyword (trimmed) for kind context + var kind = String(line[byte=0:kwlen]) + syms.append(String(kind + name)) + return syms^ + + +# ── directory walk ─────────────────────────────────────────────────────────── +def _walk( + root: String, rel: String, depth: Int, mut files: List[FileEntry], cfg: Config +) raises: + if depth > cfg.max_depth: + return + var dirpath = root if rel.byte_length() == 0 else String(root + "/" + rel) + var names = listdir(dirpath) + for ni in range(len(names)): + var name = names[ni] + var full = String(dirpath + "/" + name) + var childrel = name if rel.byte_length() == 0 else String(rel + "/" + name) + if isdir(full): + if is_excluded_dir(name): + continue + _walk(root, childrel, depth + 1, files, cfg) + else: + var ext = ext_of(name) + var size = getsize(full) + var text = is_text_ext(ext) or is_text_name(name) + if text and size <= cfg.max_text_bytes: + var f = open(full, "r") + var data = f.read() + f.close() + files.append( + FileEntry( + childrel, size, count_lines(data), fnv1a_hex(data), True + ) + ) + else: + files.append(FileEntry(childrel, size, -1, String("-"), False)) + + +def _sort_entries(mut entries: List[FileEntry]): + # insertion sort by path (small N; deterministic output for diff-friendliness) + var n = len(entries) + for i in range(1, n): + var j = i + while j > 0 and entries[j - 1].path > entries[j].path: + var tmp = entries[j].copy() + entries[j] = entries[j - 1].copy() + entries[j - 1] = tmp^ + j -= 1 + + +def scan(root: String, cfg: Config) raises -> Snapshot: + var files = List[FileEntry]() + _walk(root, String(""), 0, files, cfg) + _sort_entries(files) + return Snapshot(root, now_epoch(), files^) + + +# ── rendering ──────────────────────────────────────────────────────────────── +def _fmt1(value: Float64) -> String: + var x10 = Int(value * 10.0 + 0.5) + return String(String(x10 // 10) + "." + String(x10 % 10)) + + +def human_size(n: Int) -> String: + if n < 1024: + return String(String(n) + "B") + var kb = Float64(n) / 1024.0 + if kb < 1024.0: + return String(_fmt1(kb) + "K") + var mb = kb / 1024.0 + if mb < 1024.0: + return String(_fmt1(mb) + "M") + var gb = mb / 1024.0 + return String(_fmt1(gb) + "G") + + +def render_markdown(snap: Snapshot, cfg: Config) raises -> String: + var total_lines = 0 + var total_bytes = 0 + var text_files = 0 + var stats = Dict[String, LangStat]() + for i in range(len(snap.files)): + var e = snap.files[i] + total_bytes += e.size + if e.is_text: + text_files += 1 + if e.lines > 0: + total_lines += e.lines + var key = ext_of(e.path) + if key.byte_length() == 0: + key = String("(noext)") + var ln = e.lines if e.lines > 0 else 0 + if key in stats: + var s = stats[key] + stats[key] = LangStat(s.files + 1, s.lines + ln, s.bytes + e.size) + else: + stats[key] = LangStat(1, ln, e.size) + + var out = String("") + out += "# Snapshot — " + snap.root + "\n" + out += ( + "_epoch " + + String(snap.epoch) + + " · " + + String(len(snap.files)) + + " files (" + + String(text_files) + + " source) · " + + String(total_lines) + + " lines · " + + human_size(total_bytes) + + "_\n\n" + ) + + # language table, sorted by bytes desc + var keys = List[String]() + for entry in stats.items(): + keys.append(entry.key) + for i in range(1, len(keys)): + var j = i + while j > 0 and stats[keys[j - 1]].bytes < stats[keys[j]].bytes: + var tmp = keys[j] + keys[j] = keys[j - 1] + keys[j - 1] = tmp + j -= 1 + out += "## languages\n\n" + out += "| ext | files | lines | size |\n|---|--:|--:|--:|\n" + for i in range(len(keys)): + var k = keys[i] + var s = stats[k] + out += ( + "| " + + k + + " | " + + String(s.files) + + " | " + + String(s.lines) + + " | " + + human_size(s.bytes) + + " |\n" + ) + out += "\n" + + # file tree + out += "## files\n\n" + for i in range(len(snap.files)): + var e = snap.files[i] + out += "`" + e.path + "` " + human_size(e.size) + if e.is_text: + out += " " + String(e.lines) + "L" + out += "\n" + out += "\n" + + if cfg.brief: + return out + + # symbol map + out += "## symbols\n\n" + for i in range(len(snap.files)): + var e = snap.files[i] + if not e.is_text: + continue + var ext = ext_of(e.path) + if lang_of(ext) == LANG_NONE: + continue + var f = open(String(snap.root + "/" + e.path), "r") + var data = f.read() + f.close() + var syms = symbols_for(data, ext) + if len(syms) == 0: + continue + out += "### " + e.path + "\n" + for s in range(len(syms)): + out += "- " + syms[s] + "\n" + out += "\n" + return out + + +def render_tsv(snap: Snapshot) -> String: + # machine manifest for diffing. header + one line/file. + var out = String("") + out += ( + "#snapshot\t" + + snap.root + + "\t" + + String(snap.epoch) + + "\t" + + String(len(snap.files)) + + "\n" + ) + out += "#path\tsize\tlines\thash\tkind\n" + for i in range(len(snap.files)): + var e = snap.files[i] + var kind = String("T") if e.is_text else String("B") + out += ( + e.path + + "\t" + + String(e.size) + + "\t" + + String(e.lines) + + "\t" + + e.hash + + "\t" + + kind + + "\n" + ) + return out + + +def write_file(path: String, content: String) raises: + with open(path, "w") as f: + f.write(content) diff --git a/snapshot/tests/snapshot_test.mojo b/snapshot/tests/snapshot_test.mojo new file mode 100644 index 0000000..590498c --- /dev/null +++ b/snapshot/tests/snapshot_test.mojo @@ -0,0 +1,97 @@ +# snapshot tests — pure-function units. +# pixi run --manifest-path /home/alex/rill/pixi.toml mojo run -I . \ +# snapshot/tests/snapshot_test.mojo + +from std.testing import assert_equal, assert_true, TestSuite + +from snapshot.snapshot import ( + fnv1a_hex, + to_hex16, + count_lines, + ext_of, + is_text_ext, + is_excluded_dir, + human_size, + symbols_for, +) + + +def test_hex16() raises: + assert_equal(to_hex16(UInt64(255)), "00000000000000ff") + assert_equal(to_hex16(UInt64(0)), "0000000000000000") + + +def test_fnv1a_deterministic() raises: + var a = fnv1a_hex(String("hello world")) + var b = fnv1a_hex(String("hello world")) + var c = fnv1a_hex(String("hello worle")) + assert_equal(a, b) # same input -> same hash + assert_true(a != c) # one byte change -> different hash + assert_equal(a.byte_length(), 16) + + +def test_count_lines() raises: + assert_equal(count_lines(String("a\nb\nc\n")), 3) + assert_equal(count_lines(String("no newline")), 0) + assert_equal(count_lines(String("")), 0) + + +def test_ext_of() raises: + assert_equal(ext_of(String("foo.MOJO")), "mojo") # lowercased + assert_equal(ext_of(String("a.b.py")), "py") # final segment + assert_equal(ext_of(String("README")), "") # no dot + assert_equal(ext_of(String(".gitignore")), "gitignore") + + +def test_classify() raises: + assert_true(is_text_ext(String("mojo"))) + assert_true(is_text_ext(String("rs"))) + assert_true(not is_text_ext(String("png"))) + assert_true(is_excluded_dir(String(".git"))) + assert_true(is_excluded_dir(String("node_modules"))) + assert_true(not is_excluded_dir(String("src"))) + + +def test_human_size() raises: + assert_equal(human_size(512), "512B") + assert_equal(human_size(1024), "1.0K") + assert_equal(human_size(1024 * 1024), "1.0M") + assert_equal(human_size(1536), "1.5K") + + +def test_symbols_mojo() raises: + var src = String( + "struct Foo(Copyable):\n" + + " var x: Int\n" + + " def method(self):\n" # indented -> skipped (top-level only) + + " pass\n" + + "def top_level() -> Int:\n" + + " return 1\n" + + "trait Bar:\n" + + " pass\n" + ) + var syms = symbols_for(src, String("mojo")) + assert_equal(len(syms), 3) + assert_equal(syms[0], "struct Foo") + assert_equal(syms[1], "def top_level") + assert_equal(syms[2], "trait Bar") + + +def test_symbols_python() raises: + var src = String( + "class C:\n def m(self):\n pass\nasync def fetch():\n pass\n" + ) + var syms = symbols_for(src, String("py")) + assert_equal(len(syms), 2) + assert_equal(syms[0], "class C") + assert_equal(syms[1], "async def fetch") + + +def test_symbols_none() raises: + # non-source extension -> no symbols + var syms = symbols_for(String("# title\nbody\n"), String("md")) + assert_equal(len(syms), 0) + + +def main() raises: + TestSuite.discover_tests[__functions_in_module()]().run()