feat: write a record on one line with encode_line/decode_line - #44
Conversation
Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #43
The readable format needed a layout where one record is one line: an append-only log is appended a record at a time and compacted by cutting at a newline, and grep, tail -f and wc -l all treat a line as an event. Until now the only single-line form was the base64 one, which is unreadable. On one line a link cannot say what it is by how its rows break, so an object names itself with the 'o' link id the notation already has and writes its pairs as their own links: (o: (bytes 2827) (complete true)). A bare link is then always an array, which keeps () an empty array as in the indented form, and every value -- the empty key included -- round trips. Compact-format sniffing is tightened accordingly: a link that opens another link straight away is the readable form, and the compact null is only the whole link (null), so a line such as (null 1) is read as the array it is.
konard
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Working session summaryDone. PR #44 is ready for review: #44 What shipped — a readable single-line encoder and its exact inverse in all four implementations ( The ambiguity issue #43 delegated is answered with its option 2 — a marker that is part of the notation.
One behaviour change worth flagging: the compact-format sniffer claimed any document starting with a type marker, so the readable line Verification: shared fixtures gained a All 10 CI runs on I skipped one thing deliberately: a This summary was automatically extracted from the AI working session output. |
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost: $15.630100📊 Context and tokens usage:Claude Opus 5: (4 sub-sessions)
Total: (7.7K new + 350.0K cache writes + 16.9M cache reads) input tokens, 146.7K output tokens, $15.630100 cost 🤖 Models used:
📎 Log file uploaded as Gist (5671KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
🔄 Auto-restart 1/5Reason: CI failures detected Starting new session to address the issues. Auto-restart-until-mergeable mode is active. This run will stop after 5 restart iterations in total. |
CodeQL flagged `js/polynomial-redos` on `decodeLine`: the trailing trim `/[\n\r]+$/` restarts inside a run of newlines, so a record followed by many of them costs quadratic time — 20 s for 100k newlines, measured. Trim by scanning from both ends instead, which is what the Python, Rust and C# siblings already do with their built-in character trims. Each language gains the regression test: the padded record is still refused for holding more than one line, and returns well inside two seconds.
Working session summaryCI is green. #44 Root cause of the CodeQL failure (alert text.replace(/^[\n\r]+/, '').replace(/[\n\r]+$/, '')The end-anchored Fix (commit
Verification: JS 429/429 tests pass plus This summary was automatically extracted from the AI working session output. |
🔄 Auto-restart-until-mergeable Log 1/5This log file contains the complete execution trace of the AI solution draft process. 💰 Cost: $1.902202📊 Context and tokens usage:
Total: (100 new + 45.4K cache writes + 2.3M cache reads) input tokens, 12.8K output tokens, $1.902202 cost 🤖 Models used:
📎 Log file uploaded as Gist (6788KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
🎉 Auto-mergedThis pull request has been automatically merged by hive-mind.
Auto-merged by hive-mind with --auto-merge flag |
2 participants
Closes #43.
An append-only log wants one record per line: appending is one write, compaction
cuts at a newline, and
grep,tail -fandwc -lall treat a line as oneevent.
encode()spreads a record over many lines andencode_compact()hidesit in base64, so a downstream project invented the private dialect
((:"bytes" 2827) (:"complete" true))— which the notation's own parserrejects, and which this codec decodes into three-element arrays.
This adds a readable single-line form and its exact inverse to all four
implementations.
encode_line(&value)/decode_line(text)encodeLine({ obj })/decodeLine({ notation })encode_line(obj)/decode_line(notation)Codec.EncodeLine(obj)/Codec.DecodeLine(notation)The ambiguity, and how it is answered
Issue #43 left the pair-vs-one-element-array question to this side and offered
three ways out. This takes option 2, a marker that is part of the notation:
(o: (key value) …), an array is(value …);("a" 1)is a two-element arrayand
(o: (a 1))is a one-pair object — both spellings survive a round trip;()is the empty array,(o:)the empty object;(o: ("" 2))rather than having to be rejected.ois an ordinary link id, so a line is plain Links Notation thatlinks_notationparses. Scalars stay bare (2827,true,null), so typessurvive. A string keeps its own characters — only what would break framing is
escaped, and a string holding a newline is written as
(base64 "…")individually, so the rest of the record stays readable.
Behaviour change
The compact-format sniffer was too eager: it claimed any document starting with
a type marker, so the readable line
(null 1)was routed to the base64 reader.It now requires a compact body, so
(null 1)decodes as a line. The singledocument both forms claim,
(null)((None)in Python), stays the compactnull, so documents written before this change keep decoding.
Tests
fixtures/readable-format/cases.jsongained alinefield on all 43 cases (4 of them new),asserted four ways per case in every language (encodes to it, decodes back
from it, contains no line break, and the plain
decodereads it) — so allfour languages write byte-identical lines.
objects and arrays; the notation's own parser accepts the output;
decode(encode_line(v)) == decode(encode(v))for objects, arrays, nesting andempty collections; a string with quotes, backslashes, newlines and non-ASCII
round-trips exactly and stays on one line; the empty key round-trips; a marked
object holding a non-pair is rejected with a clear error; two lines are not
one record while a trailing newline is tolerated; and the hand-rolled dialect
from the issue is the one the parser rejects.
append_only_logexample per language writes, counts, greps and reads backa log of one record per line; the Rust one runs in CI.
Local runs: Rust tests +
clippy --all-targets+fmt --check; 428 JS tests +eslint + prettier; 395 Python tests + CI-scoped
ruff/mypy; 363 C# tests +dotnet format --verify-no-changes+ Release build with/warnaserror.Changelog fragments for all four packages are included.