Repository navigation
Conversation
An LLM chat client in Octane (chat-stream/octane/App.tsrx): a sidebar of conversations, a message list and a composer, with replies streamed a few tokens at a time and rendered as memoized Markdown. It commits like the Octane fixture of the chat-stream benchmark in the Octane repository: event handlers only set state, which the steps commit through window.__benchFlush, and each streamed chunk commits with flushSync. The corpus, Markdown parser, network stub and stylesheet are in shared/, and the build is in dist/octane.
✅ Deploy Preview for webkit-speedometer-preview ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
StreamReply1, StreamReply2 and StreamReply3 become a single StreamReplies step that sends the three prompts in turn in the first conversation, streaming each reply to completion before sending the next. Also fix the README's reference to chatStreamHelpers, which is named chatStreamTests.
Contributor
|
We have 2 very fast steps on my m4: => StreamReplies and TypePrompt should probably be bumped by 2x |
Build the Markdown views the way a design-system component catalog is built, as AssistantChat-Octane's catalog is: the app root provides theme, locale and layout contexts, each message a message context, and each reply streaming, typography and density contexts. Every block renders through a memoized chain (MarkdownBlock, the block's own component, Frame, Part), and each of them, the list items, inline spans and code lines reads the contexts it depends on. That exercises Octane's context dependency tracking across long chains of memoized ancestors, where AssistantChat-Octane's streaming steps spend much of their time. The streaming cursor's selector moves one level down, into the block frame.
Replace the catalog's decorative contexts (theme, locale, layout, message, streaming, typography, density) and the Frame and Part layers with the kinds of context the reply renderer of a large chat service's web app reads, as found in its production bundle: - Layout flow. Each reply and each list provide a new flow value whenever they render, with each child's previous sibling. Each block's wrapper and each list item read it and compute their spacing with that renderer's spacing rules, so every chunk re-renders every block's wrapper while memo skips the content of the blocks that did not grow. - The colour scheme, read by code blocks and inline code. - Streamed words. Text in a reply that streams in renders one memoized span per word. Each word reads four animation contexts and starts its fade-in from a layout effect with the delay a scheduler gives it, staggering words as that renderer does. The scheduler reads the stream's virtual clock, 80 ms per chunk, kept by shared/network.js, instead of performance.now(). The steps end the word animations once a reply is complete, through window.finishWordAnimations(), so none of that work lands in a later step.
Checking the previous commit against the reference renderer's bundle and its live DOM turned up several differences, fixed here: - The word animation contexts are whether the reply is eligible (it is streaming) and whether animation is enabled (the mode is not disabled), not "animate" and "visible". - Text splits into words only inside a text component, which provides a context saying so; paragraphs, headings and list items provide it. - There is no wrapper element per block. Each block component reads the flow and puts its spacing on its own element, and memo(Inlines) skips the content of the blocks that did not grow. - A list provides its items' layout (the marker kind), not a flow. Its items read the reply's flow, in which they have no previous sibling, so they are not spaced, as in the reference. - Paragraphs, headings and bold and italic runs resolve their text colour against the colour scheme. The previous-sibling map is now keyed by node rather than by id, since block and list-item ids are both positions and could collide.
Replies can now contain a ```ui fence: an indented tree of rows, columns, cards, titles, text, captions, badges, buttons and dividers. Every streamed reply has one, typically a row of option cards, and so do about a third of the others. The client parses the fence as it streams, so cards appear piece by piece, and unchanged subtrees keep their identity across parses. This gives the layout flow the work it does in the reference renderer. Every container provides its children a new flow on every render, and every node reads it and computes its spacing with that renderer's rules for the reply, a column and a row; a row with a fixed gap turns them off. A reply's buttons are disabled through the flow while it streams. A card's details stay out of its flow's sibling map until its Details button opens them, so opening them changes which sibling the nodes after them are spaced from. A new step, ExpandCards, opens the details of every card in the 200-message conversation one at a time.
The draft moves from App into a memoized Composer, as on the reference chat app, whose composer does not re-render the transcript. A keystroke now re-renders only the composer, not the app root and every message. App passes the composer a send callback that changes only with the active conversation and the stream, so the composer also skips the renders a streamed chunk causes. Typing no longer makes every message's memo bail, so it no longer pays the context walks those bails took after a card had updated its own state.
Measured on the reference chat app's live site, its composer: - sizes its textarea to its content in CSS (field-sizing: content, one 24 px row up to eight), so no script reads layout while typing; - shows its send button once the draft has text; - expands, the textarea over a row of actions, once a ResizeObserver on the textarea sees the text wrap, and animates its frame to the content's height, which another ResizeObserver measures; - publishes its height, from a ResizeObserver on the footer, as a CSS variable the page lays out around it, since it overlays the transcript; - saves the draft to localStorage 250 ms after the last change and at least once a second while it changes, and restores it on load. The composer now does the same. A step that types ends by settling the pending draft save and the height transition through window.flushComposer(), so none of that work lands in a later step, and the suite's prepare empties the composer and the saved draft through window.resetComposer(), so every iteration starts from the same state. TypePrompt lays out each keystroke, as each gets its own frame when a person types.
… key events The composer's state is now one signal scope (composer-state.tsrx), as the reference chat app's composer controller is: the draft, focus, the streaming state and what the ResizeObservers measure, and, derived from them, the layout, the presentation controls, the submit inputs, whether the draft can be submitted and what the actions show, under the reference's names. The textarea's value is bound to the draft signal, and each of the composer's parts reads the signals it shows as it renders, so a keystroke re-renders the parts whose state it changes. The composer moves to its own module, Composer.tsrx. Since a component reads signals as it renders, Octane tracks the signal reads of every component's render, in the transcript too, as on the reference. That makes the other steps 20-50% slower: one signal-reading component added to the previous version costs them as much. TypePrompt now focuses the composer and types each character with the events a keyboard produces: keydown, keypress, beforeinput, the edit, input and keyup.
Contributor
Author
|
I've added a lot more complexity to this benchmark. The times have gone up quite a bit. Maybe too much? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An LLM chat client in Octane (chat-stream/octane/App.tsrx): a sidebar of conversations, a message list and a composer, with replies streamed a few tokens at a time and rendered as memoized Markdown. It commits like the Octane fixture of the chat-stream benchmark in the Octane repository: event handlers only set state, which the steps commit through window.__benchFlush, and each streamed chunk commits with flushSync.
The corpus, Markdown parser, network stub and stylesheet are in shared/, and the build is in dist/octane.