Skip to content

runtime, internal/poll, compileopts: fix wasm >2GiB heap growth, wasip1 WaitWrite, and map/slice growth - #6049

Closed
teamchong wants to merge 1 commit into
tinygo-org:devfrom
teamchong:wasm-wasip1-large-heap-and-poll
Closed

teamchong wants to merge 1 commit into
tinygo-org:devfrom
teamchong:wasm-wasip1-large-heap-and-poll

Conversation

@teamchong

@teamchong teamchong commented Oct 27, 2026 •

Copy link
Copy Markdown

Summary

Fixes several WebAssembly / wasip1 compatibility and memory-scaling issues encountered when compiling typescript-go (tsgo) to wasip1:

  1. src/runtime/arch_tinygowasm.go (growHeap):

    • wasm_memory_size(wasmMemoryIndex) * wasmPageSize previously multiplied as signed int32 before converting to uintptr, overflowing to negative at 32,768 pages (2 GiB) and wrapping to 0 at 65,536 pages (4 GiB).
    • Doubling memorySize in a single wasm_memory_grow call always failed once linear memory passed 2 GiB (requesting >4 GiB) or when a host runtime enforced a ceiling below 2 * memorySize.
    • Cap single-step growth at 1,024 pages (64 MiB) up to 65,535 pages (0xffff0000 bytes) with halving backoff on failure, and convert wasm_memory_size to uintptr before multiplying by wasmPageSize.
  2. src/internal/poll/fd_wasip1.go ((*FD).WaitWrite, wait, parkUntil):

    • Go 1.26+ builds net/fd_unix.go on wasip1 (//go:build unix || js || wasip1), where (*netFD).connect calls fd.pfd.WaitWrite(). Add (*FD).WaitWrite delegating to wait / parkUntil(pollModeWrite, fd.wDeadline).
    • Guard wait and parkUntil with if task.OnSystemStack() so -target=wasip1 -scheduler=none builds dead-code-eliminate time.AfterFunc (internal/task.start) and runtime_netpoll_* (which is only built under scheduler.tasks || scheduler.asyncify).
  3. compileopts/config.go (LDFlags):

    • Forward -stack-size as -z stack-size=<bytes> to wasm-ld when -scheduler=none is used, since there are no goroutine stacks and execution uses the linear shadow stack directly.
  4. src/runtime/hashmap.go (hashmapCopy, hashmapClear):

    • Walk existing buckets directly in hashmapCopy instead of allocating temporary key and value buffers on the heap and copying every entry twice via hashmapNext.
    • Return early in hashmapClear when m.count == 0 (matching hashmapDelete).
  5. src/runtime/slice.go (sliceGrow):

    • Return early without allocating when elemSize == 0.
    • Keep power-of-two rounding for oldCap < 256 and use standard Go's 1.25x growth formula (growCap += (growCap + 3*256) / 4) for oldCap >= 256 to avoid up to 50% capacity waste on large slices.

…p1 WaitWrite, and map/slice growth

- Cap wasm growHeap step size to 64 MiB with backoff up to 65535 pages and cast wasm_memory_size to uintptr before multiplying by wasmPageSize so heaps past 2 GiB do not overflow int32 or fail doubling past 4 GiB.
- Add (*FD).WaitWrite in internal/poll/fd_wasip1.go for Go 1.26+ net/fd_unix.go compatibility on wasip1, and guard wait/parkUntil with task.OnSystemStack() so -scheduler=none builds do not pull in time.AfterFunc goroutines or netpoll symbols.
- Pass -z stack-size to wasm-ld when -scheduler=none and -stack-size are set.
- Walk buckets directly in hashmapCopy instead of heap-allocating temporary key/value buffers per rehash, and skip hashmapClear when count is 0.
- Use 1.25x slice capacity growth for oldCap >= 256 in sliceGrow instead of rounding large slices to the next power of two.
@teamchong
teamchong force-pushed the wasm-wasip1-large-heap-and-poll branch from fabf3ee to 8471d34 Compare October 27, 2026 06:25
@dgryski

dgryski commented Oct 27, 2026

Copy link
Copy Markdown
Member

@jakebailey Seems like your wheelhouse.

@dgryski

dgryski commented Oct 27, 2026

Copy link
Copy Markdown
Member

This is too many fixes in a single commit. Can you please split these up into at least different commits, if not different PRs, so we can discuss them indivudually? Thanks.

@davecheney

Copy link
Copy Markdown
Contributor

@teamchong thank you very much for your commit. As @dgryski suggested, would you please split this up into separate PRs, if they are related in a series, perhaps try GitHub's stack's feature.

I am going to close this PR as it stands, not because I am not grateful for your contribution, but because I do not see a straight forward path to merge this as presented. Thank you and I hope to see your contributions shortly.

@davecheney davecheney closed this Oct 27, 2026
@teamchong
teamchong deleted the wasm-wasip1-large-heap-and-poll branch October 27, 2026 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants