Skip to content

[Web] Retain evicted shape tuples until teardown - #20130

Open
atirna wants to merge 2 commits into
apache:mainfrom
atirna:fix/web-runtime-shape-cache-lifetime
Open

[Web] Retain evicted shape tuples until teardown#20130
atirna wants to merge 2 commits into
apache:mainfrom
atirna:fix/web-runtime-shape-cache-lifetime

Conversation

@atirna

@atirna atirna commented Aug 15, 2026

Copy link
Copy Markdown

Summary

  • retain ShapeTuples evicted from the bounded LRU until CacheState.dispose()
  • dispose each retained or still-cached tuple exactly once at TVM-instance teardown
  • add a regression test for the live borrowed-reference lifecycle

This prevents an LRU miss from disposing a tuple which a pending GPU dispatch may still use.

Validation

  • npx jest --config jest.config.js --runTestsByPath tests/node/test_cache_state.js --runInBand
  • npx eslint -c .eslintrc.cjs src/cache_state.ts tests/node/test_cache_state.js
  • npx tsc --noEmit -p tsconfig.json

The full npm test requires generated dist/ WASM artifacts not present in the sparse checkout; the existing integration suites cannot resolve ../../dist in that environment.

Relates to mlc-ai/web-llm#844.

Signed-off-by: Atirna <288419661+atirna@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants