Skip to content

Add utilities for easier memory benchmarking.#27000

Merged
CraigMacomber merged 6 commits intomicrosoft:mainfrom
CraigMacomber:benchMemEasy
Apr 13, 2026
Merged

Add utilities for easier memory benchmarking.#27000
CraigMacomber merged 6 commits intomicrosoft:mainfrom
CraigMacomber:benchMemEasy

Conversation

@CraigMacomber
Copy link
Copy Markdown
Contributor

Description

It is very easy to screw up writing memeory benchmarks.

These changes make it much easier to get them right.

Reviewer Guidance

The review process is outlined on this wiki page.

Copilot AI review requested due to automatic review settings April 10, 2026 23:27
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds higher-level utilities to help authors write correct Node.js memory benchmarks with fewer lifecycle/retention pitfalls, and adjusts the memory benchmarking defaults to be safer in the presence of finalizers.

Changes:

  • Introduces Box<T>, memoryAddedBy(...), and memoryUseOfValue(...) to simplify common memory benchmark patterns.
  • Changes MemoryUseBenchmark.enableAsyncGC default to true, with targeted test updates to keep non-async-GC baselines.
  • Updates package exports, changelog, and API report to reflect the new public surface.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tools/benchmark/src/memoryBenchmarking/memoryUtils.ts Adds new public helpers (Box, memoryAddedBy, memoryUseOfValue) for common memory benchmark authoring patterns.
tools/benchmark/src/memoryBenchmarking/getMemoryUse.ts Switches default enableAsyncGC behavior and expands benchmarkMemoryUse docs to point to the new helpers.
tools/benchmark/src/memoryBenchmarking/index.ts Re-exports the new memory benchmarking utilities from the memory benchmarking entrypoint.
tools/benchmark/src/memoryBenchmarking/configuration.ts Updates guidance docs to mention the new helper APIs and Box for retention control.
tools/benchmark/src/index.ts Exposes the new utilities from the package root entrypoint.
tools/benchmark/src/test/memoryBenchmarking/memoryUtils.spec.ts Adds unit tests for Box and benchmark-based coverage/examples for the new helpers.
tools/benchmark/src/test/memoryBenchmarking/getMemoryUse.spec.ts Pins enableAsyncGC: false for the “empty” baselines now that async GC defaults to true.
tools/benchmark/CHANGELOG.md Documents the new utilities and the enableAsyncGC default change.
tools/benchmark/api-report/benchmark.public.api.md Updates public API report to include the new exports.
Comments suppressed due to low confidence (1)

tools/benchmark/src/memoryBenchmarking/getMemoryUse.ts:87

  • enableAsyncGC now defaults to true here, but the public TSDoc on MemoryUseBenchmark.enableAsyncGC in memoryBenchmarking/configuration.ts still says it defaults to false. Please update the documentation to match the new behavior so API docs don’t mislead benchmark authors.
const defaults: Required<Omit<MemoryUseBenchmark, "benchmarkFn">> = {
	enableAsyncGC: true,
	logProcessedData: false,
	logRawData: false,
	warmUpIterations: 12,
	keepIterations: 10,
};

Comment thread tools/benchmark/src/memoryBenchmarking/memoryUtils.ts
Comment thread tools/benchmark/src/memoryBenchmarking/memoryUtils.ts
Comment thread tools/benchmark/src/memoryBenchmarking/memoryUtils.ts
Comment thread tools/benchmark/src/test/memoryBenchmarking/memoryUtils.spec.ts Outdated
Comment thread tools/benchmark/src/test/memoryBenchmarking/memoryUtils.spec.ts Outdated
Comment thread tools/benchmark/src/test/memoryBenchmarking/memoryUtils.spec.ts Outdated
* When implementing other cases, consider using {@link Box} to manage memory retention issues due to local variables.
*
* @public
* @input
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forget the exact semantics that @input intends to convey... this type does not satisfy them anymore because free functions return it, even if the functions are controlled by us and the returned object is only supposed to work as an input to other APIs in the package?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This type is "input," meaning that code outside of the library defining it should not read from it. Future versions of this type may add optional members or make typing of members more general.

Since we never do non-breaking releases of benchmark, it doesn't really matter, so I figured we might as well allow people to read from the output of these new functions. I don't think that's really required if they are only passed right back into our own APIs as the same type or used with ... into the same type, but as we don't do non breaking releases, relaxing the rules seems fine.

return this.inner;
}

public set value(v: T) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public set value(v: T) {
/**
* The value to store in the box.
* @throws if the value is undefined.
*/
public set value(v: T) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do that and API extractor will complain the documentation is supposed to be on the getter instead.

Both share the same doc comment, in intellisense and API extractor.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, interesting. Ok, pretty minor.

@CraigMacomber CraigMacomber merged commit 00985e1 into microsoft:main Apr 13, 2026
28 checks passed
@CraigMacomber CraigMacomber deleted the benchMemEasy branch April 13, 2026 16:45
@github-actions
Copy link
Copy Markdown
Contributor

🔗 No broken links found! ✅

Your attention to detail is admirable.

linkcheck output


> fluid-framework-docs-site@0.0.0 ci:check-links /home/runner/work/FluidFramework/FluidFramework/docs
> start-server-and-test "npm run serve -- --no-open" 3000 check-links

1: starting server using command "npm run serve -- --no-open"
and when url "[ 'http://127.0.0.1:3000' ]" is responding with HTTP status code 200
running tests using command "npm run check-links"


> fluid-framework-docs-site@0.0.0 serve
> docusaurus serve --no-open

[SUCCESS] Serving "build" directory at: http://localhost:3000/

> fluid-framework-docs-site@0.0.0 check-links
> linkcheck http://localhost:3000 --skip-file skipped-urls.txt

Crawling...

Stats:
  279037 links
    1880 destination URLs
    2126 URLs ignored
       0 warnings
       0 errors


agarwal-navin pushed a commit to agarwal-navin/FluidFramework that referenced this pull request Apr 13, 2026
## Description

It is very easy to screw up writing memeory benchmarks.

These changes make it much easier to get them right.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants