-
Notifications
You must be signed in to change notification settings - Fork 198
Docs: Node comparison report #7346
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
5b00dd2
node comparison report in docs
sudo-shashank c6b4474
fix spellcheck
sudo-shashank 9364967
resize charts
sudo-shashank fa62a21
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank 0f80b99
fmt
sudo-shashank 77769dd
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank 2aa724f
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank 3da2abb
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank 68b060e
new section
sudo-shashank 37d8205
fmt
sudo-shashank 12dc461
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank 937c968
cleanup
sudo-shashank 4003d95
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank cc54a72
Some improvements
sudo-shashank e583ad3
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank e2de7da
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank 5a4ae0f
Use new charts
sudo-shashank 9295fbe
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank 8734aee
Add more comparisons
sudo-shashank bfae341
fmt
sudo-shashank e153bcf
some improvements
sudo-shashank 1d2376d
grammarly suggestions
sudo-shashank 9ef09ca
Use correct hardware info
sudo-shashank 4b2890b
grammarly suggestion
sudo-shashank d5076f3
Merge branch 'main' into shashank/node-comparison-report
sudo-shashank 0e9378f
some more corrections
sudo-shashank File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "label": "Node Comparison", | ||
| "position": 2 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| --- | ||
| sidebar_position: 1 | ||
| title: Filecoin Node Comparison | ||
| --- | ||
|
|
||
| # Filecoin Node Comparison | ||
|
|
||
| There are three full-node implementations of the Filecoin protocol. | ||
|
|
||
| - **[Forest](https://github.com/ChainSafe/forest)** is written in Rust and maintained by [ChainSafe Systems](https://chainsafe.io). It focuses on chain validation, a high-performance RPC API, and snapshot generation, all with low hardware requirements. | ||
| - **[Lotus](https://github.com/filecoin-project/lotus)** is the reference implementation, written in Go. It provides the complete Filecoin feature set, including the storage-provider stack, and is typically where new protocol features land first. | ||
| - **[Venus](https://github.com/filecoin-project/venus)** is a modular Go implementation. Rather than a single daemon, it is a set of independently deployable components that together pioneered Filecoin's distributed storage pool model and offer a largely Lotus-compatible API. It aims to help small and medium-sized storage providers join the Filecoin network with a lower barrier to entry. | ||
|
|
||
| ## Feature comparison | ||
|
|
||
| | Capability | Forest | Lotus | | ||
| | ----------------------------------------------------------------- | ------------- | ----- | | ||
| | Chain synchronization and validation | Yes | Yes | | ||
| | Filecoin JSON-RPC API | Yes | Yes | | ||
| | Ethereum-compatible RPC (`eth_*`) | Yes | Yes | | ||
| | Snapshot export | Yes | Yes | | ||
| | Built-in and standalone wallet (`forest-wallet` / `lotus-wallet`) | Yes | Yes | | ||
| | Bootstrap node | Yes | Yes | | ||
| | F3 (Fast Finality) participation | Yes | Yes | | ||
| | Built-in health-check endpoints | Yes | Yes | | ||
| | Storage provider / sealing (`lotus-miner`, `lotus-worker`) | No | Yes | | ||
| | Block production / mining | No (untested) | Yes | | ||
| | Single binary for all networks (`--chain`) | Yes | No | | ||
| | Auto-download snapshot on startup | Yes | No | | ||
| | Lite / diff / archival snapshot generation | Yes | No | | ||
| | Offline RPC server from snapshot (`forest-tool api serve`) | Yes | No | | ||
| | Stateless node mode (`--stateless`) | Yes | No | | ||
| | Ethereum `trace_call` and `debug_traceTransaction` | Yes | No | | ||
| | Written in Rust 🦀 | Yes | No | | ||
|
|
||
| Forest and Lotus both expose a compatible Filecoin JSON-RPC API. Forest supports three API versions: `/rpc/v0` (deprecated; legacy Lotus-compatible methods), `/rpc/v1` (stable and recommended for production), and `/rpc/v2` (experimental; still being rolled out). See the [JSON-RPC overview](../../reference/json-rpc/overview.md), [methods reference](../../reference/json-rpc/methods.mdx), and [API conformance report](../api_conformance/report_2026-07-01.md) for supported methods and Lotus API parity. | ||
|
|
||
| ## Performance | ||
|
|
||
| For a comparable RPC workload, Forest serves requests with lower latency while using less CPU and memory than Lotus, and exports snapshots significantly faster while requiring fewer hardware resources. See the [RPC Performance Comparison](./rpc_comparison.md) and [Snapshot Generation Comparison](./snapshot_comparison.md) for the full figures and methodology. |
113 changes: 113 additions & 0 deletions
113
docs/docs/users/reports/node_comparison/rpc_comparison.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| --- | ||
| sidebar_position: 2 | ||
| title: RPC Performance Comparison | ||
| --- | ||
|
|
||
| # RPC Performance: Forest vs Lotus | ||
|
|
||
| The following measurements were taken on 17 July 2026, while a Forest node and a Lotus node handled a similar stream of real RPC provider traffic during the same time window. Because both nodes served an almost identical request stream, the differences below are attributable to the nodes themselves rather than to uneven load. The "Comparable load" section documents the checks used to support that assumption. | ||
|
|
||
| ## Environment | ||
|
|
||
| Node builds and hardware used for these measurements: | ||
|
|
||
| | Node | Version | vCPUs | RAM | | ||
| | ------ | ------------------------ | ----- | ----- | | ||
| | Forest | `0.34.1+git.0ba362e` | 12 | 64 GB | | ||
| | Lotus | `1.36.1-rc1+git.45eade8` | 12 | 80 GB | | ||
|
|
||
| ## RPC latency | ||
|
|
||
| Average wall time per HTTP request was approximately 10-15 ms for Forest and 30-50 ms for Lotus over most of the measurement window. Both traces rose toward the end as the load increased, with Forest reaching about 30 ms and Lotus about 48 ms. | ||
|
|
||
|  | ||
|
|
||
| ### Per-flow latency | ||
|
|
||
| At the median (P50), the two nodes were close: approximately 7 ms for Forest and 8 ms for Lotus. The difference was larger in the tail. At P95, Forest stayed near 50 ms and rose to roughly 115 ms under load, while Lotus stayed around 70-85 ms and reached roughly 185 ms. At P99, Forest held around 200 ms and rose to about 350 ms, while Lotus ranged from approximately 240 ms to a peak near 600 ms. | ||
|
|
||
| <p align="center"> | ||
| <img | ||
| src="/img/reports/rpc_comparison/p50-per-flow-latency.png" | ||
| alt="P50 per-flow latency" | ||
| style={{ maxWidth: '680px', width: '100%' }} | ||
| /> | ||
| </p> | ||
|
|
||
| <p align="center"> | ||
| <img | ||
| src="/img/reports/rpc_comparison/p95-per-flow-latency.png" | ||
| alt="P95 per-flow latency" | ||
| style={{ maxWidth: '680px', width: '100%' }} | ||
| /> | ||
| </p> | ||
|
|
||
| <p align="center"> | ||
| <img | ||
| src="/img/reports/rpc_comparison/p99-per-flow-latency.png" | ||
| alt="P99 per-flow latency" | ||
| style={{ maxWidth: '680px', width: '100%' }} | ||
| /> | ||
| </p> | ||
|
|
||
| ### Amortized latency | ||
|
|
||
| The amortized view accounts for batched requests. It shows the same pattern as the per-flow view: comparable values at P50 and a larger difference at P95, P99, and on average. | ||
|
|
||
| <p align="center"> | ||
| <img | ||
| src="/img/reports/rpc_comparison/p50-amortized-latency.png" | ||
| alt="P50 amortized latency" | ||
| style={{ maxWidth: '680px', width: '100%' }} | ||
| /> | ||
| </p> | ||
|
|
||
| <p align="center"> | ||
| <img | ||
| src="/img/reports/rpc_comparison/p95-amortized-latency.png" | ||
| alt="P95 amortized latency" | ||
| style={{ maxWidth: '680px', width: '100%' }} | ||
| /> | ||
| </p> | ||
|
|
||
| <p align="center"> | ||
| <img | ||
| src="/img/reports/rpc_comparison/p99-amortized-latency.png" | ||
| alt="P99 amortized latency" | ||
| style={{ maxWidth: '680px', width: '100%' }} | ||
| /> | ||
| </p> | ||
|
|
||
|  | ||
|
|
||
| ### Comparable load | ||
|
|
||
| The request rates reaching each node were similar, supporting a direct comparison between the two nodes. HTTP request counts averaged 45.2 `req/s` for Forest and 45.0 `req/s` for Lotus, with batch-aware counts at 20.4 and 20.3 `req/s`, respectively. Both nodes, therefore, received a comparable volume of traffic. | ||
|
|
||
|  | ||
|
|
||
|  | ||
|
|
||
| Concurrency followed from the latency difference: Forest kept fewer requests in flight at any instant (mean 0.43, peak 4) than Lotus (mean 1.45, peak 8). | ||
|
|
||
|  | ||
|
|
||
| ## Resource usage | ||
|
|
||
| While serving this workload, Forest used approximately 10-15% CPU, and Lotus used approximately 45-55%, briefly spiking to about 80% during the higher-load period. Forest's CPU trace was also less variable. This usage suggests Forest would likely run comfortably on 2-4 `vCPUs`. | ||
|
|
||
|  | ||
|
|
||
| Memory usage was approximately 8-10% for Forest and approximately 25-30% for Lotus, with Lotus briefly reaching about 40% during the higher-load period. | ||
|
|
||
|  | ||
|
|
||
| :::note | ||
| The absolute memory figures are higher than they appear here because of OS page caches, but the relative proportions between the two nodes still hold. | ||
| ::: | ||
|
|
||
| ## Related | ||
|
|
||
| - [Snapshot Generation Comparison](./snapshot_comparison.md) | ||
| - [Forest vs Lotus feature comparison](./index.md) | ||
| - [RPC performance Grafana dashboard](https://monitoring.chain.love/public-dashboards/229eaf7ce0224655abe26c288aab914b?orgId=1&refresh=5m&kiosk) | ||
29 changes: 29 additions & 0 deletions
29
docs/docs/users/reports/node_comparison/snapshot_comparison.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| --- | ||
| sidebar_position: 3 | ||
| title: Snapshot Generation Comparison | ||
| --- | ||
|
|
||
| # Snapshot Generation: Forest vs Lotus | ||
|
|
||
| Forest plays an important role in the timely, efficient generation of network snapshots. Lotus can also generate a snapshot, but doing so is significantly slower and more expensive. On a regular machine, Forest completes a basic chain snapshot export much faster than Lotus and with a fraction of the memory. | ||
|
|
||
| <p align="center"> | ||
| <img | ||
| src="/img/reports/snapshot_comparison/snapshot-export-comparison.png" | ||
| alt="Snapshot export duration: Forest vs Lotus" | ||
| style={{ maxWidth: '480px', width: '100%' }} | ||
| /> | ||
| </p> | ||
|
|
||
| Both implementations produce compatible snapshots and can consume snapshots produced by the other. | ||
|
|
||
| :::note | ||
|
sudo-shashank marked this conversation as resolved.
|
||
| Lotus snapshots are not compressed by default, whereas Forest compresses them during export. Both implementations can consume compressed snapshots. Compressing a Lotus snapshot after generation is possible, but it adds further to the total time. | ||
|
|
||
| Forest snapshots (`.forest.car.zst`) also embed an index that lets Forest use the file directly as a read-only blockstore, serving blocks in place without importing them into a database. The index is stored in skippable frames, so Lotus and other tools can still read the snapshot; they just ignore the index. Reading a snapshot in place this way is currently Forest-only. | ||
| ::: | ||
|
sudo-shashank marked this conversation as resolved.
|
||
|
|
||
| ## Related | ||
|
|
||
| - [RPC Performance Comparison](./rpc_comparison.md) | ||
| - [Forest vs Lotus feature comparison](./index.md) | ||
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+296 KB
docs/static/img/reports/snapshot_comparison/snapshot-export-comparison.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.