Ship a prebuilt C++ SDK in the ExecuTorch Linux wheel - #21470
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21470
Note: Links to docs will display an error until the docs builds have been completed. ❌ 33 New Failures, 160 Pending, 11 Unclassified FailuresAs of commit 1bc3075 with merge base 0b13b6a ( NEW FAILURES - The following jobs have failed:
UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
d6aa74d to
86c35bd
Compare
## Why this is needed Today, using the ExecuTorch runtime from a C++ program means building ExecuTorch from source: clone the repo, sync submodules, and run a CMake build before you can compile and link your own runner. The pip wheel only ships the Python runtime module (`_portable_lib`) plus a small set of headers meant for authoring custom operators. There is no way to just `pip install executorch` and link a standalone C++ application against the runtime. This is friction for anyone whose deployment path is C++ (the common case for on-device inference) and who already has the wheel installed for export. The runtime is compiled during the wheel build and then discarded. This change ships the runtime as a linkable shared library, its public headers, and a CMake package config inside the Linux wheel, so a C++ program can link the ExecuTorch runtime with no source checkout and no separate build. ## What is inside Added to the Linux wheel (nothing removed; other platforms unchanged): - `executorch/lib/libexecutorch.so` (SONAME-versioned, with the standard `libexecutorch.so -> .so.1 -> .so.<version>` chain): the consolidated shared runtime. It bundles the runtime core plus the common runtime extensions (module, tensor, data_loader, flat_tensor, named_data_map). - `executorch/include/executorch/extension/...`: the public headers for the Module, Tensor, DataLoader, FlatTensor (.ptd reader), NamedDataMap, and header-only MallocMemoryAllocator APIs. Runtime/Program/backend headers were already shipped and are reused. - `executorch/share/cmake/executorch-config.cmake`: a CMake package config that exposes an `executorch::runtime` imported target (plus convenience aliases `executorch::core`, `executorch::extension_*`). - `executorch/utils/cmake_prefix_path`: a small helper (mirrors `torch.utils.cmake_prefix_path`) so CMake can find the config in one line. ## Why a shared library (not static archives) The runtime is shipped shared on purpose. ExecuTorch keeps a single process-global backend/kernel registry. Shipping the runtime as one shared `libexecutorch.so` lets a separately distributed backend or delegate shared library register into that one registry: the backend `.so` is built without its own copy of the runtime (its `register_backend` reference is undefined and resolves against `libexecutorch.so` at load), and its static-init registration runs when the `.so` is loaded (via whole-archive for a C++ app, or an explicit import/dlopen for Python, which is how ExecuTorch already ships the QNN backend today). Static archives would give each consumer its own private registry, which cannot support loading multiple independently distributed backends into one runtime. The set is intentionally libtorch-free and excludes the general CPU operator/kernel libraries, because a delegate supplies its own compute. It is also Linux only: the `.so` naming and SONAME symlink chain are Unix specific, so the Windows and macOS wheels are byte-identical to before. ## How to use it ```bash pip install executorch ``` ```cmake find_package(executorch CONFIG REQUIRED) add_executable(my_runner main.cpp) target_link_libraries(my_runner PRIVATE executorch::runtime) ``` ```bash cmake -S . -B build \ -DCMAKE_PREFIX_PATH="$(python -c 'import executorch.utils as u; print(u.cmake_prefix_path)')" cmake --build build ``` Existing consumers that use `find_package(executorch)` to link the Python `_portable_lib` for custom-op extensions keep working unchanged. The new C++ SDK availability is reported separately via `EXECUTORCH_SDK_FOUND`, so the legacy `EXECUTORCH_FOUND` / `EXECUTORCH_LIBRARIES` contract is preserved. ## Test plan Verified on Linux x86_64: - Built the wheel with `python setup.py bdist_wheel` and confirmed it contains `libexecutorch.so` with its SONAME symlink chain, the CMake config, the `utils` helper, and the new extension headers, and that headers with no shipped implementation are excluded. - Installed the wheel into a clean virtual environment and built a small standalone C++ runner against it with `find_package(executorch)` and `executorch.utils.cmake_prefix_path`. The runner compiled, linked, ran, and initialized the runtime. - Built a separate "coreless" backend shared library (no bundled runtime, `register_backend` left undefined) and confirmed that loading it against the installed `libexecutorch.so` registers the backend into the runtime's registry (`get_backend_class` goes from not-found to found). This is the mechanism that lets independently distributed backends coalesce into one runtime. - Confirmed the runner and the runtime link no libtorch/libc10 (via `ldd`). - Confirmed the Windows and macOS wheel code paths add nothing new, so those wheels are unaffected. - Lint and format pass (flake8, ufmt, cmake-format). Known limitation: the wheel-build step stores the SONAME symlinks as plain file copies rather than symlinks. Linking and loading still work because the SONAME target is present as a real file; a follow-up can preserve them as true symlinks to save space. ## CI Adds a PR job (test-cpp-sdk-wheel-linux in pull.yml, calling .ci/scripts/test_cpp_sdk_wheel.sh) that builds the wheel, installs it into a clean venv, and uses find_package(executorch) to build a C++ consumer linking executorch::runtime, then loads a coreless backend shared library and asserts it registers into libexecutorch.so. This gives the feature direct coverage: before, no PR job built the full wheel or linked the C++ SDK.
86c35bd to
1bc3075
Compare
|
Superseded by #21477, which submits the same change via ghstack so follow-up work can stack on top. Closing in favor of the ghstack version. |
Why this is needed
Today, using the ExecuTorch runtime from a C++ program means building
ExecuTorch from source: clone the repo, sync submodules, and run a CMake
build before you can compile and link your own runner. The pip wheel only
ships the Python runtime module (
_portable_lib) plus a small set ofheaders meant for authoring custom operators. There is no way to just
pip install executorchand link a standalone C++ application against theruntime.
This is friction for anyone whose deployment path is C++ (the common case
for on-device inference) and who already has the wheel installed for
export. The runtime is compiled during the wheel build and then discarded.
This change ships the runtime as a linkable shared library, its public
headers, and a CMake package config inside the Linux wheel, so a C++
program can link the ExecuTorch runtime with no source checkout and no
separate build.
What is inside
Added to the Linux wheel (nothing removed; other platforms unchanged):
executorch/lib/libexecutorch.so(SONAME-versioned, with the standardlibexecutorch.so -> .so.1 -> .so.<version>chain): the consolidatedshared runtime. It bundles the runtime core plus the common runtime
extensions (module, tensor, data_loader, flat_tensor, named_data_map).
executorch/include/executorch/extension/...: the public headers for theModule, Tensor, DataLoader, FlatTensor (.ptd reader), NamedDataMap, and
header-only MallocMemoryAllocator APIs. Runtime/Program/backend headers
were already shipped and are reused.
executorch/share/cmake/executorch-config.cmake: a CMake package configthat exposes an
executorch::runtimeimported target (plus conveniencealiases
executorch::core,executorch::extension_*).executorch/utils/cmake_prefix_path: a small helper (mirrorstorch.utils.cmake_prefix_path) so CMake can find the config in one line.Why a shared library (not static archives)
The runtime is shipped shared on purpose. ExecuTorch keeps a single
process-global backend/kernel registry. Shipping the runtime as one shared
libexecutorch.solets a separately distributed backend or delegate sharedlibrary register into that one registry: the backend
.sois built withoutits own copy of the runtime (its
register_backendreference is undefinedand resolves against
libexecutorch.soat load), and its static-initregistration runs when the
.sois loaded (via whole-archive for a C++ app,or an explicit import/dlopen for Python, which is how ExecuTorch already
ships the QNN backend today). Static archives would give each consumer its
own private registry, which cannot support loading multiple independently
distributed backends into one runtime.
The set is intentionally libtorch-free and excludes the general CPU
operator/kernel libraries, because a delegate supplies its own compute. It
is also Linux only: the
.sonaming and SONAME symlink chain are Unixspecific, so the Windows and macOS wheels are byte-identical to before.
How to use it
Existing consumers that use
find_package(executorch)to link the Python_portable_libfor custom-op extensions keep working unchanged. The newC++ SDK availability is reported separately via
EXECUTORCH_SDK_FOUND, sothe legacy
EXECUTORCH_FOUND/EXECUTORCH_LIBRARIEScontract ispreserved.
Test plan
Verified on Linux x86_64:
python setup.py bdist_wheeland confirmed itcontains
libexecutorch.sowith its SONAME symlink chain, the CMakeconfig, the
utilshelper, and the new extension headers, and thatheaders with no shipped implementation are excluded.
standalone C++ runner against it with
find_package(executorch)andexecutorch.utils.cmake_prefix_path. The runner compiled, linked, ran,and initialized the runtime.
register_backendleft undefined) and confirmed that loading it againstthe installed
libexecutorch.soregisters the backend into the runtime'sregistry (
get_backend_classgoes from not-found to found). This is themechanism that lets independently distributed backends coalesce into one
runtime.
ldd).wheels are unaffected.
Known limitation: the wheel-build step stores the SONAME symlinks as plain
file copies rather than symlinks. Linking and loading still work because the
SONAME target is present as a real file; a follow-up can preserve them as
true symlinks to save space.
CI
Adds a PR job (test-cpp-sdk-wheel-linux in pull.yml, calling
.ci/scripts/test_cpp_sdk_wheel.sh) that builds the wheel, installs it into a
clean venv, and uses find_package(executorch) to build a C++ consumer linking
executorch::runtime, then loads a coreless backend shared library and asserts
it registers into libexecutorch.so. This gives the feature direct coverage:
before, no PR job built the full wheel or linked the C++ SDK.