Skip to content

Add test detecting overlapping pxr/ files across distributions#5391

Closed
AntoineRichard wants to merge 1 commit intoisaac-sim:developfrom
AntoineRichard:antoiner/test/usd-pxr-file-clashes
Closed

Add test detecting overlapping pxr/ files across distributions#5391
AntoineRichard wants to merge 1 commit intoisaac-sim:developfrom
AntoineRichard:antoiner/test/usd-pxr-file-clashes

Conversation

@AntoineRichard
Copy link
Copy Markdown
Collaborator

Summary

Adds a strict, packaging-level detector under source/isaaclab/test/install_ci/ that fails if any path under pxr/ is claimed by more than one installed distribution's RECORD.

This is intentionally opened without the fix from #5386, so CI fires and proves the detector works on the current broken state (usd-core==25.8.0 pinned alongside usd-exchange>=2.2, both of which ship the entire pxr/ tree — 81 overlapping paths in a fresh install).

Motivation

Two distributions shipping the same file paths is a silent packaging bug: whichever pip installs last overwrites the other's .so files on disk, while each dist-info keeps its own RECORD hashes. If the two were built against different USD ABIs, you get machine-specific crashes like Tf_PyEnumWrapper has not been created yet (see #5025). A runtime from pxr import Usd check is an unreliable detector — whether it crashes depends on install order, wheel resolution, and pip version, so the same pins work on one machine and fail on another. File-ownership overlap is the stable, environment-independent signal.

Expectations

  • install-ci job is expected to fail on this PR. That's the demonstration — the detector correctly flags the current develop state as broken.
  • Once Fix usd-core 25.8 / usd-exchange ABI conflict by relaxing version pin #5386 merges, the detector may still fire if the pip resolver picks usd-core==25.8.0 within the relaxed >=25.8.0,<26.0.0 range. If so, that's a signal the fix should be narrowed to exclude the overlapping versions (or that usd-exchange's packaging should stop shipping pxr/).
  • Scoped to pxr/ only; can be generalized to any shared top-level package later.

Test plan

Related: #5025, #5386

Two USD distributions currently installed together — usd-core and
usd-exchange — both ship the full pxr/ Python tree. When pip installs
both, whichever lands last silently overwrites the other's files on
disk while each dist-info's RECORD keeps its own entries. If the two
were built against different USD ABIs, the resulting mix crashes at
import time with errors like "Tf_PyEnumWrapper has not been created
yet". Whether the crash appears depends on install order and wheel
resolution, so the same version pins can produce a working tree on
one machine and a broken one on another.

Add a strict, packaging-level detector under install_ci that
enumerates every installed distribution's RECORD, collects paths
under pxr/, and fails if any path is claimed by more than one
distribution. This catches the conflict state regardless of whether
the current env happens to have selected a working "winner". See
GitHub PR isaac-sim#5386 / issue isaac-sim#5025 for the usd-core/usd-exchange case.
@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Apr 24, 2026
Copy link
Copy Markdown

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Isaac Lab Review Bot

Summary

This PR adds a new CI test that detects when multiple installed Python distributions claim ownership of the same pxr/ files, which causes silent ABI clashes and machine-specific crashes. The implementation is clean, well-documented, and correctly uses importlib.metadata to inspect distribution RECORDs. The PR is intentionally designed to fail CI to prove the detector works.

Architecture Impact

Self-contained. This is a pure test addition under test/install_ci/ with no impact on runtime code. The test uses only stdlib modules (importlib.metadata, collections) plus pytest, introducing no new dependencies.

Implementation Verdict

Ship it — Minor suggestions below, but nothing blocking.

Test Coverage

This PR is a test. The test correctly:

  • Skips when usd-core is not installed (appropriate for environments without USD)
  • Produces actionable output listing all clashing paths and their owning distributions
  • References the related issue/PR in the failure message for debugging context

CI Status

No CI checks available yet. Per the PR description, the install-ci job is expected to fail — that's the point of this PR (to prove the detector catches the known usd-core/usd-exchange conflict before #5386 fixes it).

Findings

🔵 Improvement: test_usd_pxr_file_clashes.py:47-49 — Consider checking for any pxr-shipping distribution, not just usd-core

The skip condition only checks for usd-core, but the clash could theoretically occur between other distributions (e.g., usd-exchange and some future usd-foo). If usd-core is removed but usd-exchange remains, the test would skip even though clashes could still exist.

Consider:

owners = _pxr_file_owners()
if not owners:
    pytest.skip("No pxr/ files found in any installed distribution")

This makes the test more robust to future packaging changes. However, the current implementation is acceptable since the known problematic case is specifically usd-core + usd-exchange.

🔵 Improvement: test_usd_pxr_file_clashes.py:33-42 — Minor: dist.files iteration could benefit from early filtering

The current implementation iterates all files for all distributions. For large environments this is fine (metadata is small), but you could add a fast-path check for distributions known to ship pxr/:

# Optional optimization: only check distributions that could ship pxr/
PXR_DISTS = {"usd-core", "usd-exchange", "usd"}
for dist in md.distributions():
    if dist.name is None or dist.name.lower() not in PXR_DISTS or dist.files is None:
        continue

This is a minor optimization and the current approach is more thorough (catches unknown future offenders), so the existing code is arguably better.

🔵 Improvement: test_usd_pxr_file_clashes.py:40 — Path matching could use pathlib for clarity

if path == "pxr" or path.startswith("pxr/"):

This works correctly, but using pathlib.PurePosixPath would be more explicit:

from pathlib import PurePosixPath
p = PurePosixPath(path)
if p.parts and p.parts[0] == "pxr":

The current string-based approach is fine and arguably clearer for this simple case.

🔵 Improvement: CHANGELOG.rst:4 — Date format uses future year (2026)

The changelog entry shows 4.6.13 (2026-04-24). This appears to be a placeholder/template date. If this is intentional project convention, ignore this comment.


Overall: This is a well-designed, focused PR that adds valuable CI protection against a nasty class of packaging bugs. The implementation is correct, the documentation is thorough, and the test output is actionable. The expected CI failure is explicitly documented. Ready to merge once CI demonstrates the expected failure behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant