Repository navigation
Run the Python test suite in parallel with pytest-xdist - #17520
Conversation
The main pytest run has been taking six to eight minutes in CI in a single process. With -n auto on the four-vCPU hosted runners each worker gets its own in-memory test database and pytest-cov combines the per-worker data. PYTEST_XDIST_WORKERS=0 runs the suite in one process again when debugging. Running in parallel changes which test precedes which, and that exposes state one test leaves for the next. Three autouse fixtures in bedrock/conftest.py close the gaps found while doing the same in Springfield: the Django cache is cleared before each test, because Wagtail keeps its site root paths there and a database rollback does not invalidate them; the URL resolvers are reset, because Django caches reversed URLs per language while our LocalePrefixPattern reads the language through a function some tests mock, so a mocked test can poison the cache for the real language; and the active language is deactivated after each test. AcceptedLocalesTest used to move the real data/www-l10n checkout aside for the duration of the class, which under xdist can leave another worker reading an empty Fluent repo. It now builds its stand-in locale tree in a temporary directory and patches FLUENT_REPO_PATH to point at it. The integration test runner moves from pytest-parallel, which has had no release since 2021 and hangs on macOS, to xdist as well. PYTEST_PROCESSES maps straight onto -n, and tblib leaves the lock with it.
There was a problem hiding this comment.
🔵 Needs a closer look
Per-worker collection with live HTTP-generated download parametrization requires deployment-backed execution of the full integration matrix to verify consistent collections and report generation.
No open issues 🎉
What changed in this PR
This PR enables parallel Python testing with pytest-xdist and improves test isolation to prevent shared state and filesystem changes from interfering with workers.
Changes:
- Replace pytest-parallel with pytest-xdist for unit and integration test runners, retaining configurable worker counts.
- Add per-test cache, URL resolver, and language-state cleanup under
bedrock/. - Move locale test data into a temporary directory instead of modifying the shared localization checkout.
| File | Description |
|---|---|
uv.lock |
Lock pytest-xdist and execnet; remove pytest-parallel and tblib. |
pyproject.toml |
Replace the parallel-testing dependency with pytest-xdist. |
bin/run-tests.sh |
Enable automatic worker selection with a serial-debugging override. |
bin/integration_tests/run_integration_tests.sh |
Map the existing process setting to xdist’s -n option. |
bedrock/conftest.py |
Add fixtures that isolate cache, URL resolver, and language state. |
bedrock/base/tests/test_accepted_locales.py |
Use a temporary locale tree and patch its configured path. |
Review effort: Balanced
|
@copilot please fix the merge conflicts in this pull request. |
# Conflicts: # bin/run-tests.sh Co-authored-by: stevejalim <101457+stevejalim@users.noreply.github.com>
|
Thanks for this, the parallel run is a nice win on a big machine. 😊 I ran the suite several times. 3 runs each:
So the gain only shows up with many cores. With few cores or serial, the autouse fixtures in
Do you know how many cores CI has? If it's 4 or fewer, this may not speed it up. If you get a chance, could the fixtures be scoped? E.g. clear only after tests that mock Smaller question: |
|
@janriokrause the runners have 4 vcpu and based on what we saw on Springfield we do get a speed bump. Let's merge and see |
The main pytest run takes six to eight minutes in CI in a single process. With
-n autoon the four-vCPU hosted runners each worker gets its own in-memory test database and pytest-cov combines the per-worker data.PYTEST_XDIST_WORKERS=0runs the suite in one process again when debugging;pytest bedrockoutside Docker was already serial.What parallel ordering exposes
Running in parallel changes which test precedes which, and that surfaces state one test leaves behind for the next. Doing the same change in Springfield turned up three such leaks, and the fixes come across in
bedrock/conftest.pyas autouse fixtures for everything underbedrock/:get_language(), while the locale prefix inside them comes fromLocalePrefixPattern, which reads the language throughlib.l10n_utils.translation. A test that mocksdjango.utils.translation.get_languagewhile a URL is reversed fills the cache entry for the real language with the mocked language's prefix, and every later test on that worker reverses wrong URLs. Serial runs never see it because an earlier test has always populated the cache first. Clearing costs under 9 ms per test.AcceptedLocalesTestused to move the realdata/www-l10ncheckout aside for the class's duration, which under xdist can leave another worker reading an empty Fluent repo. It now builds its stand-in locale tree in a temporary directory and patchesFLUENT_REPO_PATHto point at it.Integration tests
The integration runner moves from pytest-parallel, which has had no release since 2021 and hangs on macOS, to xdist as well.
PYTEST_PROCESSESmaps straight onto-nand accepts the same values the workflows already set (1,auto). pytest-parallel and its only dependant, tblib, leave the lock. Worth arun-integration-testspush before merge.Ported from mozmeao/springfield#2027.