Skip to content

[TEMP] Run only the nightly arm valgrind cell for iteration - #29

Closed
aaronj0 wants to merge 2 commits into
mainfrom
nightly-arm-vg-repro
Closed

[TEMP] Run only the nightly arm valgrind cell for iteration#29
aaronj0 wants to merge 2 commits into
mainfrom
nightly-arm-vg-repro

Conversation

@aaronj0

@aaronj0 aaronj0 commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Attempt to fix the nightlies. The only remaining ARM vg nightlies fail because of a bug in valgrind:

It is deterministic: the same "Unrecognised instruction at 0x35b8702c" on every nightly run. The instruction is 0x38BFC108 = LDAPRB, an ARMv8.3 FEAT_LRCPC load-acquire byte, i.e. an atomic acquire load. The JIT emits it because the arm runner's CPU supports LRCPC, but Ubuntu 24.04's valgrind 3.22.0 can't decode it, so valgrind raises SIGILL and kills pytest mid-suite. Memcheck itself reports 0 errors. The last failure was test_concurrent.py addressed in #19

Bug: https://bugs.kde.org/show_bug.cgi?id=476465
Fix: https://sourceware.org/git/?p=valgrind.git;a=commit;h=41e2f95cf129191555e0048ccbbb392ee0fb155e

But as mentioned that release that has the fix is not published for the ubuntu version in the CI.

@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown

Test Results

Configuration Result
ubuntu-24.04-arm-llvm22-py3.13-cxx20-vg ====== 537 passed, 25 skipped, 35 xfailed, 1 xpassed in 86.08s (0:01:26) =======
ubuntu-24.04-llvm21-py3.14-cxx20-vg ====== 553 passed, 25 skipped, 18 xfailed, 2 xpassed in 92.48s (0:01:32) =======

@mcbarton

Copy link
Copy Markdown

@aaronj0 @vgvassilev if you need a newer version of Valgrind than Ubuntu 24.04 offers due to this bug, then you can get the latest version via conda/mamba like xeus-cpp does https://github.com/compiler-research/xeus-cpp/blob/82f9064929138f60995bfa12d0b82a382ff6a1b0/.github/workflows/main.yml#L194

@aaronj0

aaronj0 commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

@aaronj0 @vgvassilev if you need a newer version of Valgrind than Ubuntu 24.04 offers due to this bug, then you can get the latest version via conda/mamba like xeus-cpp does https://github.com/compiler-research/xeus-cpp/blob/82f9064929138f60995bfa12d0b82a382ff6a1b0/.github/workflows/main.yml#L194

Oh nice that's pretty useful, thanks.

Comment thread test/conftest.py Outdated
@aaronj0
aaronj0 force-pushed the nightly-arm-vg-repro branch from ec18e8d to 1ff5c83 Compare August 19, 2026 19:42
@aaronj0

aaronj0 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator Author

This PR temporarily picks up the workflow built by compiler-research/ci-workflows#101 which proves ti fix it. We can now run cppjit tests without suppressing on on vg + arm. Thanks for the conda suggestion @mcbarton

@aaronj0 aaronj0 closed this Aug 20, 2026
vgvassilev pushed a commit to compiler-research/ci-workflows that referenced this pull request Aug 20, 2026
ARM valgrind nightly failures in cppjit were caused by a bug in valgrind. They are deterministic: the same "Unrecognised instruction at 0x35b8702c" on every nightly run. The instruction is 0x38BFC108 = LDAPRB, an ARMv8.3 FEAT_LRCPC load-acquire byte, i.e. an atomic acquire load. The JIT emits it because the arm runner's CPU supports LRCPC, but Ubuntu 24.04's valgrind 3.22.0 can't decode it, so valgrind raises SIGILL and kills pytest mid-suite. Memcheck itself reports 0 errors.

Bug: https://bugs.kde.org/show_bug.cgi?id=476465
Fix: https://sourceware.org/git/?p=valgrind.git;a=commit;h=41e2f95cf129191555e0048ccbbb392ee0fb155e

The valgrind release carrying the fix is not published for the Ubuntu version on the CI runners, so
this installs valgrind from conda-forge, which packages latest valgrind for both linux-64 and
linux-aarch64. I've verified this works on compiler-research/cppjit#29: both cells install valgrind 3.27.1 and pass on both architectures, with the previously-gated concurrency tests re-enabled on arm.
@aaronj0
aaronj0 deleted the nightly-arm-vg-repro branch August 20, 2026 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants