Add musl and glibc bindings for getdents{,64} by cosmicexplorer · Pull Request #4522 · rust-lang/libc

cosmicexplorer · 2025-07-03T04:47:51Z

Description

The method posix_getdents() has recently been added in POSIX issue 8 (from 2024). This method offers a standard interface to retrieve multiple directory entries at once into a preallocated buffer. This typically translates directly to a syscall using the existing SYS_getdents{,64} key.

However, as posix_getdents() is so new, it has not been universally implemented yet. musl added it immediately last year, but glibc does not yet have it. Instead, glibc >= 2.30 has the method getdents64(), which is Linux-only.

While musl appears to implement posix_getdents() for all supported platforms, it appears that the version of musl rust pulls in is not from the local machine, but from its own bundled copy. This copy does not have posix_getdents(), but instead simply has getdents64() to match glibc on linux, as well as a getdents() macro for compatibility with the BSD standard.

The extant BSDs just name their version of this method getdents(), without any 64 extension, and expose this as a libc method call instead of a syscall.

Surprisingly, macOS does not support any form of the getdents*() method. I mentioned this in passing to an Apple engineer a few weeks ago.

I have been in contact with a glibc developer who plans to support this, but cannot make any timeline guarantees. Since it appears that the BSDs also have yet to add support for the POSIX standardized posix_getdents() too, it does not seem terribly pressing to block this PR until upstream Rust pulls in a more recent version of musl. Instead, a followup PR should be able to triumphantly add posix_getdents() to all the POSIX platforms at once! ^_^ :D

Sources

musl (posix_getdents()): https://github.com/bminor/musl/blob/86373b4999bfd9a9379bc4a3ca877b1c80a2a340/src/dirent/posix_getdents.c#L6
musl (getdents()): https://github.com/bminor/musl/blob/86373b4999bfd9a9379bc4a3ca877b1c80a2a340/src/linux/getdents.c#L6
musl (getdents64()): https://github.com/bminor/musl/blob/86373b4999bfd9a9379bc4a3ca877b1c80a2a340/include/dirent.h#L80
glibc (getdents64()): https://github.com/bminor/glibc/blob/7eed691cc2b6c5dbb6066ee1251606a744c7f05c/sysdeps/unix/sysv/linux/bits/dirent_ext.h#L29
freebsd (getdents()): https://man.freebsd.org/cgi/man.cgi?query=getdents&sektion=2&manpath=freebsd-release-ports
netbsd (getdents()): https://man.netbsd.org/getdents.2
openbsd (getdents()): https://man.openbsd.org/getdents.2
dragonflybsd (getdents()): https://man.dragonflybsd.org/?command=getdents&section=ANY

Checklist

Relevant tests in libc-test/semver have been updated
No placeholder or unstable values like *LAST or *MAX are
included (see #3131)
Tested locally (cd libc-test && cargo test --target mytarget);
especially relevant for platforms that may not be checked in CI

@rustbot label +stable-nominated

cosmicexplorer · 2025-07-03T04:59:11Z

I do not understand why freebsd tests would be failing if I have not modified any code or semver tests for freebsd.

It appears the tests are failing on methods I have not modified (https://cirrus-ci.com/task/5573619788546048):

warning: libc-test@0.1.0: /tmp/cirrus-ci-build/target/x86_64-unknown-freebsd/debug/build/libc-test-ec9ac07491fdefaa/out/main.c:33518:59: error: invalid application of 'sizeof' to an incomplete type 'struct xktls_session_onedir'

cosmicexplorer · 2025-07-03T05:20:43Z

Ok, now I'm getting a failure in a musl shard (aarch64-unknown-linux-musl) which I'm not sure about (https://github.com/rust-lang/libc/actions/runs/16041997001/job/45265389790?pr=4522):

warning: libc-test@0.1.0: /checkout/target/aarch64-unknown-linux-musl/debug/build/libc-test-c7eda42317ad320c/out/main.c: In function '__test_size_posix_dent':
warning: libc-test@0.1.0: /checkout/target/aarch64-unknown-linux-musl/debug/build/libc-test-c7eda42317ad320c/out/main.c:39376:56: error: invalid application of 'sizeof' to incomplete type 'struct posix_dent'
warning: libc-test@0.1.0: 39376 |  uint64_t __test_size_posix_dent(void) { return sizeof(struct posix_dent); }

The posix_dent struct is definitely defined in musl (https://github.com/bminor/musl/blob/86373b4999bfd9a9379bc4a3ca877b1c80a2a340/include/dirent.h#L21-L27). I did add posix_dent to libc-test/semver/linux-musl.txt -- I'm not sure if I'm supposed to do that or not for struct definitions. I was not able to get the local musl testing working on my machine due to a failure in build.rs for libc-test (see OP), so I'm having difficulty debugging this on my own.

Any pointers on how one would typically debug this would be appreciated. I know the header linux/can/j1939.h exists within /usr/include on my machine, but the build.rs is not picking it up, causing a failure to build main.c when running cargo test within the libc-test subdir.

cosmicexplorer · 2025-07-03T10:43:33Z

Ok, after testing this locally against a separate project (since the libc-test crate build script continues to be broken and unable to find headers), I found that posix_getdents() was not found in the musl rust was linking against for some reason, even though my local musl definitely has posix_getdents(). I'm assuming rust is pulling in an earlier version of musl that's not the one on my machine, so I simply removed it for now. That means this only adds getdents64() on glibc for linux, and getdents{,64}() on musl for linux. I'll update the OP to match this.

cosmicexplorer · 2025-07-03T14:17:21Z

Ok, it looks like the freebsd shards are failing on main too, so I'm going to assume that's not because of my changes.

tgross35 · 2025-07-28T08:49:22Z

Sorry this took a while to get to. Could you explain why this is needed? The manpage says:

This is not the function you are interested in. Look at readdir(3) for the POSIX conforming C library interface. This page documents the bare kernel system call interface.

Which sounds deprecated

tgross35 · 2025-08-27T05:16:16Z

For the followup needed,
@rustbot author

rustbot · 2025-08-27T05:16:20Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

rustbot · 2025-09-13T21:05:14Z

Some changes occurred in OpenBSD module

cc @semarie

rustbot · 2025-09-13T21:37:40Z

Some changes occurred in solarish module

cc @jclulow, @pfmooney

Some changes occurred in the Android module

cc @maurer

Some changes occurred in OpenBSD module

cc @semarie

tgross35 · 2025-07-08T05:43:16Z

+extern "C" {
+    pub fn getdents(fd: c_int, buf: *mut crate::dirent, len: usize) -> c_int;
+}


Can you combine this with the extern "C" block above?

tgross35 · 2025-07-08T05:44:39Z

+extern "C" {
+    // There is no glibc wrapper for the getdents system call, and getdents64() is only available
+    // from 2.30 onwards.
+    /// `buffer` points to a buffer of [`crate::dirent64`] structs.
+    pub fn getdents64(fd: c_int, buffer: *mut c_void, nbytes: usize) -> isize;
+}


Could you merge this with the extern "C" block above?

Also no doc comments please, for this crate we defer to the platform manpages.

tgross35 · 2025-10-29T08:52:14Z

@@ -0,0 +1 @@
+getdents64


Since the file doesn't exist yet and Hurd is T3, it's not being checked. So this can just be dropped

tgross35 · 2025-10-29T09:48:05Z

+
+    pub fn getdents(fd: c_int, buf: *mut c_char, nbytes: usize) -> c_int;


This one needs a link_name https://github.com/NetBSD/src/blob/bcc4c5ab138642408f293cdfc6eb84dbd7b7b1fb/include/dirent.h#L122

tgross35 · 2025-10-29T09:50:40Z

+extern "C" {
+    pub fn getdents(fd: c_int, buf: *mut c_char, nbytes: usize) -> isize;
+}


Can this move into an existing extern "C" block?

tgross35 · 2025-12-02T23:32:29Z

@cosmicexplorer gentle ping on this, it only needs a rebase and the few small changes noted above. (No pressure of course)

cosmicexplorer · 2025-12-30T01:32:02Z

thank you @tgross35 for the ping. i was very stressed out and i really appreciate your attention here and the opportunity to get this in.

rustbot · 2026-03-14T00:51:15Z

☔ The latest upstream changes (possibly #5011) made this pull request unmergeable. Please resolve the merge conflicts.

tgross35 · 2026-06-25T04:12:32Z

Musl 1.2.6 added posix_getdents, with #5215 we'll be able to test it in CI.

rustbot added O-gnu O-linux O-musl O-unix S-waiting-on-review stable-nominated This PR should be considered for cherry-pick to libc's stable release branch labels Jul 3, 2025

cosmicexplorer force-pushed the posix-getdents branch 2 times, most recently from d6e29f6 to 5d4300a Compare July 3, 2025 04:51

cosmicexplorer force-pushed the posix-getdents branch from a5a28a1 to f16f1da Compare July 3, 2025 05:03

rustbot added S-waiting-on-author and removed S-waiting-on-review labels Aug 27, 2025

rustbot added the O-dragonfly label Sep 13, 2025

cosmicexplorer force-pushed the posix-getdents branch 2 times, most recently from 3ffae61 to 0ea4693 Compare September 13, 2025 21:37

rustbot added A-CI Area: CI-related items ctest Issues relating to the ctest crate O-android O-arm O-bsd O-linux-like O-macos labels Sep 13, 2025

tgross35 requested changes Oct 29, 2025

View reviewed changes

tgross35 changed the title ~~add musl and glibc wrappers for getdents{,64}~~ Add musl and glibc binings for getdents{,64} Oct 29, 2025

tgross35 changed the title ~~Add musl and glibc binings for getdents{,64}~~ Add musl and glibc bindings for getdents{,64} Oct 30, 2025

tgross35 added S-waiting-on-author and removed O-arm O-android O-dragonfly O-gnu O-windows O-unix O-macos O-mips O-sparc O-solarish O-redox O-riscv O-linux-like O-wasi O-newlib O-x86 O-powerpc S-waiting-on-review labels Nov 2, 2025

cosmicexplorer mentioned this pull request Nov 27, 2025

ignore: gracefully quit worker threads upon panic in ParallelVisitor BurntSushi/ripgrep#3010

Closed

cosmicexplorer mentioned this pull request Dec 4, 2025

Tracking issue for RFC 2033: Experimentally add coroutines to Rust rust-lang/rust#43122

Open

cosmicexplorer mentioned this pull request Dec 27, 2025

library: update comment around close() rust-lang/rust#133330

Merged

cosmicexplorer mentioned this pull request Jan 16, 2026

ignore: parallel walker occasionally hangs if visitor panics BurntSushi/ripgrep#3009

Open

1 task


		pub fn getdents(fd: c_int, buf: *mut c_char, nbytes: usize) -> c_int;

		@@ -0,0 +1 @@
		getdents64

Uh oh!

Conversation

cosmicexplorer commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Sources

Checklist

Uh oh!

cosmicexplorer commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cosmicexplorer commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cosmicexplorer commented Jul 3, 2025

Uh oh!

cosmicexplorer commented Jul 3, 2025

Uh oh!

tgross35 commented Jul 28, 2025

Uh oh!

tgross35 commented Aug 27, 2025

Uh oh!

rustbot commented Aug 27, 2025

Uh oh!

rustbot commented Sep 13, 2025

Uh oh!

rustbot commented Sep 13, 2025

Uh oh!

tgross35 Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

tgross35 Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

tgross35 Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

tgross35 Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

tgross35 Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

tgross35 commented Dec 2, 2025

Uh oh!

cosmicexplorer commented Dec 30, 2025

Uh oh!

rustbot commented Mar 14, 2026

Uh oh!

tgross35 commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cosmicexplorer commented Jul 3, 2025 •

edited

Loading

cosmicexplorer commented Jul 3, 2025 •

edited

Loading

cosmicexplorer commented Jul 3, 2025 •

edited

Loading