feat: support directory references in <filepath@store> (fixes #1410)#1415
Open
dimitri-yatsenko wants to merge 2 commits intomasterfrom
Open
feat: support directory references in <filepath@store> (fixes #1410)#1415dimitri-yatsenko wants to merge 2 commits intomasterfrom
dimitri-yatsenko wants to merge 2 commits intomasterfrom
Conversation
Previously is_dir was hardcoded to False in FilepathCodec.encode(), and StorageBackend.exists() used Path.is_file() which returned False for directories. Together these caused directory paths to fail the existence check and never set is_dir correctly. Changes: - storage.py: StorageBackend.exists() now uses Path.exists() so directories pass the check; add isdir() method for both local and remote (fsspec) backends. - filepath.py: encode() calls backend.isdir() to detect directories dynamically; size is set to None for directories. - objectref.py: _verify_folder() returns True (unverified-but-valid) when no manifest is present, rather than raising IntegrityError. Directories stored without a manifest are accepted. Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Two tests that mock the storage backend were not setting mock_backend.isdir.return_value = False. The new encode() logic calls backend.isdir() before backend.size(), so without the mock isdir() returned a truthy MagicMock, making is_dir=True and skipping the size() call entirely (leaving size=None). Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1410.
<filepath@store>only supported individual files.is_dirwas hardcoded toFalseandStorageBackend.exists()usedPath.is_file(), so directory paths always failed the existence check.Four targeted changes:
storage.py—StorageBackend.exists()now usesPath.exists()(accepts both files and dirs); newisdir()method added for local and fsspec backends.filepath.py—FilepathCodec.encode()callsbackend.isdir(path)to detect directories dynamically;sizeisNonefor directories.objectref.py—_verify_folder()returnsTrue(unverified-but-valid) when no manifest file is present, instead of raisingIntegrityError. Directories stored without a manifest are accepted gracefully.Test plan
<filepath@store>column pointing to a directory; confirmis_dir=Trueis storedObjectRefis returned withis_dir=Truepixi run test -k filepath🤖 Generated with Claude Code