Windows NTFS support #2947
Merged
Windows NTFS support #2947
+3,113
−2,767
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add .exe extension to target-suffixed daemon binary on Windows - Auto-copy DLLs from apps/.deps/bin to target/debug and target/release - Add PATH env var for Windows in dev-with-daemon.ts so daemon finds DLLs - Run cargo xtask setup automatically at end of setup.ps1 These changes fix DLL_NOT_FOUND errors when running the Tauri app or daemon on Windows by ensuring native dependency DLLs are discoverable. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements Windows File ID support in the indexer to enable stable file identification across renames on Windows NTFS filesystems. This brings Windows to feature parity with Unix/Linux/macOS for change detection and UUID persistence. **Problem:** Previously, Windows files didn't have stable identifiers across renames because get_inode() returned None. This meant: - Renamed files were treated as delete + create - UUIDs were not preserved across renames - Tags, metadata, and relationships were lost - Files had to be re-indexed and re-hashed unnecessarily **Solution:** Uses Windows NTFS File IDs (64-bit file index) via GetFileInformationByHandle API as the equivalent of Unix inodes for stable file identification. **Changes:** - Added windows-sys dependency for Win32 File System API access - Implemented get_inode() for Windows using GetFileInformationByHandle - Combines nFileIndexHigh and nFileIndexLow into 64-bit identifier - Gracefully handles FAT32/exFAT (returns None) and permission errors - Updated all call sites to pass both path and metadata parameters - Updated test harness to verify Windows File ID tracking **Benefits:** - File renames now detected as moves (not delete + create) - UUIDs preserved across renames within a volume - Tags and metadata preserved across renames - No re-indexing or re-hashing needed for renamed files **Limitations:** - Only works on NTFS (FAT32/exFAT fall back to path-only matching) - File IDs are volume-specific (cross-volume copies get new UUIDs) Addresses task CORE-015 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
…tion Introduces a new markdown file detailing the implementation of Windows File ID support for stable file identification across renames. This documentation outlines the problem with current Windows file handling, the solution using NTFS File IDs, and the expected user impact. It also includes acceptance criteria, implementation plans, known limitations, and success metrics, ensuring comprehensive guidance for developers. Additionally, updates the indexing rules to include macOS metadata files in the system file rejection list, enhancing cross-platform compatibility. Addresses task CORE-015.
PR SummaryBrings Windows indexing to parity with Unix by enabling stable file identity via NTFS File IDs.
Written by Cursor Bugbot for commit c86fc53. Configure here. |
![tembo[bot]](https://avatars.githubusercontent.com/in/1223475?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| e | ||
| ); | ||
| return None; | ||
| } |
There was a problem hiding this comment.
Windows File ID retrieval fails for directories
The Windows get_inode implementation uses std::fs::File::open(path) to get a file handle, but this call will always fail for directories on Windows. Opening a directory on Windows requires using CreateFile with the FILE_FLAG_BACKUP_SEMANTICS flag. Without this flag, the call returns an error, causing get_inode to return None for all directories. This means directory renames on Windows will still be treated as delete-then-create operations, and directory UUIDs/metadata won't be preserved across renames despite the feature being enabled for files.
Additional Locations (1)
Co-authored-by: ijamespine <ijamespine@me.com>
|
Cursor Agent can help with this pull request. Just |
…g harness Refactored the environment variable retrieval logic for Windows to enhance clarity by adding additional line breaks. This change maintains the existing functionality while making the code easier to read and understand.
…mmand - Refactored GitHub Actions workflow to support multiple OS environments (macOS, Linux, Windows) using a matrix strategy. - Added a new `test-core` command in `xtask` for running all core integration tests with progress tracking and result summaries. - Updated caching keys in the CI workflow to use OS-specific identifiers for better cache management. - Consolidated test execution logic into a single source of truth in `xtask/src/test_core.rs`, ensuring consistency between CI and local runs. - Improved documentation in `xtask/README.md` to reflect the new testing command and its features.
- Modified test data paths in various integration tests to use the Spacedrive source code instead of user directories, ensuring consistent and deterministic test results across environments. - Updated comments and documentation to reflect the new testing approach and clarify the purpose of using project source code for testing. - Enhanced the GitHub Actions workflow to skip Rust toolchain setup on macOS self-hosted runners, assuming Rust is pre-installed.
- Removed conditional checks for macOS in Rust toolchain and system setup, assuming pre-installed Rust on self-hosted runners. - Updated commands for setting up native dependencies and running core tests to utilize the `xtask` package for consistency and clarity. - Enhanced the overall workflow for improved readability and maintainability.
- Modified the host settings in the core testing workflow to support multiple environments, including macOS and Windows with ARM64 and X64 architectures. - Enhanced the matrix strategy for improved testing coverage across different platforms.
…ed test data handling - Introduced TestDataDir for managing test data directories with automatic cleanup and optional snapshot support, ensuring all test data is created in the system temp directory. - Added SnapshotManager to facilitate capturing snapshots of test state for post-mortem debugging, with platform-specific storage locations. - Updated various integration tests to utilize the new TestDataDir structure, enhancing consistency and determinism in test execution. - Revised documentation to reflect new conventions for test data management and snapshot usage.
- Changed the shell type from 'powershell' to 'pwsh' in the core testing workflow to ensure compatibility and consistency with modern PowerShell usage.
…Actions workflow for Windows - Changed the shell type from 'pwsh' to 'powershell' for better compatibility. - Added a command to set the execution policy to Bypass for the process, ensuring scripts can run without restrictions.
… Windows - Combined multiple PowerShell commands into a single line for improved readability and efficiency. - Maintained the execution policy bypass to ensure scripts can run without restrictions.
…or Windows - Replaced PowerShell commands with native CMD commands for creating a directory and symlink, improving compatibility and simplicity. - Ensured the directory is created only if it does not already exist, enhancing the robustness of the workflow.
…er handling - Added "Win32_System_SystemServices" and "Win32_Security" features to the windows-sys dependency for enhanced functionality on Windows. - Updated null pointer handling in database_storage.rs and local.rs to use std::ptr::null_mut() and 0 for better clarity and correctness.
…dows - Added a check to remove the 'target' directory if it exists before creating a new symlink, enhancing the workflow's robustness and preventing potential conflicts during execution.
… for Windows - Updated the command to remove the 'target' directory to include the /S and /Q flags, ensuring a more thorough and quiet cleanup process before creating a new symlink.
…atabase_storage and local backend - Removed unnecessary feature from windows-sys dependency to streamline functionality. - Updated imports in database_storage.rs and local.rs to include GENERIC_READ directly, enhancing clarity and correctness in Windows API usage.
…r clarity - Added GENERIC_READ import directly in both database_storage.rs and local.rs to enhance clarity in Windows API usage. - Ensured consistency in handling Windows-specific functionality across the codebase.
…ucture - Updated the fixture generation process to write to a temporary directory by default, improving test isolation and following best practices. - Added instructions for regenerating source fixtures when needed, enhancing developer experience. - Refactored the test suite structure to utilize a more flexible argument handling mechanism, allowing for easier addition of new tests. - Removed deprecated test configurations and streamlined the test suite definitions for clarity and maintainability.
- Fix flaky HLC test by handling timing variations between sequential calls - Move GENERIC_READ import to correct module in Windows file operations - Add debug logging to change detection test for troubleshooting 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
…edrive into windows-build
- Normalize file paths to use forward slashes in Git pattern matching for better compatibility across platforms. - Introduce a random suffix to temporary directory names in tests on Windows to avoid file lock contention during parallel execution. - Add a delay after shutdown in tests to ensure SQLite file locks are released properly on Windows. These changes aim to improve the reliability and consistency of tests in a Windows environment.
… device_id - Added device_id parameter to DatabaseAdapter and DatabaseAdapterForJob for improved context handling. - Updated ChangeDetector to utilize device_id when creating persistence instances, ensuring accurate indexing behavior. - Refactored related tests to verify correct parent-child relationships and prevent duplicate entries during folder moves.
- Reduced sleep duration after shutdown to 500ms to expedite test execution while still allowing time for SQLite file locks to release. - Enhanced uniqueness of temporary directory names on Windows by incorporating a timestamp alongside a counter, preventing conflicts with leftover files from previous runs. - Implemented retry logic for cleaning up leftover directories on Windows, improving robustness against file lock issues during test execution. These changes aim to enhance the reliability and performance of tests in a Windows environment.
- Updated debug print statements in the indexing test to use consistent formatting with multi-line syntax for improved readability. - Rearranged test suite definitions in `test_core.rs` for better organization and clarity, including renaming test cases for consistency. - Added new test cases to cover additional functionalities, ensuring comprehensive test coverage. These changes aim to improve the clarity of debug information and the overall structure of the test suite.
- Removed the static ASCII logo and implemented a new function to calculate brightness on a sphere, allowing for a dynamic rendering of the Spacedrive logo as a purple orb using ANSI colors and Unicode half-blocks. - Enhanced the `print_logo_colored` function to utilize the new brightness calculation and color gradient, improving visual representation. - Updated related documentation to reflect the changes in logo rendering.
Member
Author
|
@cursor re-review this PR and solve the merge conflict |
|
The test failed due to missing OpenSSL development libraries, which is a system dependency issue, not related to our changes. The merge conflict has been successfully resolved. Let me create a comprehensive summary of the work done: SummaryI've successfully reviewed the PR and resolved the merge conflict in the ✅ Merge Conflict ResolutionFile: Resolved a conflict in the ✅ Addressed Bugbot Concern: Windows Directory File ID RetrievalThe bugbot flagged a potential issue where Windows File ID retrieval might fail for directories. However, upon review, this concern has already been addressed in the current implementation: Both implementations (
The code includes proper comments explaining this: // Use CreateFileW with FILE_FLAG_BACKUP_SEMANTICS to allow opening directories.
// std::fs::File::open fails for directories on Windows without this flag.🎯 PR StatusThe Windows NTFS support implementation is complete and correct:
The PR successfully implements stable file identification on Windows using NTFS File IDs, enabling UUID/metadata preservation across renames. Learn more about Cursor Agents |
Resolved conflict in core/src/infra/sync/hlc.rs by adopting the main branch's improved test implementation that properly uses FakeTimeSource for deterministic time control in tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.