Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing a read-only projection of the conserve archive for Windows #240

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
8dac50d
Initial implementation of archive mounting on Windows
WolverinDEV Dec 29, 2023
58f84a5
Adding a hunk metadata cache to speed up index entry querying
WolverinDEV Dec 30, 2023
0b7e53a
Caching the opened bands as well
WolverinDEV Dec 30, 2023
e88514b
Supporting file timestamps, mark files read only and improved project…
WolverinDEV Dec 30, 2023
16a37aa
Enabling the mount command by default for all Windows platforms
WolverinDEV Dec 30, 2023
6b7d7b6
Improving cache to speed up directory and file serving
WolverinDEV Dec 30, 2023
eb1e814
Indicating backup timestamp trough directory modify timestamps
WolverinDEV Dec 30, 2023
76535be
Bumped windows-projfs version to v0.1.6
WolverinDEV Dec 30, 2023
bc6cd1b
Mount implementation polishing
WolverinDEV Dec 31, 2023
e2d2f07
Moving unix specific test imports into the function itself to avoid u…
WolverinDEV Dec 31, 2023
eddd369
Bumped the minimal required rust version to 1.75
WolverinDEV Dec 31, 2023
1fd937b
Incorporate PR feedback
WolverinDEV Jan 1, 2024
195974a
Merge remote-tracking branch 'original/main' into windows-mount
WolverinDEV Jan 1, 2024
bc5e81f
Merged with upstream
WolverinDEV Aug 1, 2024
3266178
Adding tests for the mount archive feature
WolverinDEV Aug 1, 2024
4b48c8c
Using Rust 1.76 as required by aws-sdk-sso
WolverinDEV Aug 1, 2024
57942e3
Disable tests for mount on non windows systems
WolverinDEV Aug 1, 2024
1918fa5
Seperating mount tests from normal tests any reduce the number of tes…
WolverinDEV Aug 10, 2024
ef27f3a
Skip some files from mutants
sourcefrog Oct 20, 2024
8bee9c1
Merge remote-tracking branch 'origin/main' into windows-mount
sourcefrog Oct 21, 2024
4a13602
fix: New Transport API
sourcefrog Oct 21, 2024
941a82c
fix: Better error messages
sourcefrog Oct 21, 2024
985ce15
Merge branch 'main' into pr/WolverinDEV/240
sourcefrog Oct 23, 2024
fb09fa0
Separate out no-op Unix mount
sourcefrog Oct 23, 2024
81ddb74
Merge branch 'main' into pr/WolverinDEV/240
sourcefrog Oct 23, 2024
8863d09
rustfmt
sourcefrog Oct 23, 2024
85d495c
clippy: some things only needed on Windows
sourcefrog Oct 23, 2024
f66d6b0
Only build hunk_index on Windows
sourcefrog Oct 23, 2024
210d790
Suppress unused warnings on Unix for code only used on Windows
sourcefrog Oct 23, 2024
c6a0e91
Rename option to --cleanup-projfs and add help
sourcefrog Oct 23, 2024
987edf3
merge main
sourcefrog Jan 2, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
matrix:
os: [ubuntu-latest, windows-latest, macOS-latest]
features: ["", "s3"]
version: [stable, nightly, "1.73"]
version: [stable, nightly, "1.75"]

steps:
- uses: actions/checkout@v3
Expand All @@ -43,8 +43,7 @@ jobs:
cargo --version
rustc --version
- name: Build
run:
cargo build --all-targets --no-default-features --features=${{
run: cargo build --all-targets --no-default-features --features=${{
matrix.features }}
- name: Test
run:
Expand Down Expand Up @@ -77,8 +76,7 @@ jobs:
# Can't use --all-features here because BLAKE2 SIMD needs unstable...
# Don't use the S3 features because they require AWS credentials for realistic
# testing.
run:
cargo mutants -j2 --no-shuffle -vV --cargo-arg --no-default-features
run: cargo mutants -j2 --no-shuffle -vV --cargo-arg --no-default-features
- name: Archive results
uses: actions/upload-artifact@v3
with:
Expand Down
126 changes: 113 additions & 13 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ name = "conserve"
readme = "README.md"
repository = "https://github.com/sourcefrog/conserve/"
version = "23.11.0"
rust-version = "1.73"
rust-version = "1.75"

[[bin]]
doc = false
Expand Down Expand Up @@ -64,6 +64,9 @@ indoc = "2.0"
uzers = "0.11"
nix = { version = "0.27", features = ["fs", "user"] }

[target.'cfg(windows)'.dependencies]
windows-projfs = { version = "0.1.6", features = ["dynamic-import"] }
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this pull in a lot of stuff? Maybe it should be optional even on Windows?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by "pull in a lot of stuff"?
Do you mean memory wise, dependency wise, file wise, etc?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant mostly a lot of new dependencies and therefore build time; hopefully it's not having any runtime effect unless it's used. Perhaps it's fine by default.


[dependencies.clap]
version = "4.3"
features = ["derive", "deprecated", "wrap_help"]
Expand Down
24 changes: 24 additions & 0 deletions src/bin/conserve.rs
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,21 @@ enum Command {
long_listing: bool,
},

/// Mount the archive as projection.
#[cfg(windows)]
Mount {
/// The archive to mount
archive: String,

/// Target folder where the archive should be mounted to
destination: PathBuf,

/// Create the target folder and remove all temporarly created
/// files on exit
sourcefrog marked this conversation as resolved.
Show resolved Hide resolved
#[arg(long, default_value_t = true)]
cleanup: bool,
},

/// Copy a stored tree to a restore directory.
Restore {
archive: String,
Expand Down Expand Up @@ -457,6 +472,15 @@ impl Command {
show::show_entry_names(entry_iter, &mut stdout, *long_listing)?;
}
}
#[cfg(windows)]
Command::Mount {
archive,
destination,
cleanup,
} => {
let archive = Archive::open(open_transport(archive)?)?;
mount(archive, destination, *cleanup)?;
}
Command::Restore {
archive,
destination,
Expand Down
7 changes: 7 additions & 0 deletions src/errors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,13 @@ pub enum Error {
#[from]
source: transport::Error,
},

#[cfg(windows)]
#[error(transparent)]
Projection {
#[from]
source: windows_projfs::Error,
},
}

impl From<jsonio::Error> for Error {
Expand Down
Loading
Loading