Refactor file store

[nymius]

Description

The Store::open method doesn't recovers the previous Store state saved in the file and emplaces the file pointer just after the magic bytes prefix, this later is agravated by Store::append_changeset which sets the file pointer after the last written changeset. The combination of both causes the lost of any previous changeset there may have been in the file.

Is natural to think this shouldn't be the expected behavior, as @KnowWhoami pointed out in #1517, and the Store should recover the previous changesets stored in the file store.

The approach proposed in #1632 triggered a discussion about more changes in Store, which motivated the current refactor.

Besides the fix for #1517, the following methods have been changed/renamed/repurposed in Store:

• create: create file and retrieve store, if exists fail.
• load: load changesets from file, aggregate them and return aggregated changeset and Store. If there are problems with decoding, fail.
• dump: aggregate all changesets and return them.
• load_or_create: try load or fallback to create.

Fixes #1517.
Overrides #1632.

Notes to the reviewers

Load return type is Result<(Option<C>, Store), StoreErrorWithDump> to allow methods which doesn't use WalletPersister to get the aggregated changeset right away.

Dump is kept to allow WalletPersister::initialize method to retrieve the aggregated changeset without forcing the inclusion of the additional parameters needed by store::load to the trait, which would also affect the rusqlite implementation.

Changelog notice

Added:

• StoreError enum, which includes Io, Bincode and InvalidMagicBytes.
• "not intended for production" notice in general README for file store.

Changed:

• Store::create_new to Store::create, with new return type: Result<Self, StoreError>
• Store::open to Store::load, with new return type: Result<(Option<C>, Self), StoreErrorWithDump<C>>
• Store::open_or_create to Store::load_or_create, with new return type: Result<(Option<C>, Self), StoreErrorWithDump<C>>
• Store::aggregate_changesets to Store::dump, with new return type: Result<Option<C>, StoreErrorWithDump<C>>
• FileError to StoreError
• AggregateChangesetsError to StoreErrorWithDump, which now can include all the variants of StoreError in the error field.

Deleted:

• IterError deleted.

Checklists

All Submissions:

• I've signed all my commits
• I followed the contribution guidelines
• I ran cargo fmt and cargo clippy before committing

New Features:

• I've added tests for the new feature
• I've added docs for the new feature

Bugfixes:

• This pull request breaks the existing API
• I've added tests to reproduce the issue which are now passing
• I'm linking the issue being fixed by this PR

[ValuedMammal]

I did a first pass. Overall it looks good. I also made suggestions in this commit ValuedMammal/bdk@6312ca3

If you plan to rebase make sure commits compile on their own. For example changing a method name and updating the method call sites should be in the same commit in my opinion.

[nymius]

@ValuedMammal thanks for the review, I'm already working in the changes.

[nymius]

Rebased on 03a08bb
Haven't applied changes solicited in #1684 (comment) because now I find more valuable to report the error and allow the user to recover from it instead of returning whatever aggregated changeset we could parse in a best effort approach.

[ValuedMammal]

In 0f104cc:

error[E0432]: unresolved import `crate::FileError`
 --> crates/file_store/src/store.rs:1:41
  |
1 | use crate::{bincode_options, EntryIter, FileError, IterError};
  |                                         ^^^^^^^^^ no `FileError` in the root

For more information about this error, try `rustc --explain E0432`.
error: could not compile `bdk_file_store` (lib) due to 1 previous error

Maybe just squash in the next commit 8897a20 but keep the commit message from 8897a20 ?

[nymius]

In 0f104cc:

error[E0432]: unresolved import `crate::FileError`
 --> crates/file_store/src/store.rs:1:41
  |
1 | use crate::{bincode_options, EntryIter, FileError, IterError};
  |                                         ^^^^^^^^^ no `FileError` in the root

For more information about this error, try `rustc --explain E0432`.
error: could not compile `bdk_file_store` (lib) due to 1 previous error

Maybe just squash in the next commit 8897a20 but keep the commit message from 8897a20 ?

Thanks! should be fixed now. I checked each commit with:

 git rebase --exec 'cargo build; cargo test; cargo clippy; cargo fmt' 0f104c^

[ValuedMammal]

ACK 18aa75e I tested on example_esplora