raft_serverpb: add the extra trait use_plain_file for SnapshotMeta
#1260
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
| @@ -31,7 +30,6 @@ var _ status.Status | |||
| var _ = runtime.String | |||
| var _ = utilities.NewDoubleArray | |||
| var _ = descriptor.ForMessage | |||
| var _ = metadata.Join | |||
There was a problem hiding this comment.
It's cleaned by script automatically.
| @@ -59,6 +59,9 @@ message SnapshotMeta { | |||
| // It should only be used for v2 to send snapshot to v1. | |||
| // See https://github.com/pingcap/tiflash/issues/7568 | |||
| uint64 commit_index_hint = 7; | |||
| // Flag reflects whether the snapshot is persisted with plain files | |||
| // or sst files. `true` means using plain files. | |||
| bool use_plain_file = 8; | |||
There was a problem hiding this comment.
How about adding it to SnapshotCFFile? It's more flexible, eg. index regions have little data in the default CF but a large amount of data in the write CF.
There was a problem hiding this comment.
Reasonable but heavy on scanning. One concern is that we needs to manual scan to get the size of each cf, which will costs a lot on the IO and CPU when dumping the snapshot.
So, it's added into SnapshotMeta, and whether the region should use_plain_file == true is depends on the approximate size of the region is smaller than the given threshold.
|
To ensure the forward compatibility of TiKV, this change can be replaced with another implementation, by doing apply check on the |
Description
This PR is a pre-requisite for TiKV #https://github.com/tikv/tikv/pull/17408/files, used for adding a new trait
use_plain_filefor SnapshotMeta. And this trait is introduced for marking that the relative snapshot can be flushed withplain fileorsst files.