-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support metadata compaction #270
Comments
I am interested in taking this if no one has started working on it. |
Based on offline discussion with @Fokko, I will first focus on implementing the
The MergeAppend will become the default append method since BTW, it seems |
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible. |
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' |
Feature Request / Improvement
Add support for compaction. This rewrites the existing manifests into a single one, reducing the number of calls to the object store. This should follow the Java configuration keys:
commit.manifest-merge.enabled
: Controls whether to automatically merge manifests on writes.commit.manifest.min-count-to-merge
: Minimum number of manifests to accumulate before merging.commit.manifest.target-size-bytes
: Target size when merging manifest files.The text was updated successfully, but these errors were encountered: