Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete reuse space #1539

Draft
wants to merge 38 commits into
base: main
Choose a base branch
from
Draft

Delete reuse space #1539

wants to merge 38 commits into from

Conversation

lvca
Copy link
Contributor

@lvca lvca commented Apr 5, 2024

Pending PR to analyze differences after a long time working on this branch.

This PR is for #1391.

@lvca lvca self-assigned this Apr 5, 2024
@lvca lvca added the enhancement New feature or request label Apr 5, 2024
lvca added 5 commits April 4, 2024 23:02
The issue was with the missing deletion of the whole linked list segments. Only the head was deleted. This issue was found thanks to the CHECK DATABASE command.
Copy link

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
+0.04% 86.45%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (a1716e9) 68118 43409 63.73%
Head commit (c0ba4ec) 68259 (+141) 43524 (+115) 63.76% (+0.04%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#1539) 214 185 86.45%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

Codacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more

@lvca lvca requested a review from robfrank December 21, 2024 17:43
@lvca lvca marked this pull request as draft December 21, 2024 17:43
@lvca
Copy link
Contributor Author

lvca commented Dec 21, 2024

The main code is in LocalBucket.compressPage() which compress the page checking for holes created by record deletion. The compression is basically a shifting of the deleted record (hole) to the left. ArcadeDB page is separated in 2 parts: header and content. The header makes the RID and it contains the pointer to the actual record content. If the record is bigger than the space left in the page, then a multi-page record is created: basically the record is chunked and saved in next pages.

This PR is the first big step towards having 100% recyclable space. Once this is merged, there are still work to do to reach 100%. Here a brief summary of the TODO list:

  • compress the LinkedList used for edges. New edges are added at the beginning (because the most typical use case is browsing from the latest). We have to come up with a compaction policy there too
  • we need a way to reuse the space in the page instead of checking the space in the last page. At open time, we could easily have a quick scan of the first X pages of a bucket and save the information about how much space left is in each page. When the space is needed (new record, or update with a larger dimension), the deleted space will be reused based on convenience. For example, if a new record requires 5K of space, but the first 10 pages have less then 5K free, and the 11th page has 5K or more, the 11th page should be selected. This means the whole record can be on one page only

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant