Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: optimize delta log deletion #20158

Merged
merged 2 commits into from
Jan 20, 2025
Merged

refactor: optimize delta log deletion #20158

merged 2 commits into from
Jan 20, 2025

Conversation

zwang28
Copy link
Contributor

@zwang28 zwang28 commented Jan 14, 2025

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Since compatibility with the etcd meta store is no longer necessary, delta log deletion can now be performed with a single SQL query.

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • My PR contains critical fixes that are necessary to be merged into the latest release.

Documentation

  • My PR needs documentation updates.
Release note

@zwang28 zwang28 force-pushed the wangzheng/refactor branch from 3a28281 to 274fd27 Compare January 14, 2025 12:06
Copy link
Contributor

@Li0k Li0k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM, thanks for the PR

// If there is any safe point, skip this to ensure meta backup has required delta logs to
// replay version.
if !context_info.version_safe_points.is_empty() {
return Ok((0, deltas_to_delete_count));
return Ok(0);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An implicit problem may cause the request to time out if the number of bulk deletions is too large.

Optional Handling Means

  • Stop the service
  • Log in to pg and actively clean it up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just for the record, as we think its probability is low.

@zwang28 zwang28 added this pull request to the merge queue Jan 16, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 16, 2025
@zwang28 zwang28 added this pull request to the merge queue Jan 20, 2025
Merged via the queue into main with commit d00c6e3 Jan 20, 2025
31 of 32 checks passed
@zwang28 zwang28 deleted the wangzheng/refactor branch January 20, 2025 05:41
github-actions bot pushed a commit that referenced this pull request Jan 20, 2025
zwang28 added a commit that referenced this pull request Jan 20, 2025
github-merge-queue bot pushed a commit that referenced this pull request Jan 21, 2025
Co-authored-by: zwang28 <70626450+zwang28@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants