Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CNDB-12407: lazily load token in PrimaryKeyWithSource #1500

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

michaeljmarshall
Copy link
Member

Also, add sstable overlap check to avoid certain
lookups.

What is the issue

This is a possible fix for https://github.com/riptano/cndb/issues/12407.

What does this PR fix and why was it fixed

This PR skips loading the PrimaryKey's token in the case where the sstables/memtables do not overlap.

It is implemented via a subtle change to the PrimaryKeyWithSource class that only loads the token info when the token is needed.

Checklist before you submit for review

  • Make sure there is a PR in the CNDB project updating the Converged Cassandra version
  • Use NoSpamLogger for log lines that may appear frequently in the logs
  • Verify test results on Butler
  • Test coverage for new/modified code is > 80%
  • Proper code formatting
  • Proper title for each commit staring with the project-issue number, like CNDB-1234
  • Each commit has a meaningful description
  • Each commit is not very long and contains related changes
  • Renames, moves and reformatting are in distinct commits

Also, add sstable overlap check to avoid certain
lookups.
@michaeljmarshall michaeljmarshall self-assigned this Jan 14, 2025
Copy link

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice !

we should add some unit tests to cover the new code

@michaeljmarshall
Copy link
Member Author

I pushed this up to see what tests fail, if any, and to see the code coverage. I'll modify once I get those reports back.

The bug here was caught by testHybridSearchHoleInClusteringColumnOrdering.
It failed because the PartitionKey-only primary key
said it was equal to one key but not another key
because of the order of operations in the PrimaryKeyListUtil
logic. One solution could have been to update the
way that findBoundaryIndex works, but I think a more
robust solution is to just load a fully qualified
min/max PrK since those are used for boundary checks
anyway.
Now that we have fully qualified PrK
objects as boundaries, we can remove
the check for the entry at the next
index.
The min/max partition key is not always a pointer
to the segment metadata's min/max sstable row id.
I thought it was. Removing that assertion.
@cassci-bot
Copy link

❌ Build ds-cassandra-pr-gate/PR-1500 rejected by Butler


22 new test failure(s) in 8 builds
See build details here


Found 22 new test failures

Showing only first 15 new test failures

Test Explanation Branch history Upstream history
...WriteLifecycle[CompoundKeyWithStaticsDataModel] regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
...mpoundKeyWithStaticsTest.testWriteLifecycle[aa] regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
...taset=bigint,wide=true,scenario=MEMTABLE_QUERY] regression 🔴🔵🔵🔵🔵🔵🔵 🔵🔵🔵🔵🔵🔵🔵
....LegacyOnDiskFormatTest.canReadPerIndexMetadata regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
....s.d.v.LegacyOnDiskFormatTest.canSearchBDKIndex regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
....d.v.LegacyOnDiskFormatTest.canSearchTermsIndex regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
...d.v.SegmentFlushTest.testFlushBetweenRowIds[ca] regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
...d.v.SegmentFlushTest.testFlushBetweenRowIds[db] regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
...d.v.SegmentFlushTest.testFlushBetweenRowIds[dc] regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
...d.v.SegmentFlushTest.testFlushBetweenRowIds[eb] regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
...v.SegmentFlushTest.testNoFlushBetweenRowIds[aa] regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
...v.SegmentFlushTest.testNoFlushBetweenRowIds[eb] regression 🔴🔴🔴🔴🔴🔴🔵 🔵🔵🔵🔵🔵🔵🔵
...onIteratorTest.testTwoIteratorsWithSingleValues regression 🔴🔴🔵🔵🔵🔵🔵 🔵🔵🔵🔵🔵🔵🔵
...yRangeUnionIteratorTest.testUnionOfIntersection regression 🔴🔴🔵🔵🔵🔵🔵 🔵🔵🔵🔵🔵🔵🔵
o.a.c.u.b.BinLogTest.testTruncationReleasesLogS... regression 🔴🔴🔵🔵🔵🔵🔵 🔵🔵🔵🔵🔵🔵🔵

Found 51 known test failures

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants