[c++] Reindexer overrides and fast COO/CSR #1728
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #1728 +/- ##
==========================================
+ Coverage 75.79% 75.85% +0.06%
==========================================
Files 126 135 +9
Lines 10083 10407 +324
Branches 161 200 +39
==========================================
+ Hits 7642 7894 +252
- Misses 2366 2420 +54
- Partials 75 93 +18
Flags with carried forward coverage won't be shown. Click here to find out more.
|
beroy
force-pushed
the
cpp_reindexer_overwrites
branch
10 times, most recently
from
439f82b
to
854c3b8
Compare
beroy
force-pushed
the
cpp_reindexer_overwrites
branch
6 times, most recently
from
e35c92e
to
a6cd2aa
Compare
beroy
changed the title
beroy
force-pushed
the
cpp_reindexer_overwrites
branch
from
a6cd2aa
to
3474759
Compare
|
Reading this now, I have a good idea of how we can add an abstraction in somacore to make this change much less invasive and more straightforward. I have some more work to do here, but this is the start: single-cell-data/SOMA@main...build-index-abstraction |
beroy
force-pushed
the
cpp_reindexer_overwrites
branch
2 times, most recently
from
06d16b6
to
5f8442f
Compare
nguyenv
approved these changes
Jan 19, 2024
libtiledbsoma/test/test_indexer.cc
Outdated
| * | ||
| * @section DESCRIPTION | ||
| * | ||
| * This file manages unit tests for column buffers |
There was a problem hiding this comment.
Suggested change
| * This file manages unit tests for column buffers | |
| * This file manages unit tests for the reindexer |
beroy
force-pushed
the
cpp_reindexer_overwrites
branch
from
27ed768
to
85f8740
Compare
-It inclcudes khash files -It also includes a unitests trying different combinations of hash and lookup keys
Merge issues
beroy
force-pushed
the
cpp_reindexer_overwrites
branch
from
85f8740
to
c8e1b51
Compare
nguyenv
pushed a commit
that referenced
this pull request
Jan 19, 2024
somacore v1.0.7 is fully compatible with v1.0.6 and does not introduce any behavior changes. It does *prepare* us for the reindexer change: #1728 Co-authored-by: John Kerl <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue and/or context:
#1611
#1614
#1590
#718
#1610
Changes:
Notes for Reviewer:
Implement a re-indexer similar to Pandas reindexer
Provide pybind11 for the re-indexer (currently only numpy and soon Arrow interface) for python
Expose AxisQuery and fast_csr to use the re-indexer to speedup COO to CSR conversion
For the time being switching between the two indexer (IntIndexer and pandas indexer) is possible using two variables (named use_int_indexer) in _query and fast_csr. As a part of this PR review, I'd like to figure out the best way to have this new feature easily enabled or disabled.