Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: build text index when loading field data #39070

Merged
merged 5 commits into from
Jan 9, 2025

Conversation

SpadeA-Tang
Copy link
Contributor

fix: #39053
may fix #38644 which could be caused by #39053

@sre-ci-robot sre-ci-robot added the size/M Denotes a PR that changes 30-99 lines. label Jan 8, 2025
@mergify mergify bot added dco-passed DCO check passed. kind/bug Issues or changes related a bug labels Jan 8, 2025
@@ -274,6 +274,15 @@ SegmentGrowingImpl::LoadFieldData(const LoadFieldDataInfo& infos) {
storage::GetByteSizeOfFieldDatas(field_data));
}

// build text match index
if (field_meta.enable_match()) {
auto index = GetTextIndex(field_id);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can get index at this interface ? index not build before ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

index is created when at the constructor of SegmentGrowingImpl

Comment on lines 204 to 209
int64_t offset = 0;
for (const auto& data : field_datas) {
auto n = data->get_num_rows();
wrapper_->add_data(
static_cast<const std::string*>(data->Data()), n, offset);
offset += n;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to consider nullable ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch.

Copy link
Contributor

mergify bot commented Jan 8, 2025

@SpadeA-Tang cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

Signed-off-by: SpadeA-Tang <[email protected]>
@sre-ci-robot sre-ci-robot added size/L Denotes a PR that changes 100-499 lines. and removed size/M Denotes a PR that changes 30-99 lines. labels Jan 8, 2025
Signed-off-by: SpadeA-Tang <[email protected]>
Copy link
Contributor

mergify bot commented Jan 8, 2025

@SpadeA-Tang cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

Signed-off-by: SpadeA-Tang <[email protected]>
Copy link
Contributor

mergify bot commented Jan 8, 2025

@SpadeA-Tang E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link

codecov bot commented Jan 8, 2025

Codecov Report

Attention: Patch coverage is 80.76923% with 5 lines in your changes missing coverage. Please review.

Project coverage is 81.16%. Comparing base (47e7ea2) to head (4330bd7).
Report is 9 commits behind head on master.

Files with missing lines Patch % Lines
internal/core/src/index/TextMatchIndex.cpp 76.19% 5 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #39070      +/-   ##
==========================================
+ Coverage   81.14%   81.16%   +0.01%     
==========================================
  Files        1389     1389              
  Lines      196556   196624      +68     
==========================================
+ Hits       159491   159581      +90     
+ Misses      31473    31459      -14     
+ Partials     5592     5584       -8     
Components Coverage Δ
Client 79.53% <ø> (ø)
Core 69.64% <80.76%> (+0.10%) ⬆️
Go 83.10% <ø> (+<0.01%) ⬆️
Files with missing lines Coverage Δ
internal/core/src/segcore/SegmentGrowingImpl.cpp 82.36% <100.00%> (+2.10%) ⬆️
internal/core/src/index/TextMatchIndex.cpp 44.88% <76.19%> (+4.24%) ⬆️

... and 43 files with indirect coverage changes

@SpadeA-Tang
Copy link
Contributor Author

/run-cpu-e2e

@mergify mergify bot added the ci-passed label Jan 8, 2025
@SpadeA-Tang SpadeA-Tang requested a review from zhagnlu January 9, 2025 02:26
@czs007
Copy link
Collaborator

czs007 commented Jan 9, 2025

/approve
/lgtm

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: czs007, SpadeA-Tang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot merged commit 8abf6c9 into milvus-io:master Jan 9, 2025
20 checks passed
SpadeA-Tang added a commit to SpadeA-Tang/milvus that referenced this pull request Jan 9, 2025
sre-ci-robot pushed a commit that referenced this pull request Jan 9, 2025
fix: #39053 may fix
#38644 which could be caused
by #39053

---------

Signed-off-by: SpadeA-Tang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved ci-passed dco-passed DCO check passed. kind/bug Issues or changes related a bug lgtm size/L Denotes a PR that changes 100-499 lines.
Projects
None yet
4 participants