Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: [json-inverted] Failed to query with filter 'exists json_1["id"]' #39017

Open
1 task done
ThreadDao opened this issue Jan 6, 2025 · 1 comment
Open
1 task done
Assignees
Labels
kind/bug Issues or changes related a bug severity/critical Critical, lead to crash, data missing, wrong result, function totally doesn't work. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Milestone

Comments

@ThreadDao
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: JsDove-optimization_json-42d5114-20250106
- Deployment mode(standalone or cluster): cluster
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

test

  1. create a collection with 3 fields: a pk int64 field + a vector field + a json field
  2. create HNSW index for vector field
  3. insert 10m entities, The data generation rules of the batch json column are as follows:
values = [{"id": i, "values": {"float": float(i), "varchar": str(i)}} for i in pks]
# example:
row_0: {"id": 0, "values": {"float": 0.0, "varchar": '0'}
row_1: {"id": 1, "values": {"float": 1.0, "varchar": '1'}
  1. flush and create index again -> load
  2. query with Strong consistency level: -> wrong query results
c.query('exists json_1["id"]', output_fields=["count(*)"])
2025-01-06 16:14:54,912 [ERROR][handler]: RPC error: [query], <MilvusException: (code=65535, message=fail to Query on QueryNode 14: worker(14) query failed: N6milvus21ExecOperatorExceptionE :Operator::GetOutput failed for [Operator:PhyFilterBitsNode, plan node id: 1054] : Assert "(segment_->type() == SegmentType::Sealed && num_data_chunk_ == 1)"  at /go/src/github.com/milvus-io/milvus/internal/core/src/exec/expression/ExistsExpr.cpp:105
)>, <Time:{'RPC start': '2025-01-06 16:14:54.286641', 'RPC error': '2025-01-06 16:14:54.912436'}> (decorators.py:140)
Traceback (most recent call last):
  File "/home/zong/Downloads/pycharm-community-2024.3.1/plugins/python-ce/helpers/pydev/pydevconsole.py", line 364, in runcode
    coro = func()
  File "<input>", line 1, in <module>
  File "/home/zong/zong/projects/milvus/tests/python_client/venv3/lib/python3.8/site-packages/pymilvus/orm/collection.py", line 1074, in query
  File "/home/zong/zong/projects/milvus/tests/python_client/venv3/lib/python3.8/site-packages/pymilvus/decorators.py", line 141, in handler
  File "/home/zong/zong/projects/milvus/tests/python_client/venv3/lib/python3.8/site-packages/pymilvus/decorators.py", line 137, in handler
  File "/home/zong/zong/projects/milvus/tests/python_client/venv3/lib/python3.8/site-packages/pymilvus/decorators.py", line 176, in handler
  File "/home/zong/zong/projects/milvus/tests/python_client/venv3/lib/python3.8/site-packages/pymilvus/decorators.py", line 116, in handler
  File "/home/zong/zong/projects/milvus/tests/python_client/venv3/lib/python3.8/site-packages/pymilvus/decorators.py", line 86, in handler
  File "/home/zong/zong/projects/milvus/tests/python_client/venv3/lib/python3.8/site-packages/pymilvus/client/grpc_handler.py", line 1603, in query
  File "/home/zong/zong/projects/milvus/tests/python_client/venv3/lib/python3.8/site-packages/pymilvus/client/utils.py", line 63, in check_status
pymilvus.exceptions.MilvusException: <MilvusException: (code=65535, message=fail to Query on QueryNode 14: worker(14) query failed: N6milvus21ExecOperatorExceptionE :Operator::GetOutput failed for [Operator:PhyFilterBitsNode, plan node id: 1054] : Assert "(segment_->type() == SegmentType::Sealed && num_data_chunk_ == 1)"  at /go/src/github.com/milvus-io/milvus/internal/core/src/exec/expression/ExistsExpr.cpp:105
)>

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

json-inverted-op-zong-milvus-datanode-f798d95cb-k5sbm             1/1     Running                  0               59m     10.104.13.81    4am-node16   <none>           <none>
json-inverted-op-zong-milvus-indexnode-7d644f6cbb-4b22g           1/1     Running                  0               64m     10.104.13.80    4am-node16   <none>           <none>
json-inverted-op-zong-milvus-indexnode-7d644f6cbb-6zpzm           1/1     Running                  0               63m     10.104.9.160    4am-node14   <none>           <none>
json-inverted-op-zong-milvus-indexnode-7d644f6cbb-75f25           1/1     Running                  0               63m     10.104.14.235   4am-node18   <none>           <none>
json-inverted-op-zong-milvus-mixcoord-7d8cbccb6b-rgz2f            1/1     Running                  0               62m     10.104.32.190   4am-node39   <none>           <none>
json-inverted-op-zong-milvus-proxy-7d84fd44b-brz44                1/1     Running                  0               59m     10.104.33.16    4am-node36   <none>           <none>
json-inverted-op-zong-milvus-querynode-1-7f54bf948f-k4ttg         1/1     Running                  0               61m     10.104.34.62    4am-node37   <none>           <none>

Anything else?

No response

@ThreadDao ThreadDao added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 6, 2025
@ThreadDao ThreadDao added the severity/critical Critical, lead to crash, data missing, wrong result, function totally doesn't work. label Jan 6, 2025
@ThreadDao ThreadDao added this to the 2.5.3 milestone Jan 6, 2025
@JsDove
Copy link
Contributor

JsDove commented Jan 6, 2025

It has been fixed already.The reason is that SealSegment supports splitting into multiple chunks. The solution is to remove the assert.

@yanliang567 yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 8, 2025
@yanliang567 yanliang567 removed their assignment Jan 8, 2025
@yanliang567 yanliang567 modified the milestones: 2.5.3, 2.5.4 Jan 16, 2025
@yanliang567 yanliang567 modified the milestones: 2.5.4, 2.5.5 Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug severity/critical Critical, lead to crash, data missing, wrong result, function totally doesn't work. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

No branches or pull requests

3 participants