Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Cannot trigger stale L0 compaction when other collections are busy #39187

Open
1 task done
XuanYang-cn opened this issue Jan 13, 2025 · 0 comments
Open
1 task done

[Bug]: Cannot trigger stale L0 compaction when other collections are busy #39187

XuanYang-cn opened this issue Jan 13, 2025 · 0 comments
Assignees
@XuanYang-cn
Labels
kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Milestone
2.4.21

Comments

@XuanYang-cn
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version:
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Triggering L0 compaction shouldn't be influenced by other collections

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response
@XuanYang-cn XuanYang-cn added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 13, 2025
@XuanYang-cn XuanYang-cn self-assigned this Jan 13, 2025
@XuanYang-cn XuanYang-cn added this to the 2.4.21 milestone Jan 13, 2025
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue Jan 13, 2025
@XuanYang-cn
fix: Record active collections for l0Policy
cdbe1a0 
By recording the active collection lists, The l0 compaction trigger
of view change and idle won't influence each other.

Also this pr replace the L0View cache with real L0 segments' change.
Save some memory and make L0 trigger more accurate.

See also: milvus-io#39187

Signed-off-by: yangxuan <[email protected]>
@XuanYang-cn XuanYang-cn mentioned this issue Jan 13, 2025
Merged
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue Jan 14, 2025
@XuanYang-cn
fix: Record active collections for l0Policy
960d3b0 
By recording the active collection lists, The l0 compaction trigger
of view change and idle won't influence each other.

Also this pr replace the L0View cache with real L0 segments' change.
Save some memory and make L0 trigger more accurate.

See also: milvus-io#39187

Signed-off-by: yangxuan <[email protected]>
sre-ci-robot pushed a commit that referenced this issue Jan 16, 2025
@XuanYang-cn
fix: Record active collections for l0Policy (#39217)
74b4369 
By recording the active collection lists, The l0 compaction trigger
of view change and idle won't influence each other.

Also this pr replaces the L0View cache with real L0 segments' change.
Save some memory and make L0 compaction triggers more accurate.

See also: #39187

Signed-off-by: yangxuan <[email protected]>

---------

Signed-off-by: yangxuan <[email protected]>
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue Jan 17, 2025
@XuanYang-cn
fix: [cp25]Record active collections for l0Policy (milvus-io#39217)
8e5e399 
By recording the active collection lists, The l0 compaction trigger
of view change and idle won't influence each other.

Also this pr replaces the L0View cache with real L0 segments' change.
Save some memory and make L0 compaction triggers more accurate.

See also: milvus-io#39187
pr: milvus-io#39217

Signed-off-by: yangxuan <[email protected]>

---------

Signed-off-by: yangxuan <[email protected]>
XuanYang-cn added a commit to XuanYang-cn/milvus that referenced this issue Jan 17, 2025
@XuanYang-cn
fix: [cp24]Record active collections for l0Policy (milvus-io#39217)
6da5eff 
By recording the active collection lists, The l0 compaction trigger
of view change and idle won't influence each other.

Also this pr replaces the L0View cache with real L0 segments' change.
Save some memory and make L0 compaction triggers more accurate.

See also: milvus-io#39187
pr: milvus-io#39217

Signed-off-by: yangxuan <[email protected]>

---------

Signed-off-by: yangxuan <[email protected]>
This was referenced Jan 17, 2025
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@XuanYang-cn XuanYang-cn

Labels
kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
None yet
Milestone
2.4.21
Development

No branches or pull requests
1 participant
@XuanYang-cn