Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Exclude utilization checks for unallocated resources #1820

Merged
merged 3 commits into from
Jan 11, 2024

Conversation

fregataa
Copy link
Member

@fregataa fregataa commented Jan 4, 2024

resolves https://github.com/lablup/giftbox/issues/592

When sessions(or kernels) are created without allocating cuda or any other devices, the value of kernel.occupied_slots field would be like below.

{'cpu': Decimal('1'), 'mem': Decimal('335544320')}

If we set thresholds of resources which are not in kernel.occupied_slots field to util checker, the checker determines the utilization with default 0 value resource. Example values below.

# utilization of a kernel
{'cpu': Decimal('1'), 'mem': Decimal('335544320'), 'cuda': 0}

# thresholds
{'cpu': Decimal('1'), 'mem': Decimal('10'), 'cuda': 2}

In this PR, util checker does not determine utilization of resources that is not in kernel.occupied_slots value.

Checklist: (if applicable)

  • Milestone metadata specifying the target backport version
  • Mention to the original issue
  • Installer updates including:
    • Fixtures for db schema changes
    • New mandatory config options
  • Test case(s) to:
    • Demonstrate the difference of before/after
    • Demonstrate the flow of abstract/conceptual models with a concrete implementation
  • Documentation
    • Contents in the docs directory
    • docstrings in public interfaces and type annotations

@fregataa fregataa added this to the 23.09 milestone Jan 4, 2024
@fregataa fregataa requested a review from adrysn January 4, 2024 07:19
@fregataa fregataa self-assigned this Jan 4, 2024
@github-actions github-actions bot added comp:manager Related to Manager component size:S 10~30 LoC labels Jan 4, 2024
@achimnol achimnol added urgency:blocker IT SHOULD BE RESOLVED BEFORE NEXT RELEASE! type:bug Reports about that are not working labels Jan 4, 2024
@fregataa fregataa added urgency:3 Must be finished within a certain time frame. and removed urgency:blocker IT SHOULD BE RESOLVED BEFORE NEXT RELEASE! labels Jan 9, 2024
@kyujin-cho kyujin-cho added this pull request to the merge queue Jan 11, 2024
Merged via the queue into main with commit 9e88820 Jan 11, 2024
43 checks passed
@kyujin-cho kyujin-cho deleted the fix/idle-checker-exclude-unalloc-res branch January 11, 2024 09:59
fregataa added a commit that referenced this pull request Jan 11, 2024
Backported-from: main
Backported-to: 23.09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:manager Related to Manager component size:S 10~30 LoC type:bug Reports about that are not working urgency:3 Must be finished within a certain time frame.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants