Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vine: efficient resource allocation #4006

Closed
Changes from 1 commit
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
329f2c9
log the res allocation
JinZhou5042 Dec 10, 2024
95bf025
Merge branch 'cooperative-computing-lab:master' into resource_allocation
JinZhou5042 Dec 10, 2024
2ed9c0e
efficient resource allocation
JinZhou5042 Dec 11, 2024
5620893
lint
JinZhou5042 Dec 11, 2024
e4aeea0
test
JinZhou5042 Dec 11, 2024
aa85792
test
JinZhou5042 Dec 11, 2024
b6e17e0
remove test comment
JinZhou5042 Dec 11, 2024
7ac0d39
BYTES_TO_MEGABYTES w->inuse_cache
JinZhou5042 Dec 11, 2024
ac81d81
remove comment
JinZhou5042 Dec 11, 2024
62c7856
change variable name
JinZhou5042 Dec 11, 2024
fa663af
limit available
JinZhou5042 Dec 11, 2024
18e09dd
lint
JinZhou5042 Dec 11, 2024
3733253
count the global usable cores
JinZhou5042 Dec 12, 2024
0c2d22d
count slots and cores
JinZhou5042 Dec 12, 2024
6e8649d
rename
JinZhou5042 Dec 12, 2024
a3e286f
remove comments
JinZhou5042 Dec 12, 2024
e264f74
variable init
JinZhou5042 Dec 12, 2024
e207615
check worker cores
JinZhou5042 Dec 12, 2024
1f3cb2d
log resource allocation failures
JinZhou5042 Dec 12, 2024
9c9caa4
lint
JinZhou5042 Dec 12, 2024
47e27b0
don't use %ld
JinZhou5042 Dec 12, 2024
b95f32b
check mem and disk as well
JinZhou5042 Dec 12, 2024
1b5a23b
fix
JinZhou5042 Dec 12, 2024
1074d45
shortcut on time
JinZhou5042 Dec 13, 2024
b13309c
two proportional allocations
JinZhou5042 Dec 16, 2024
04f760d
update proportional
JinZhou5042 Dec 16, 2024
d1d1f2b
check input_files_size
JinZhou5042 Dec 16, 2024
91883fb
Merge branch 'cooperative-computing-lab:master' into resource_allocation
JinZhou5042 Dec 19, 2024
f81f3eb
use total instead of available
JinZhou5042 Dec 19, 2024
2ff15d4
add an api
JinZhou5042 Dec 19, 2024
b423522
divide disk by 2 on proportional
JinZhou5042 Dec 19, 2024
365de3c
lint
JinZhou5042 Dec 19, 2024
9ebb422
change func name
JinZhou5042 Dec 19, 2024
03ca4f6
fix check_worker_have_enough_disk_with_inputs
JinZhou5042 Dec 19, 2024
ccbca01
expose input compute api
JinZhou5042 Dec 19, 2024
53a125a
more apis
JinZhou5042 Dec 19, 2024
079a4e2
more apis
JinZhou5042 Dec 19, 2024
2d31bfa
update
JinZhou5042 Dec 20, 2024
e1460e3
combine the check inputs
JinZhou5042 Dec 20, 2024
e63a426
temp hack
JinZhou5042 Dec 20, 2024
232bbc2
check inputs
JinZhou5042 Dec 20, 2024
42981e1
do not count inputs when choose resource
JinZhou5042 Dec 21, 2024
d2e35f7
remove comment
JinZhou5042 Dec 21, 2024
f1f8b7d
lint
JinZhou5042 Dec 21, 2024
74cd93e
solve conflict
JinZhou5042 Jan 14, 2025
43e0e6f
check exec capacity
JinZhou5042 Jan 14, 2025
933f919
fix comment
JinZhou5042 Jan 14, 2025
5d98a7a
skip function tasks
JinZhou5042 Jan 14, 2025
89e2257
fix comments
JinZhou5042 Jan 14, 2025
5063baa
lint
JinZhou5042 Jan 14, 2025
18cfaac
add more comments
JinZhou5042 Jan 16, 2025
4e9b577
add disk_allocation_throttle_factor
JinZhou5042 Jan 17, 2025
5044b77
Merge branch 'master' into resource_allocation
JinZhou5042 Jan 17, 2025
098fc58
initiate disk_allocation_throttle_factor
JinZhou5042 Jan 17, 2025
8e83f5f
change name and test
JinZhou5042 Jan 17, 2025
e9d4f8b
lint
JinZhou5042 Jan 17, 2025
46aec03
name fix
JinZhou5042 Jan 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions taskvine/src/manager/vine_manager.c
Original file line number Diff line number Diff line change
Expand Up @@ -4002,6 +4002,7 @@ struct vine_manager *vine_ssl_create(int port, const char *key, const char *cert
q->large_task_check_interval = VINE_LARGE_TASK_CHECK_INTERVAL;
q->hungry_check_interval = VINE_HUNGRY_CHECK_INTERVAL;
q->option_blocklist_slow_workers_timeout = vine_option_blocklist_slow_workers_timeout;
q->disk_allocation_throttle_factor = 0.75;
JinZhou5042 marked this conversation as resolved.
Show resolved Hide resolved

q->manager_preferred_connection = xxstrdup("by_ip");

Expand Down
Loading