✨ Support async create VM #781
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
akutz
force-pushed
the
feature/async-create-vm
branch
3 times, most recently
from
67dbe35
to
c8e1590
Compare
github-actions
bot
added
size/XXL
akutz
force-pushed
the
feature/async-create-vm
branch
7 times, most recently
from
10cab32
to
4cf4f6d
Compare
akutz
force-pushed
the
feature/async-create-vm
branch
5 times, most recently
from
0cf97b1
to
8ac0447
Compare
This patch introduces an async variant on the provider's `CreateOrUpdateVirtualMachine` function. This new function, `CreateOrUpdateVirtualMachineAsync`, returns `<-chan error, error`, for when a non-blocking create is used. Please note, support for async create is only enabled when the async signal feature is enabled. This is because the new async create workflow no longer: - Falls into a post-create reconfigure, instead allowing async signal to enqueue a new reconcile when the create has completed. - Requeues the VM after N time based on the VM's state, ex. when the VM is powered on and does not yet have an IP address. This also now relies on async signal when the feature is enabled. While a non-blocking create does mean the reconciler threads are no longer consumed by create operations, it does not mean VM Op will allow unbounded, concurrent creates. Because each non-blocking create operation consumes a goroutine, the number of concurrent create operations is still limited. The limit is the same as the number of threads previously allowed to do create operations. The difference is, with async create disabled, if 16 threads are doing creates, that is 16 threads that cannot do anything else. With async create enabled, if 16 goroutines are doing create, there are still 16 reconciler threads available to do other things. This features requires the WorkloadDomainIsolation feature to be enabled and for the environment variable ASYNC_SIGNAL_DISABLED to be unset or set to a non-truth-y value. Even if those conditions are met, the environment variable ASYNC_CREATE_DISABLED may be set to a truth-y value in order to fall back to the previous behavior.
akutz
force-pushed
the
feature/async-create-vm
branch
from
8ac0447
to
52505c0
Compare
Minimum allowed line rate is |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do, and why is it needed?
This patch introduces an async variant on the provider's
CreateOrUpdateVirtualMachinefunction. This new function,CreateOrUpdateVirtualMachineAsync, returns<-chan error, error, for when a non-blocking create is used.Please note, support for async create is only enabled when the async signal feature is enabled. This is because the new async create workflow no longer:
While a non-blocking create does mean the reconciler threads are no longer consumed by create operations, it does not mean VM Op will allow unbounded, concurrent creates. Because each non-blocking create operation consumes a goroutine, the number of concurrent create operations is still limited. The limit is the same as the number of threads previously allowed to do create operations. The difference is, with async create disabled, if 16 threads are doing creates, that is 16 threads that cannot do anything else. With async create enabled, if 16 goroutines are doing create, there are still 16 reconciler threads available to do other things.
This features requires the WorkloadDomainIsolation feature to be enabled and for the environment variable
ASYNC_SIGNAL_DISABLEDto be unset or set to a non-truth-y value. Even if those conditions are met, the environment variableASYNC_CREATE_DISABLEDmay be set to a truth-y value in order to fall back to the previous behavior.Which issue(s) is/are addressed by this PR? (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes
NAAre there any special notes for your reviewer:
This is WiP while I:
Fix any lingering unit/integration test issuesReplace mention of "create threads" with "create goroutines"Validation this on a real testbedPer the log, this obviously has the issue of duplicate creates, although it must be stated the only result of this is an error in the task collector that the item already exists. Anyway, the next action item will remove the duplicates.
Prevent overlapping creates for the same VMThere is now a single async create op per VM:
Please add a release note if necessary: