[GPU] Improvement of event-related primitives code #27337
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vladimir-paramuzov
force-pushed
the
ev_update
branch
2 times, most recently
from
e5a5b26 to
78ab0ce
Compare
vladimir-paramuzov
force-pushed
the
ev_update
branch
2 times, most recently
from
b6e9892 to
b7a66e1
Compare
Signed-off-by: Vladimir Paramuzov <[email protected]>
Signed-off-by: Vladimir Paramuzov <[email protected]>
Signed-off-by: Vladimir Paramuzov <[email protected]>
Signed-off-by: Vladimir Paramuzov <[email protected]>
vladimir-paramuzov
force-pushed
the
ev_update
branch
from
b7a66e1 to
43c0957
Compare
sshlyapn
reviewed
Nov 18, 2024
| @@ -36,7 +36,7 @@ struct network_output { | |||
| // TODO: in_order queue doesn't create proper output event in some cases which leads to syncronization issues with user app | |||
| // So call finish for associated stream to enusre that the output data is ready. | |||
| if (do_sync) { | |||
| if (_stream->get_queue_type() == QueueTypes::in_order) { | |||
| if (_stream->get_queue_type() == QueueTypes::in_order || !_event) { | |||
There was a problem hiding this comment.
When could event not be created for out-of-order queue?
If it's really possible, then this can add additional overhead in some cases, because currently some outputs can be processed asynchronously in SyncInferRequest::wait() function, however with this change it has to wait for all kernels to finish
There was a problem hiding this comment.
I think it happened when output primitive has CPU impl type which doesn't produce events anymore for barrier-based synchronization
There was a problem hiding this comment.
make_output_event seems to handle this case as well?
sshlyapn
approved these changes
Nov 18, 2024
ababushk
pushed a commit
to ababushk/openvino
that referenced
this pull request
Nov 18, 2024
…7337) ### Details: - Removed `_events` map from network class. Now dependency and result events are stored in `kernel_impl_params` for each primitive - User events are not created for CPU impls with barrier based synchronization to avoid useless OCL API calls (clCreateUserEvent -> clSetUserEventStatus -> clReleaseEvent). Overall, methods can return nullptr instead of user event. - Update ocl_stream::wait_for_events impl to deal with C event handles (cl_event) instead of C++ wrapper to avoid redundant clRetainEvent call - Introduced ExecutionFlags structure which reflects an execution status of primitive. Now methods which prepares dynamic primitive for execution modify/check some flags instead of some primitive_inst attributes or explicit function arguments. --------- Signed-off-by: Vladimir Paramuzov <[email protected]>
NishantPrabhuFujitsu
pushed a commit
to NishantPrabhuFujitsu/openvino
that referenced
this pull request
Nov 26, 2024
…7337) ### Details: - Removed `_events` map from network class. Now dependency and result events are stored in `kernel_impl_params` for each primitive - User events are not created for CPU impls with barrier based synchronization to avoid useless OCL API calls (clCreateUserEvent -> clSetUserEventStatus -> clReleaseEvent). Overall, methods can return nullptr instead of user event. - Update ocl_stream::wait_for_events impl to deal with C event handles (cl_event) instead of C++ wrapper to avoid redundant clRetainEvent call - Introduced ExecutionFlags structure which reflects an execution status of primitive. Now methods which prepares dynamic primitive for execution modify/check some flags instead of some primitive_inst attributes or explicit function arguments. --------- Signed-off-by: Vladimir Paramuzov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Details:
_eventsmap from network class. Now dependency and result events are stored inkernel_impl_paramsfor each primitive