Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GPU] fix memory conflict for multi iteration in loop. #28056

Closed
wants to merge 13 commits into from
Closed

[GPU] fix memory conflict for multi iteration in loop. #28056

Changes from 6 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
317f8a6
copy memory when input shape change on single_shared
timxu826 Dec 13, 2024
85a95ce
prune redundent codes
timxu826 Dec 13, 2024
1cc9af1
Merge branch 'master' into cvs_158017
timxu826 Dec 31, 2024
d83c8b6
Merge branch 'master' into cvs_158017
timxu826 Jan 9, 2025
164ebea
changed the dynamic input from single_shared to single
timxu826 Jan 15, 2025
071cdfa
Merge remote-tracking branch 'refs/remotes/origin/cvs_158017' into cv…
timxu826 Jan 15, 2025
4244dcb
fix loop shape change
timxu826 Jan 16, 2025
a9a6d04
Revert "changed the dynamic input from single_shared to single"
timxu826 Jan 16, 2025
6dc130d
Revert "Merge remote-tracking branch 'refs/remotes/origin/cvs_158017'…
timxu826 Jan 16, 2025
ef7fe71
Revert "changed the dynamic input from single_shared to single"
timxu826 Jan 16, 2025
0522d88
Revert "prune redundent codes"
timxu826 Jan 16, 2025
cf70410
Merge branch 'cvs_158017_c' into cvs_158017
timxu826 Jan 16, 2025
9b88f45
revert "copy memory when input shape change on single_shared"
timxu826 Jan 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions src/plugins/intel_gpu/src/graph/loop.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -556,7 +556,8 @@ void loop_inst::preprocess_backedge_memory() {
<< backedged_sliced_output << "), initial_mem(" << initial_mem << ")" << std::endl;
// Set backedge mode to SINGLE when backedge_from_prim has multiple users.
} else if ((output_mapping.empty() && backedge_to_prim.get() == backedge_from_prim->dependencies().front().first)
|| (backedge_to_prim->get_users().size() > 1) ) {
|| (backedge_to_prim->get_users().size() > 1)
|| (backedge_from_prim->is_dynamic() && !backedge_to_prim->is_dynamic())) {
// SINGLE mode, from and to primitives in backedge are connected directly
backedge_memory_mappings.emplace_back(
backedge_from_prim, backedge_to_prim, initial_mem, body_network->get_stream());
Expand Down Expand Up @@ -1008,6 +1009,12 @@ void loop_inst::set_memory_in_body_network(cldnn::network::ptr body_network,
"impl_params layout size(", impl_layout.to_short_string(),
") should not exceed memory size(", updated_mem->get_layout().to_short_string(), ")");
// Set need_to_check_memory_to_set to false to set output memory even if the input node has static shape,
if (impl_layout.bytes_count() < updated_mem->get_layout().bytes_count()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this check condition creates redundant memory copies in cases where there is no memory conflict.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@timxu826 From the description, you seem to resolve memory conflicts issue?
If so, is this enough?
Shouldn't we prevent memory reuse for the problematic case?

auto& inst_engine = body_network->get_engine();
auto new_mem = inst_engine.allocate_memory(updated_mem->get_layout(), updated_mem->get_allocation_type(), true);
new_mem->copy_from(body_network->get_stream(), *updated_mem);
updated_mem = new_mem;
}
body_network->set_input_data(inst->id(), updated_mem, false);
// Update impl_params.output_layouts[0] to updated_mem's layout
inst->update_shape();
Expand Down Expand Up @@ -1083,6 +1090,7 @@ std::vector<event::ptr> loop_inst::handle_buffers_for_next_iteration(const loop_
to_mem = body_network->get_engine().allocate_memory(mapping.initial_mem->get_layout(), false);

body_network->set_input_data(to_id, to_mem);
mapping.to_primitive->update_shape();
ev = to_mem->copy_from(body_network->get_stream(), *(mapping.initial_mem));
GPU_DEBUG_LOG << iter << ") [SINGLE] Backedge_to node(" << to_id << ") is set to new memory("
<< to_mem << ", " << to_mem->get_layout().to_short_string()
Expand All @@ -1108,7 +1116,8 @@ std::vector<event::ptr> loop_inst::handle_buffers_for_next_iteration(const loop_
<< to_mem << ", " << to_mem->get_layout().to_short_string()
<< ") because of shape update from backedge_from()" << from_id
<<")'s memory(" << from_mem << "," << from_mem->get_layout().to_short_string() << ")" << std::endl;
body_network->set_input_data(to_id, to_mem);
body_network->set_input_data(to_id, to_mem, false);
mapping.to_primitive->update_shape();
ev = to_mem->copy_from(body_network->get_stream(), *(from_mem));
} else {
ev = to_mem->copy_from(body_network->get_stream(), *(from_mem));
Expand Down
Loading