[GPU] set output from variable's memory if kv-cache

[sungeunk]

Tickets:

• 157514

[isanghao]

I have some questions about this PR:

• This is setting _outputs[0] on realloc_if_needed. In case realloc does not happen, are we using proper buffer for _outputs[0]? I'm afraid this should be set before this function.
• Does shape_predictor work correctly for multi-ireq scenario? From the code, it seems like that the shape_predictor is placed within the network and it may have issue for multi-ireq scenario as it does not receive ireq information. We may need to take ireq information as an argument for that. Could you comment on it? @sshlyapn

[sshlyapn]

@isanghao I think there shouldn't be any problems with shape_predictor, since each ireq has unique instance of shape_predictor (which is passed to the network after the network is assigned to ireq):

openvino/src/plugins/intel_gpu/include/intel_gpu/plugin/sync_infer_request.hpp

Line 89 in 0f149e3

std::shared_ptr<cldnn::ShapePredictor> m_shape_predictor = nullptr;

[isanghao]

oh you already did the work ;) Thanks for confirmation.

[sshlyapn]

@sungeunk Could you please describe the issue in more detail? Does this mean that the KV-cache from one inference request is being used in another request in some cases? If so, then we probably need to adjust scales and zero points memory buffers as well

[sungeunk]

@sshlyapn Memory of variable-state in two infer-requests are shared at this sample code.minicpm_multi_inferreq.cpp
results of query_state() for llmIR are changed when it calls llmIR2.infer().

[sshlyapn]

@sungeunk , thank you for the details. In that case, it seems we have to update the scales/zero points buffers as well