Skip to content

Commit

Permalink
[GPU] Enable KV-cache compression by default for non-systolic platforms
Browse files Browse the repository at this point in the history
  • Loading branch information
sshlyapn committed Nov 5, 2024
1 parent 9f6826a commit 277a7ca
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions src/plugins/intel_gpu/src/runtime/execution_config.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,11 @@ void ExecutionConfig::apply_user_properties(const cldnn::device_info& info) {
set_property(ov::intel_gpu::queue_type(QueueTypes::in_order));
}

// Enable KV-cache compression by default for non-systolic platforms
if (!info.supports_immad) {
set_property(ov::hint::kv_cache_precision(ov::element::i8));
}

user_properties.clear();
}

Expand Down

0 comments on commit 277a7ca

Please sign in to comment.