This repository has been archived by the owner on Oct 25, 2024. It is now read-only.

Gaudi Tensor split for memory optimization #4957

Job	Run time
call-inference-llama-2-7b-chat-hf / inference test	39m 30s
call-inference-mpt-7b-chat / inference test	8m 21s
	47m 51s

Provide feedback