Allow serving llama models with tensor parallel#592

Draft

Jackmin801 wants to merge 1 commit intobigscience-workshop:mainfrom Jackmin801:llama-tp

+106-1

Commits on Jul 20, 2024

hangs in line 320 of slicer_wrapper.py of tensor_parallel package doing nccl ops
Jackmin801
committed