Support Multi-Node Inference and Serving.

[Jooho]

/kind feature

Describe the solution you'd like
As models continue to grow in size, it has become increasingly challenging to fit these large models into the memory of a single GPU. However, they can often be accommodated within the combined memory of multiple GPUs. Existing techniques such as tensor parallelism and pipeline parallelism allow for the division of models, enabling them to run parallel across multiple Nodes/GPUs, significantly enhancing performance.

Anything else you would like to add:

• New API - add a new API for multi-node/multi-gpu #3871
• POC - https://github.com/Jooho/jhouse_openshift/tree/main/Kserve/poc/multi-node
• Implementation - Multi-Node Inference Implementation #3972
• Documents - document for huggingface(vllm) servingruntime for multi-node website#402

Links to the design documents:

• Presentation ppt
• Design Doc