qwen2_5_vllm_fastapi

使用FastAPI+vLLM部署Qwen2.5

Install

git clone https://github.com/Xiaolong-RRL/qwen2_5_vllm_fastapi.git
cd qwen2_5_vllm_fastapi
pip install vllm, transformers, fastapi, uvicorn

python qwen2_5_vllm_fastapi_server.py

python qwen2_5_vllm_fastapi_client.py

python qwen2_5_vllm_fastapi_webui_gradio.py

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
LICENSE		LICENSE
README.md		README.md
qwen2_5.png		qwen2_5.png
qwen2_5_vllm_fastapi_client.py		qwen2_5_vllm_fastapi_client.py
qwen2_5_vllm_fastapi_server.py		qwen2_5_vllm_fastapi_server.py
qwen2_5_vllm_fastapi_utils.py		qwen2_5_vllm_fastapi_utils.py
qwen2_5_vllm_fastapi_webui_gradio.py		qwen2_5_vllm_fastapi_webui_gradio.py