backenddevelopment
serving-llms-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
maintainer
Orchestra-Research
更新于 11/20/2025
星标
6563
分支
515
quick start
Installation and usage
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
安装
$ install --globalskills.sh
使用
安装后，您可以通过在终端运行以下命令来使用此技能：
skills use serving-llms-vllm