home/categories/machine-learning/orchestra-research-ai-research-skills-12-inference-serving-sglang-skill-md
machine-learningdata-ai
sglang
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
maintainer
Orchestra-Research
更新于 11/20/2025
星标
6563
分支
515
quick start
Installation and usage
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
安装
$ install --globalskills.sh
使用
安装后，您可以通过在终端运行以下命令来使用此技能：
skills use sglang