home/categories/machine-learning/orchestra-research-ai-research-skills-12-inference-serving-sglang-skill-md
machine-learningdata-ai
sglang
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
maintainer
Orchestra-Research
更新於 11/20/2025
星標
6563
分支
515
quick start
Installation and usage
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
安裝
$ install --globalskills.sh
使用
安裝後，您可以通過在終端運行以下命令來使用此技能：
skills use sglang