home/categories/machine-learning/orchestra-research-ai-research-skills-12-inference-serving-sglang-skill-md
machine-learningdata-ai
sglang
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
maintainer
Orchestra-Research
Actualizado 11/20/2025
Estrellas
6563
Forks
515
quick start
Installation and usage
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
Instalación
$ install --globalskills.sh
Uso
Después de instalarlo, puedes usar este skill ejecutando el siguiente comando en tu terminal:
skills use sglang