home/categories/machine-learning/davila7-claude-code-templates-cli-tool-components-skills-ai-research-emerging-techniques-knowledge-distillation-skill-md
machine-learningdata-ai
knowledge-distillation
Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.
maintainer
davila7
更新於 2026/1/20
星標
17577
分支
1576
quick start
Installation and usage
Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.
安裝
$ install --globalskills.sh
使用
安裝後，您可以透過在終端機執行以下指令來使用此技能：
skills use knowledge-distillation