home/categories/debugging/orchestra-research-ai-research-skills-11-evaluation-lm-evaluation-harness-skill-md
debuggingtools
evaluating-llms-harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
maintainer
Orchestra-Research
更新於 11/20/2025
星標
6563
分支
515
quick start
Installation and usage
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
安裝
$ install --globalskills.sh
使用
安裝後,您可以透過在終端機執行以下指令來使用此技能:
skills use evaluating-llms-harness