home/categories/lab-tools/hamelsmu-evals-skills-skills-validate-evaluator-skill-md
lab-toolsresearch

validate-evaluator

Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with standard unit tests).

hamelsmu
maintainer
hamelsmu
์—…๋ฐ์ดํŠธ๋จ 3/3/2026
์Šคํƒ€
1105
ํฌํฌ
117
quick start

Installation and usage

Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with standard unit tests).

์„ค์น˜
$ install --globalskills.sh
์‚ฌ์šฉ๋ฒ•

์„ค์น˜ ํ›„ ํ„ฐ๋ฏธ๋„์—์„œ ๋‹ค์Œ ๋ช…๋ น์„ ์‹คํ–‰ํ•˜์—ฌ ์ด ์Šคํ‚ฌ์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค:

skills use validate-evaluator