lab-toolsresearch
validate-evaluator
Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with standard unit tests).
maintainer
hamelsmu
Обновлено 3/3/2026
Звёзды
1105
Форки
117
quick start
Installation and usage
Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with standard unit tests).
Установка
$ install --globalskills.sh
Использование
После установки вы можете использовать этот skill, выполнив следующую команду в терминале:
skills use validate-evaluator