lab-toolsresearch
validate-evaluator
Calibrate LLM-as-Judge evaluators against human labels. Computes TPR, TNR, precision, recall, F1, and Cohen's kappa. Detects systematic biases and recommends prompt corrections. Produces a calibration report with confidence intervals. Triggers on: "validate evaluator", "calibrate judge", "judge accuracy", "evaluator validation", "judge metrics"
maintainer
Miosa-osa
업데이트됨 3/19/2026
스타
172
포크
41
quick start
Installation and usage
Calibrate LLM-as-Judge evaluators against human labels. Computes TPR, TNR, precision, recall, F1, and Cohen's kappa. Detects systematic biases and recommends prompt corrections. Produces a calibration report with confidence intervals. Triggers on: "validate evaluator", "calibrate judge", "judge accuracy", "evaluator validation", "judge metrics"
설치
$ install --globalskills.sh
사용법
설치 후 터미널에서 다음 명령을 실행하여 이 스킬을 사용할 수 있습니다:
skills use validate-evaluator