home/categories/debugging/posthog-posthog-products-llm-analytics-skills-exploring-llm-evaluations-skill-md
debuggingtools

exploring-llm-evaluations

Investigate LLM analytics evaluations of both types — `hog` (deterministic code-based) and `llm_judge` (LLM-prompt-based). Find existing evaluations, inspect their configuration, run them against specific generations, query individual pass/fail results, and generate AI-powered summaries of patterns across many runs. Use when the user asks to debug why an evaluation is failing, surface common failure modes, compare results across filters, dry-run a Hog evaluator, prototype a new LLM-judge prompt, or manage the evaluation lifecycle (create, update, enable/disable, delete).

PostHog
maintainer
PostHog
Обновлено 4/8/2026
Звёзды
32541
Форки
2505
quick start

Installation and usage

Investigate LLM analytics evaluations of both types — `hog` (deterministic code-based) and `llm_judge` (LLM-prompt-based). Find existing evaluations, inspect their configuration, run them against specific generations, query individual pass/fail results, and generate AI-powered summaries of patterns across many runs. Use when the user asks to debug why an evaluation is failing, surface common failure modes, compare results across filters, dry-run a Hog evaluator, prototype a new LLM-judge prompt, or manage the evaluation lifecycle (create, update, enable/disable, delete).

Установка
$ install --globalskills.sh
Использование

После установки вы можете использовать этот skill, выполнив следующую команду в терминале:

skills use exploring-llm-evaluations