perf-host-analysis
Analyze host/CPU overhead in TensorRT-LLM inference from nsys traces. Phase 1 (Detection): determine whether host overhead is the bottleneck via a binary YES/NO verdict with metric evidence (GPU idle ratio, host prep exposed ratio, per-phase breakdown). Phase 2 (Root Cause): isolate forward step iterations via allreduce kernel patterns, compare NVTX-instrumented host operations across versions, and pinpoint scheduling/request-management regressions. Usable standalone or as a sub-step of perf-analysis. Triggers: host overhead, inter-step gap, scheduling overhead, forward step isolation, nsys iteration analysis, NVTX breakdown, request management overhead, inference loop overhead, between-step gap, GPU idle, host bottleneck detection, host prep exposed.
Installation and usage
Analyze host/CPU overhead in TensorRT-LLM inference from nsys traces. Phase 1 (Detection): determine whether host overhead is the bottleneck via a binary YES/NO verdict with metric evidence (GPU idle ratio, host prep exposed ratio, per-phase breakdown). Phase 2 (Root Cause): isolate forward step iterations via allreduce kernel patterns, compare NVTX-instrumented host operations across versions, and pinpoint scheduling/request-management regressions. Usable standalone or as a sub-step of perf-analysis. Triggers: host overhead, inter-step gap, scheduling overhead, forward step isolation, nsys iteration analysis, NVTX breakdown, request management overhead, inference loop overhead, between-step gap, GPU idle, host bottleneck detection, host prep exposed.
์ค์น ํ ํฐ๋ฏธ๋์์ ๋ค์ ๋ช ๋ น์ ์คํํ์ฌ ์ด ์คํฌ์ ์ฌ์ฉํ ์ ์์ต๋๋ค:
skills use perf-host-analysis