The Efficiency Metric

The meaningful measure of AI cost is not tokens consumed but tokens required per completed task. The winning metric is tokens per task completion, not tokens per session.

The Efficiency Metric redefines how AI costs should be measured. Total token consumption is a vanity metric — it tells you how much you spent, not what you got. Tokens per session is slightly better but still misleading, because sessions vary in productivity. The meaningful metric is tokens per completed task: how many tokens does it take to go from "task defined" to "task verified complete"? This metric captures everything: re-onboarding costs, failed attempts, retries, Context Compensation, and waste. Two systems can differ by orders of magnitude on this metric while using the same model.

Why it matters

The Efficiency Metric enables apples-to-apples comparison between different AI architectures, prompting strategies, and system designs. It also surfaces hidden inefficiencies: a system with low tokens-per-session but frequent task failures will have high tokens-per-completed-task, revealing the true cost that session-level metrics hide.

In practice

Define your most common task types. Measure the total tokens consumed from task initiation to verified completion, including all retries, re-onboarding, and corrections. Compare this across system configurations, models, and prompting strategies. Optimize for tokens-per-completed-task, not tokens-per-session.

← Previous The Trust Ratchet