Terms  /  Measurement  /  Quality-Per-Token
09 · Measurement

Quality-Per-Token

A measure of output quality relative to tokens consumed. Distinct from Token ROI (value) — this measures correctness, completeness, and relevance per unit of cost.

Quality-Per-Token fills a gap that Token ROI leaves open. Token ROI measures value — did the output produce a business result? Quality-Per-Token measures craftsmanship — was the output correct, complete, and well-executed? You can have high Token ROI (the feature shipped) with low Quality-Per-Token (the code has technical debt, missing tests, and poor error handling). Quality-Per-Token catches the difference between "it works" and "it works well."

Example
Two agents complete the same task. Agent A: 3,000 tokens, produces a function with correct logic, proper error handling, comprehensive tests, and clear documentation. Agent B: 8,000 tokens, produces a function with correct logic but no error handling, minimal tests, and no documentation. Both functions "work" — Token ROI is similar. But Agent A's Quality-Per-Token is dramatically higher: better output at lower token cost. Measuring Quality-Per-Token reveals that Agent A's system architecture (better context, better prompting) produces superior craftsmanship per unit of spending.

More in Measurement