Terms  /  Reasoning Cost  /  Token Burn
02 · Reasoning Cost

Token Burn

Token waste caused by a client sending the entire session history to the LLM on every call.

Token Burn is an architectural problem, not a user problem. Many AI applications are built with the simplest possible approach: append the user's message to the full conversation history and send everything to the model. This means that by turn 20, the model is re-reading turns 1-19 — including all the dead ends, corrections, and superseded decisions. By turn 50, the token cost per call has grown dramatically, and most of what the model is processing is historical noise. The user doesn't see this; they just see their bill climbing and the model's responses slowing down.

Example
An agentic coding tool sends the complete conversation to the model on every turn. At turn 1, the call is 500 tokens. By turn 30, it's 45,000 tokens — and the model is processing 30 turns of debugging history, three abandoned approaches, and 12 error messages that were already resolved. The user's actual question at turn 30 could have been answered with 3,000 tokens of relevant context. The other 42,000 tokens are Token Burn — architectural waste that a stateful context management system would eliminate.

More in Reasoning Cost