The cost the token table doesn't show

Why intelligence and cost per task change how models compare

.png)

When it is time to pick a model, the most common path is still to open the pricing table and compare input and output cost per million tokens. The familiar names show up first, the numbers look objective, and the decision closes on that comparison. That habit makes sense. It is the unit vendors publish, it is what spreadsheets understand, and for a long time it was the most practical way to compare options.

The problem is that this reading, on its own, leaves a lot out.

What token price actually measures

Input price and output price tell you how much it costs to move a unit of text. They do not tell you how much text the model will generate to reach a result, how many retries it needs, or whether the provider's counting rules match the previous version.

Two models with similar token prices can produce very different bills on the same task. One answers briefly and finishes. Another writes more, revisits its reasoning, or simply counts tokens differently.

A concrete example: with Claude Sonnet 5, Anthropic documents a new tokenizer in which the same text produces about 30% more tokens than on Sonnet 4.6. The exact increase depends on the content. That does not automatically mean a 30% higher bill in dollars, because the price per million tokens also changed. It means something more important for comparison: looking only at $/1M tokens across generations (or across models) misleads, because the unit called "token" does not represent the same slice of text for everyone.

So token price remains useful as a reference, and incomplete as a single criterion.

The other half: quality

If we only optimize for price, the cheapest model always wins. In practice nobody chooses that way, and for good reason. Without a quality floor, "cheap" can just mean weak for that job.

The cleaner reading joins the two: how much the model costs to deliver a result, and how capable it is on that result. Taken alone, each metric lies a little. Together, they start to look like a decision.

Cost per task as a lens

Artificial Analysis publishes a chart that helps with exactly that: Intelligence Index on the vertical axis, and average cost per task for that index on the horizontal axis (log scale). Instead of asking only "how much does the token cost?", the question becomes "how much does it cost, on average, to complete a task at this intelligence level?".

This does not replace testing on your own workload. It replaces the illusion that the $/1M token table is already the answer. Cost per task gets closer to what you pay for a result, because it absorbs differences in generation, path, and partly in model behavior.

A walk through the data

Looking at the map, three regions become clear.

In the most efficient band, good-enough quality shows up with low cost per task. Variants like GPT-5.6 Luna sit at the cheap end, with solid intelligence in the low-to-mid 30s for a few cents per task. GLM-5.3-Flash sits near the frontier, with intelligence around 42 for about US$ 0.25 per task. DeepSeek V4.1 Flash delivers something near 39 for about US$ 0.28. Gemini 3.8 Flash enters the same kind of attractive zone, with intelligence in the 40s at a fraction of what many "premium" models charge per task. In the middle of the curve, names like GPT-5.6 Terra and GPT-5.6 Sol compete with competitive intelligence without the absolute top-end ticket.

At the other extreme, very high-intelligence models (some Claude Opus / Fable variants, for example) reach 50+ on the index, with cost per task in the range of several dollars. That may be exactly what a hard task needs. It does not need to be the default for everything.

Claude Sonnet 5 (max) helps show the distortion. Intelligence near 38, cost around US$ 5 per task. On the same chart, Terra, Gemini 3.8 Flash, and even Luna appear with similar or better intelligence for a fraction of the cost. The useful reading is not "Sonnet is bad". It is "the quality perception tied to a mainstream name can be misaligned with the cost per task the data shows".

Looking beyond the mainstream

One thing the chart makes hard to ignore is how much efficiency appears outside the usual shortlist. Models like GLM, DeepSeek, and Kimi often show up near the frontier: respectable quality with low cost per task.

If the comparison is only token price among two or three famous names, that band barely enters the conversation. If the comparison is minimum acceptable quality plus cost per task, it enters naturally. Sometimes the best value is not in the model everyone cites first.

How to use this when choosing

A simple playbook, and enough for most decisions:

1. Write the task clearly. What needs to come out well, and what is unacceptable.
2. Set a quality floor. Use a public index like the Intelligence Index as a starting point, and confirm with real examples from your use case.
3. In that quality band, compare cost per task, not only $/1M input and output tokens.
4. Put at least one or two options outside the famous default on the table, especially if they sit on the efficiency frontier.
5. Run the same example set on the finalists and look at real spend, stability, and quality side by side.
6. When a provider changes price or counting rules, redo the reading. Yesterday's table may be describing a different product.

None of this asks for an infinite spreadsheet. It asks for the habit of not stopping at the first visible number.

Source: Artificial Analysis, Intelligence Index vs Cost per Intelligence Index Task chart (21 Sep 2026). Approximate values from the public panel. On the Sonnet 5 tokenizer: Anthropic docs (What's new / Migration guide).