How to read an AI benchmark table

On 28 October 2026 Google launched Gemini 4 Carbon with a scorecard that was reproduced everywhere within hours: seventeen measures, five models, most cells going to Carbon. Not one number in it is invented. But read it like an exam result and you will misunderstand almost all of it.

This piece is not about which model is better. It is about how a benchmark table gets built, and the three places where the way it is built decides the winner more than the models do.

The table everyone saw

Here is the core of Google's table, numbers untouched:

On this view Carbon takes four of six rows. Now take them one at a time.

First: the table is missing the current leader

The Terminal-Bench 4.0 row gives Carbon 65.2%, the highest figure in the table. But the source leaderboard at Artificial Analysis, the very one Google cites, has Kimi K3.1 Pro at 67.2%. Kimi has no column in Google's table.

There is more. For Opus 5.5 Google uses the 59.6% that Artificial Analysis measured. Anthropic ran its own model on the same benchmark and published 66.4%. The two differ by nearly seven points, and Google took the lower figure for the rival while using its own run for itself.

Nobody is cheating here. One side runs its own configuration, the other a standardised harness. The problem is that the table does not tell you so.

Second: 77.4% is the generous column

Agents' Last Exam scores agents on real professional work on a real machine. Snorkel's source board has two columns: one for partial credit, one for finishing a task outright. Google quotes the first.

Read "77.4%" and you assume three quarters of the job is done. The strict figure on that board is 42.6%, and that is the highest any agent has reached. Google's numbers for Argon and Opus also run a few points above the source board, and Carbon does not appear on it at all.

Third: the ranking changes when you add a column

The Vending-Bench 2 row has a model run a shop for a simulated year. Google places Opus 5.5 at $9,235, which looks like fourth of five. Andon Labs' source board has GPT-6 Sol at $14,428 sitting in between, and across the full field Opus 5.5 lands around tenth, below both Opus 5 and Sonnet 5.5.

The number is right. The ranking is wrong, purely because the table picked five columns.

And the most telling figure from that benchmark is in no table at all: a competent human operator makes about $63,000 over the same year. The best model reaches roughly a quarter of that.

Seven boards, seven winners

If Google's table is suspect, which board should you trust? The uncomfortable answer: it depends which one you ask.

Every lab can claim first place without telling a single lie. They only have to pick the board.

One more detail worth keeping: Epoch is the only source that publishes confidence intervals, and the intervals of its top six models overlap completely. Statistically, those six cannot be told apart.

The numbers worth more than any scorecard

There is another family of measurements, quoted far less often, because they use real work graded by real people.

To reach that 42.6% on professional work, Claude Opus 5.5 spent 160 hours 30 minutes and 19.2 million output tokens across 122 tasks. That is the real picture of AI replacing people right now.

What nobody measures: Vietnamese

This matters most to readers in Vietnam. No live leaderboard measures Vietnamese for the current generation of models.

SEA-HELM is the only board still ranking Vietnamese, updated 18 September 2026. It contains no Opus 5.5, no Gemini 4, no GPT-6.1, no Kimi K3 and no Grok 4.x, and its top five overlap in confidence intervals. On VMLU the best closed model is still GPT-4 from January 2024.

Any page claiming model X is best at Vietnamese is asserting something nobody has measured.

So which model should you use

Drop the question of which model is strongest and ask which model does your job well. That one has an answer, and the answer is usually not the most expensive model.

A note on price: since Claude 4.7, the same text produces about 30% more tokens than before. So comparing Claude's $4 per million tokens against Gemini's $2 is not like-for-like. What compares is the money spent on a whole task.

Three questions before trusting a table

• Did the vendor run this number, or did a third party measure it? The two differ often enough to flip first place.
• How many columns does the source board have, and who was left out?
• Is the quoted column the strict one or the generous one? Many benchmarks publish both.

And a fourth, specific to Gemini 4 Carbon: the model leading the most boards is not on sale. Google released it first to a subset of cyber defenders, then to partners. The one you can buy is Gemini 4 Argon, about a point behind on most boards.

Every table here, plus six more detailed ones with a link to each primary source, lives on the AI plan guide.