Stop Asking Which Model Is Best. Ask Which Mistakes You Can Live With

At a dinner recently, a friend who runs a small architecture practice leaned across the table and asked me the question I get more than any other: "Which AI is the best one?"

I gave her the honest answer, which disappointed her: it depends what you want it to get wrong.

She laughed. I wasn't joking.

"Best" doesn't last long

Every few weeks a new model tops a leaderboard, and a fresh wave of posts declares a winner. Some of those results mean something. Many measure tasks you'll never do, under conditions you'll never recreate. And the top spot changes hands so often that choosing on that basis is like picking a phone plan by last week's weather.

For most people, whether that's a small business owner, a teacher or a four-person marketing team, the difference between the top few models on any given day is smaller than the difference between using one well and using one badly.

There's a quieter problem too: the product you use isn't quite the model on the leaderboard. The same model can behave differently inside different apps, because each one wraps it in its own instructions, settings and tools. What you're really choosing is a package, and nobody ranks packages.

Every model fails differently

Leaderboards flatten something important: every model has its own way of failing. One is a confident improviser that fills gaps with plausible inventions. Another is cautious to the point of uselessness and hedges every sentence. One writes beautifully and miscounts. Another counts perfectly and writes like a tax form.

These aren't footnotes. They're the actual experience of using the thing. If you run a practice where a wrong measurement costs a fortune, a confident improviser is dangerous however high it ranks. If you write marketing copy, a model that hedges everything is exhausting however accurate it is.

So the question isn't "which is best?" It's "whose mistakes can I catch?"

Three better questions

Before you choose or switch, ask these:

1. What does it get wrong, and how would I notice? Test it on ten real tasks and read the failures. If its mistakes are the kind you'd spot in two seconds, it's manageable. If they're subtle and confident, be careful.
2. What does it cost at my real volume? Ignore the price per million tokens on a pricing page. Work out the cost of the way your team actually uses it, including the time spent fixing its output.
3. Can I leave? If your prompts, files and workflows are locked into one product, you've traded a little convenience for a lot of future leverage. Prefer setups where swapping the model underneath takes a Tuesday afternoon, not a migration project.

The habits matter more than the model

The people I know who get the most out of AI aren't the ones with the newest model. They're the ones with the best habits. They give context, ask for sources and keep a library of prompts that work. They check the parts that matter and skim the parts that don't.

Those habits carry over between models. Leaderboard positions don't.

The model is the engine and your habits are the driving. Most crashes aren't engine trouble.

There's an uncomfortable corollary. When a team tells me their AI rollout failed, the first question is usually "which model did you use?" The better question is what they asked it to do, how they checked its work and who was responsible when it was wrong. A brilliant model with no review habits fails more expensively than a modest one with good habits.

What I told my friend

I didn't give her a brand name. I told her to pick two models and give each the same five jobs from her real week: a planning-application summary, a client email, a materials comparison, a fee proposal and a site-visit checklist. Then she should watch where each one stumbled.

She called me a week later. "The one everyone raves about invented a building regulation," she said. "The other one was boring and asked me which council we were in."

She chose boring, and I think she chose well.