The Case for Boring AI Audits

Nobody gets excited about a financial audit. There are no launch events, no viral threads, no breathless op-eds about the thrill of reconciling accounts. And yet audits are one of the quiet reasons you can trust the numbers a company publishes, the pension you pay into and the charity you donate to.

AI needs the same kind of boredom. Not more summits. Not more grand principles. Routine, standardized, slightly tedious checks, done regularly by people who know what they're looking for.

The problem with principles

Over the past few years, there's been no shortage of AI principles. Fairness. Transparency. Accountability. Human oversight. Almost everyone agrees with them, which is exactly the problem. A principle everyone agrees with rarely tells you what to do on a Tuesday afternoon when a model update ships.

Principles describe a destination. Audits check whether you're on the road.

Consider a fictional but entirely plausible case: a mid-sized insurer, which we'll call Harborline, uses an AI system to prioritize claims for review. Its website promises "fair and transparent" decisions. Fine. But fair according to what test? Measured how often? Checked by whom? If nobody can answer those questions with a document and a date, the promise is marketing.

What a boring audit looks like

A good AI audit doesn't need to be exotic. In most cases, it would check a short list of things on a regular schedule:

• Purpose: Is the system being used for what it was designed and tested for?
• Performance: Does it work as well as claimed, and does performance hold up across different groups of people?
• Data: Where did the training and input data come from, and is it still representative?
• Oversight: Are humans actually reviewing decisions, or rubber-stamping them?
• Incidents: What went wrong since the last audit, and what was done about it?
• Change log: What changed in the system, and was each change tested?

That last one is underrated. AI systems change constantly — new model versions, new data, new prompts. A system audited in January may be meaningfully different by June. Audits that check the change log are what keep a one-time certificate from becoming a permanent excuse.

The objections, fairly stated

There are real objections to all this, and they deserve better than a wave of the hand.

"It's too expensive for small companies." True, if every audit is bespoke. That's an argument for standardized, proportionate audits — light checks for low-risk uses, deeper ones for high-stakes decisions — not for no audits at all. Financial auditing scaled the same way.

"We don't know how to audit AI yet." Partly true. The methods are still maturing. But we'll never develop good methods without doing audits and learning from them. The first financial auditors weren't perfect either.

"Audits become box-ticking." This is the most serious objection. Any audit regime can decay into paperwork. The defense is to make audits specific (test real outputs, not just policies), independent (auditors who don't depend on the client's goodwill) and public enough that outsiders can spot a sham.

"It will slow innovation." Maybe slightly, at the margins. But trust is also an input to innovation. Hospitals, schools and governments are far more willing to adopt tools they can verify. Audits can grow the market as much as they constrain it.

Who should do the auditing

This is where I'm least certain, and I think honest people disagree. Options include government regulators, independent accredited firms, academic researchers and internal teams with external review. Each has tradeoffs between independence, expertise and cost.

My instinct is that we'll end up with something like the financial model: accredited independent auditors, professional standards, and regulators who audit the auditors. It's not elegant. It's what has worked before.

The case for boredom

There's a temptation in technology to treat every problem as unprecedented and every solution as a breakthrough. Sometimes that's warranted. But a lot of what we need for AI is old and dull: written records, regular checks, independent review, consequences for getting it wrong.

The best sign that AI governance is maturing won't be a dramatic new law. It will be the day an AI audit report is so routine that nobody bothers to write a headline about it.

I'm looking forward to being bored.