Nobody Checks the Math

By Jack Butcher

Nobody Checks the Math

A £144 million foreign exchange loss became £16.65 with one email.

Not a typo. Not a rounding error. Off by 8.6 million percent.

The Transport for London expenditure report sat online for months. Automated systems processed it. Humans reviewed it. Journalists referenced it. Nobody questioned the number.

One person sent a Freedom of Information request. The error vanished instantly.

Failure = feedback.
Failure = feedback.

Organizations publish millions of data points daily. Spreadsheets with formulas copying down incorrectly. Decimal points wandering three places left. Currency conversions applied twice.

Everyone assumes someone else verified the math.

The bigger the number, the less likely anyone checks. Small errors get caught because they matter to someone's daily work. Million-dollar mistakes slide through because they're someone else's problem.

"There are three types of people: those who see, those who see when they are shown, those who do not see."
"There are three types of people: those who see, those who see when they are shown, those who do not see."

Most people scan for reasonableness. Does this look right? Could this be true? Few people run the calculation.

The person who sent that FOI request wasn't smarter than the system. They were the only one who cared enough to ask.

Every public dataset contains errors like this. Budget reports with phantom line items. Economic statistics with misplaced decimals. Research papers with impossible sample sizes.

The errors persist because verification is expensive and questioning authority is uncomfortable.

The more we know, the more we know we don't know.
The more we know, the more we know we don't know.

Professional fact-checkers verify quotes and dates. Financial auditors verify controls and procedures. Almost nobody verifies basic arithmetic on published numbers.

The opportunity is obvious. Most people trust published data completely. A few people distrust everything equally. Almost nobody applies selective skepticism to specific claims they can actually verify.

Pick any organization publishing regular financial reports. Download their monthly expenditures. Sort by largest amounts. Pick the round numbers over £100,000.

Send three FOI requests. Wait for the responses.

The person who found TfL's error wasn't hunting for corruption or conspiracy. They were checking if big numbers in public documents are actually accurate.

They found their answer.

Go deeper.

Install the full system — lessons, tools, workflows, and everything we build. $9/month or $99/year.

Stay in the loop.

New ideas, tools, and work. No spam.

Visuals

View All
Distraction.“Those who have knowledge, don’t predict. Those who predict, don’t have knowledge.”"If you don't know where you are going, any road will get you there."

Keep reading

All Articles