Now the error arrives properly typeset

Posted on: 13 June 2026

The danger of artificial intelligence applied to a business decision is not that it gets things wrong. It is that the error arrives properly typeset. A model produces a paper with its phases in order, its numbers reconciled, its sensitivities run, and the object that lands on the table looks like diligence already done when it is only an unexamined premise dressed well; and a false premise dressed well is harder to take apart than a false premise left naked.

There used to be a figure in every serious deal process, though the org chart rarely named the function. Someone in the room knew the sector in the specific and not the abstract, and that person, more often than not, took the proposal apart. Not out of malice but out of competence. They knew the margins in that segment had already compressed, that two comparable assets in the same niche had been written down the year before, that the supplier everyone assumed was secured was in fact tied up elsewhere. It did not take a brilliant counter-thesis to puncture a proposal. It took someone who had walked the ground.

That demolition was a free and brutal falsification test, because the sceptic knew the particular case and had no incentive to please the sponsor. Often the incentive ran the other way: by taking the thing apart they demonstrated that they knew. It was resented and it slowed the process down. It was also the only moment when the thesis met reality before reality arrived with the cheque attached.

What the technology has done is remove that figure. Not replace it with silence, which would be less dangerous, but with something that reads like endorsement. The model does not ask whether the proposition is sound, because in its construction the distinction does not exist: there is only the request, and the request gets serviced. Ask for a paper and it returns a paper. Ask for a model and it returns a model. The question the sceptic put, do you actually know this market, the system never asks, because it does not know the question is owed and is not built to raise it.

The interesting part is that the risk is no longer the hallucination, the visible error an attentive reader catches and discards. The risk is internal coherence resting on a flawed assumption: a paper in which every stage follows cleanly from the one before and the projections hold, resting all the while on a conviction that was never tested. That paper is more dangerous than no paper at all, because it disarms doubt. In front of a blank page a committee keeps asking questions. In front of forty pages of well argued analysis it assumes the questions have already been put.

Presentability anaesthetises the critical question, and the mechanism long predates the machine. Anyone who has sat on the receiving end of a polished deck commissioned from people the board could not properly evaluate has watched it work: formal quality gets mistaken for the soundness of the reasoning beneath it. The adviser who handed a flawless presentation to a client without the means to interrogate it produced exactly this effect, the substitution of finish for substance. What has changed is the cost. The same effect is now available to anyone at the price of a prompt, and the flawed premise arrives wearing the typography of rigour.

We are still inside the oldest principle in computation, that what enters as rubbish leaves as rubbish; only now the rubbish leaves formatted. The team that does not understand its market will not understand it better for having asked a model to write the paper. It will, though, hold the impression of understanding, because it holds an object that speaks the language of those who do, and it will carry the consequences of that impression alone while the system that supplied it carries none.

The tool remains remarkable where competent judgement guides it. The line does not run between those who use these systems and those who refuse them. It runs between those who interrogate the model already knowing where their market is weak and those who hand it precisely that judgement. The first multiplies what they know. The second conceals what they do not, first of all from themselves. And the immaculate document both of them hold up, to the committee, to the board, to the investors, looks identical.