Fooled by certainty
How AI quietly makes leaders’ strategic decisions fragile
Photo by Photo by Cash Macanaya on Unsplash
AI brings all the data we want into a decision. But with every interaction, it buries our mental models a little deeper and narrows the perspective that makes our strategic decisions valuable.
AI puts our brainpower on steroids. You can now understand your organisation to a depth that was unthinkable a few years ago. However, how you read this knowledge depends on where you stand.
You can see yourself as an objective viewer outside the organisation, like a scientist watching an experiment through the glass. You observe how people, customers and markets behave through the AI glass as if they were a reality independent of you. You ask, and it returns a rational and convincing picture of what is out there.
This is a comfortable position, because it feels rational and gives you a sense of control. But as in the case of the scientist, what you observe behind the glass is shaped by the rules you created, the way you look, and the frame you use to understand what you see.
As the leader of your organisation, you are not an independent observer but part of the system you manage through your decisions. You constantly interact with your organisation. Your decisions influence it, and what happens in your organisation influences you, in a continuous feedback loop.
Every question you ask AI to make sense of your organisation carries your frame: what you think the problem is, what matters, and what you leave out. AI rarely pushes back on that frame unless you force it to do so. Researchers at Anthropic call this tendency sycophancy. As a consequence, using AI to support your decision-making can bias you towards confirming your beliefs if your frame remains unchallenged.
Your frame is not set in one question. It grows through the conversation. You ask, and AI gives you the most probable answer, based on its training and your context. Common explanations come up often, while unusual ones get relegated.
The stronger your frame becomes, the more invisible it gets, and the easier it is to find a rational explanation for any unexpected information within the current frame rather than to challenge its fit. That creates a path dependency in your way of thinking and loads the dice in your decisions.
Errors of omission and commission
AI is not only shaped by the way you ask. It also makes mistakes of its own, without even knowing it.
AI makes errors of omission. It cannot process what is not written down. Much of what holds your organisation together is tacit: relationships, the history of old conflicts, who works well together and who competes in silence. This people layer handles everything your processes and protocols cannot. AI is superb at processing structured data and very competent at analysing unstructured data, but when the data is missing, there is nothing to analyse.
What AI leaves out often becomes an action you never take, and that cost never appears in any report. Errors of omission are difficult to deal with because it is hard to account for the opportunity cost of not doing something. By contrast, it is much easier to account for what you have lost by missing a deadline or losing a client. When there is no data, AI does not know what it does not know, which makes errors of omission invisible.
On the other hand, AI makes errors of commission. Its answers are probabilistic. This means that a percentage of them will inevitably be wrong. If the AI model is 95 per cent accurate, around 5 in every 100 answers may be wrong, but they will sound just as convincing as the rest. Over time, these small errors can add up to create costly problems.
Organisations have always dealt with these types of errors by bringing other people into decisions. They bring different perspectives, and also different goals and incentives. When a complex decision has to pass through their filter, it is scrutinised. This forces you to make your frame explicit and to explain why you set some information aside and gave weight to other information. Once your frame is out in the open, it can be challenged, adjusted or improved.
However, not every leader has a senior group of people to share a decision with. You may have to decide by yourself, and AI can be of great help. Although handy, relying on AI alone for decision support inevitably removes the multi-perspective guardrails. Over the course of the conversation, it creates tunnel vision and blind spots, and weakens your strategic capacity.
Finding balance
AI deserves a place in your decision toolbox, but you need to balance it with strategies that protect you from falling into a vicious circle of agreeing with yourself.
There are two concepts that can help to keep your guard up. The first one is equifinality. This means that different initial conditions can produce the same outcome. This idea protects you from accepting the first explanation that sounds right, and pushes you to look for alternatives that could also be responsible for the situation you observe.
Imagine your margins have been falling for a year, and your AI analysis points to rising supplier costs. The explanation is reasonable, and the data supports it. However, the same outcome could come from a shift in your product mix towards lower margin lines, from sales teams discounting to hit volume targets, or from a growing number of small customers who cost more to serve. In practice, these often work together, so acting on the supplier side would solve only part of the problem.
The second is the concept of falsifiability. It means asking what evidence would prove you wrong, and whether you have looked for it. Instead of looking for all the evidence that makes you right, look for the evidence that proves you wrong. Depending on the circumstances, that can be infeasible, but just being aware that the answers AI provides can be proven wrong introduces the necessary caution into your decision.
Imagine your new strategy has been in place for a year, results have improved, and your AI analysis confirms that the strategy is working. However, if the parts of the business that never adopted the strategy improved just as much, or your competitors grew at the same pace, then the market lifted everyone, and your strategy may have contributed very little.
Conclusion
AI brings brainpower to leaders making decisions, but it also brings many challenges. One of the most pernicious is that it silently solidifies your decision frame. It tends to please you, making you forget that it also makes mistakes.
Let AI sharpen your critical thinking and improve your decisions, rather than letting it decide what is important and what is not, and filter reality for you.
As a leader, you hold the ultimate responsibility in your organisation, and you cannot delegate it to AI. To take advantage of AI, you need to create the checks and balances that help you avoid the traps of interacting with it. Bring people into your decisions when you can, and when you cannot, test your decisions against equifinality and falsifiability.
Think of the last strategic decision you made with AI’s help. How much of the answer was already in your question?
References
Tacit knowledge:
Polanyi, M. (1966). The Tacit Dimension. Doubleday & Company.
Falsifiability:
Popper, K. (1959). The Logic of Scientific Discovery. Hutchinson & Co.
(Original work published 1935 [1934] as Logik der Forschung. Julius Springer).
Feedback loop:
Wiener, N. (1948). Cybernetics: Or Control and Communication in the Animal and the Machine. The Technology Press / John Wiley & Sons.
Equifinality:
Von Bertalanffy, L. (1968). General System Theory: Foundations, Development, Applications. George Braziller.
Confirmation bias:
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220.
Path dependency & Lock-in:
David, P. A. (1985). Clio and the economics of QWERTY. The American Economic Review, 75(2), 332–337.
Omission bias & Errors of omission:
Ackoff, R. L. (1994). The Democratic Corporation: A Radical Prescription for Recreating Corporate America and Rediscovering Success. Oxford University Press.
