How sure are you about the things you believe?
Not the obvious things. Your name. Your address. Whether gravity works.
I mean the things you think you understand.
The political argument you are convinced you have figured out. The economic policy you know will fail. The technology you believe will change everything. The competitor whose strategy you consider doomed.
Or, perhaps the business decision you are about to make because the data clearly supports it.
How much of what you believe is the result of knowledge, and how much is simply the feeling of knowing?
And how would you know the difference?
It is an uncomfortable question because we need certainty.
We cannot rebuild our understanding of reality from first principles every morning. We need assumptions. We need cultural references, trusted sources, models and intellectual shortcuts. Without them, thinking itself would become almost impossible, overwhelming.
The problem begins when we forget that they are shortcuts.
And artificial intelligence may be giving us an extraordinarily powerful way to forget.
We were biased long before AI
Our cultural background gives us a framework through which we interpret reality. Our education, professional experience and social environment reinforce parts of that framework.
And this is perfectly normal: we need mental models to navigate complexity.
But we also naturally gravitate towards information that makes sense within the models we already have.
Psychology has several names for pieces of this machinery.
Confirmation bias describes our tendency to give disproportionate attention to information that supports what we already believe.
Motivated reasoning goes further: we are remarkably good at constructing arguments that lead towards conclusions we want to be true.
And then there is a more fundamental problem.
Sometimes the knowledge required to understand something is also the knowledge required to understand how poorly we understand it.
In other words:
sometimes not knowing enough makes it harder to understand how much you don’t know.
There is another force at work too (as we mentioned here:The Price of Complexity): disagreement is expensive!
It requires effort. It creates friction. It introduces uncertainty. Sometimes it forces us to reconsider beliefs that have become part of our professional, political or even personal identity.
Well, agreement feels considerably better…
None of this started with the internet, but the internet learned how to monetize it.
CONCEPT: Cognitive Bias (click to expand the box)
When thinking takes shortcuts
A cognitive bias is a systematic tendency in judgment that can lead us away from a more accurate assessment of reality.
This does not mean humans are simply irrational. Many biases are connected to heuristics: mental shortcuts that allow us to make decisions quickly when information, time or cognitive resources are limited.
These shortcuts are often useful.
But under certain conditions they produce systematic errors.
For this discussion, the important point is simple: AI does not make the cognitive machinery disappear. If anything, it gives that machinery a much more powerful instrument to work with.
Key reference: Amos Tversky & Daniel Kahneman, Judgment under Uncertainty: Heuristics and Biases, Science, 1974.
From filter bubbles to a filter bubble of one
In 2011, Eli Pariser popularized the idea of the filter bubble: personalization systems increasingly determining what information we encounter according to what algorithms believe we are likely to engage with.
Click on certain ideas, follow certain people, interact with certain positions, and the system learns.
Gradually, your information environment becomes increasingly compatible with the person you already are.
Social networks added another layer.
We built communities around shared interests, values and beliefs. We followed people we agreed with. We joined groups where certain assumptions were rarely questioned because everyone had already agreed on them.
The echo chamber was not imposed entirely upon us; we participated in building it!
But filter bubble and echo chamber are not quite the same thing, and the distinction matters.
CONCEPT: Filter Bubble (click to expand the box)
When the information reaching you is selected for you
A filter bubble is an information environment shaped by personalization.
Algorithms observe signals about us — what we click, search, watch, follow or engage with — and use them to predict what information will be most relevant or attractive to us.
The danger is not necessarily that false information is inserted into our environment.
It is that other information gradually disappears from it.
We may therefore experience a highly personalized version of reality without being fully aware of what has been filtered out.
Key reference: Eli Pariser, The Filter Bubble: What the Internet Is Hiding from You, 2011.
CONCEPT: Echo Chamber (click to expand the box)
When disagreement loses credibility
An echo chamber is not simply a place where everyone happens to agree.
A useful distinction proposed by philosopher C. Thi Nguyen is between an epistemic bubble, where relevant outside voices are absent, and an echo chamber, where outside sources are actively discredited.
That distinction is important.
In a bubble, you may simply not hear the opposing argument.
In an echo chamber, you learn why the opposing argument should not be trusted.
That makes echo chambers particularly resilient: contradictory evidence can itself become evidence that outsiders are unreliable.
Key reference: C. Thi Nguyen, Echo Chambers and Epistemic Bubbles, Episteme, 2020.
Yet even inside the social-media bubble there was usually something outside.
Another newspaper, another community, another colleague sitting across the table saying:
“No. I think you’re wrong.”
And that matters more than we sometimes realize.
Another human being has no obligation to complete our reasoning for us… but AI, well, by design… tries to help.
And that apparently small difference changes a great deal.
The most agreeable intellectual opponent in history
Ask an AI system a question and it will usually try to provide a useful answer.
That is the entire point.
But consider what happens when the question already contains the conclusion.
“Why is remote work destroying productivity?”
“Why are electric vehicles worse for the environment?”
“Why is this economic policy going to fail?”
Or, in a management meeting:
“Analyze our sales data and explain why our pricing strategy is causing the decline.”
Yes, a well-designed AI system may challenge the premise, particularly when that premise contains obvious flaws.
But complex problems rarely come with obvious flaws.
A serious analysis depends on identifying the right variables, understanding causal relationships, and performing a thorough root-cause analysis. A missing variable, an incomplete dataset, or a seemingly reasonable inductive assumption can completely change the picture.
And AI cannot challenge what neither you nor the available evidence has given it reason to question.
So it may accept the frame we have given it and begin working inside it.
It can provide arguments, organize evidence, identify correlations, generate hypotheses, run analyses, summarize research, and turn a vague intuition into something that looks remarkably like rigorous reasoning.
And suddenly something strange has happened:
You haven’t necessarily discovered that you were right.
You’ve become better at explaining why you think you’re right.
Unfortunately, those are not the same thing.
CONCEPT: The Dunning-Kruger Effect (click to expand the box)
The problem isn’t stupidity. It’s calibration.
The Dunning-Kruger effect is frequently summarized as:
“The less you know, the more you think you know.”
Catchy.
But misleading.
The original work by Justin Kruger and David Dunning investigated a more interesting metacognitive problem.
In some domains, the skills required to perform well overlap with the skills required to evaluate whether you have performed well.
This can create a double problem: limited competence may produce errors while simultaneously making those errors harder to recognize.
The important lesson is therefore not that incompetent people believe they are geniuses.
It is that our ability to evaluate our own knowledge is itself dependent on knowledge.
And that becomes particularly relevant when AI can make limited understanding sound remarkably sophisticated.
Key reference: Justin Kruger & David Dunning, Unskilled and Unaware of It, Journal of Personality and Social Psychology, 1999.
Confidence as a service
This may be one of the least discussed risks of widespread AI adoption.
Not misinformation or hallucinations, and not even deliberate manipulation:
Confidence amplification!
Before generative AI, maintaining a sophisticated wrong opinion required work.
You had to search for supporting information. Read articles. Find arguments. Ignore contradictory evidence. Build spreadsheets. Perhaps convince colleagues.
Now much of that cognitive labour can be outsourced:
You provide the intuition, and AI provides the intellectual scaffolding.
And because the resulting argument is coherent, structured and expressed with confidence, something psychologically powerful happens:
the quality of the explanation begins to feel like evidence for the quality of the conclusion.
But eloquence is not evidence, and coherence is not truth: and a beautifully constructed argument can still be beautifully wrong…
When bias gets a spreadsheet
This problem becomes considerably more important when we move from internet arguments to business decisions.
Imagine a company whose sales are declining.
The CEO believes pricing is the problem.
That belief may be perfectly reasonable. Perhaps customers are becoming more price-sensitive. Perhaps competitors have lowered their prices…
So the team asks an AI system to analyze the available data and investigate how pricing is affecting sales.
The system discovers that certain customer segments have reacted more strongly to recent price increases.
It identifies correlations, it produces tables, and perhaps it runs statistical models.
It generates a polished executive summary explaining why pricing appears to be driving the decline.
Everyone leaves the meeting feeling that the original intuition has now been validated by data.
Except perhaps nobody asked the more important question:
What if pricing isn’t the problem?
Maybe distribution changed, or product quality deteriorated, or customer acquisition shifted towards lower-value segments.
Maybe a competitor launched a better product, or seasonality was incorrectly modeled, or the apparent price effect disappears once another variable is controlled for.
Or maybe several things are happening simultaneously.
The analysis can be perfectly competent and still lead us in the wrong direction because the question itself constrained the universe of possible answers.
There is a comforting idea in modern business that data protects us from subjective judgment.
Sometimes it does. Sometimes.
But data doesn’t remove bias.
Sometimes it just gives bias a spreadsheet.
And AI can give your bias a spreadsheet, a regression model, a presentation and a very convincing executive summary.
That is a rather dangerous upgrade.
Reasoning versus rationalization
This suggests a distinction that may become increasingly important in AI-assisted organizations:
AI-assisted reasoning versus AI-assisted rationalization.
They can look remarkably similar, but they move in opposite directions.
AI-assisted reasoning starts with uncertainty:
“Sales are falling. What are the plausible explanations?”
It generates competing hypotheses.
It asks what evidence would support or contradict each one, and it searches for alternative explanations.
It attempts to eliminate hypotheses that fail against the evidence.
And only then does it move towards a conclusion.
AI-assisted rationalization starts somewhere else:
“Sales are falling because our prices are too high. Analyze the data and explain why.”
Now the conclusion is embedded in the question.
The AI’s job quietly changes.
Instead of helping us discover what is true, we have asked it to help us construct the strongest possible case for something we already believe.
And there is an even more troubling version of this problem:
The frame itself can be manipulated.
Data is rarely presented in an organizational vacuum. It is selected, cleaned, aggregated and contextualized by people… people with assumptions, incentives, responsibilities and, sometimes, personal interests.
Someone who wants a particular conclusion does not necessarily need to falsify the data. It may be enough to decide which data to provide, which variables to emphasize, which time period to analyze, which questions to ask, and which questions never get asked.
AI can then produce an apparently rigorous analysis of an informational landscape that was already shaped to favor a particular conclusion.
This is what makes the distinction between reasoning and rationalization particularly dangerous: rationalization rarely feels like rationalization.
From inside our own heads, it feels remarkably similar to reasoning.
And inside an organization, it can look remarkably similar to evidence-based decision making.
Your private filter bubble
This is where Pariser’s filter bubble evolves into something different.
The old filter bubble was built around us.
Algorithms observed our behaviour and gradually selected information compatible with our preferences.
The AI filter bubble can be built by us, in real time.
It requires no social network, no community, no recommendation algorithm gradually learning our politics.
Just a conversation.
And unlike the traditional echo chamber, this one can feel intellectually sophisticated.
The machine does not merely say:
“I agree.”
It can produce evidence, counterarguments, historical analogies, statistical analyses, frameworks and apparently rigorous reasoning.
The result can feel less like confirmation and more like independent validation.
That distinction is crucial.
Because the most dangerous echo chamber may not be the one in which everyone agrees with you.
It may be the one in which you believe you have been challenged when you haven’t.
And perhaps this is the real evolution of the filter bubble:
The first generation was created by algorithms choosing what information to show us.
The second was reinforced by communities of people who thought like us.
The third may be something much more intimate:
a filter bubble with a population of one.
What would prove you wrong?
So perhaps the important question in the age of AI isn’t:
“Can AI help me prove that I’m right?”
It is:
“Can I use AI to discover that I’m wrong?”
There is nothing particularly new about this idea.
In fact, it takes us back to Karl Popper and one of the most influential ideas in twentieth-century philosophy of science.
Popper challenged the idea that scientific progress consists simply of accumulating observations that confirm our theories.
There will often be another confirming example, another observation compatible with the hypothesis, another explanation for why apparently contradictory evidence does not really contradict it.
The more powerful intellectual move is to ask:
What would have to happen for this idea to be wrong?
And then genuinely expose the idea to that possibility.
CONCEPT: Falsifiability (click to expand the box)
Don’t just ask what confirms your idea. Ask what could kill it.
Karl Popper argued that a scientific theory must expose itself to possible refutation.
A theory that can accommodate every conceivable outcome cannot be meaningfully tested.
This does not mean that one failed prediction mechanically settles every scientific question. Real science is more complicated than that.
The methodological principle is what matters here:
A serious attempt to prove an idea wrong tells us more than another convenient example that appears to confirm it.
For AI-assisted reasoning, this suggests a radically different prompting philosophy.
Don’t just ask:
“What supports my hypothesis?”
Ask:
“What would falsify it?”
Key reference: Karl Popper, The Logic of Scientific Discovery and Conjectures and Refutations.
The Popperian machine
This distinction becomes extraordinarily important when AI enters the picture.
An AI system can generate an impressive number of arguments compatible with almost any plausible hypothesis.
Give it your conclusion and ask for supporting evidence, and the process can become an industrialized version of confirmation bias.
One more argument, one more correlation, one more historical analogy, one more chart, one more apparently independent reason to believe what you already believed.
The quantity of confirmation increases.
But the hypothesis may never have faced a serious attempt at refutation.
A more Popperian use of AI reverses the direction of the conversation.
Don’t ask:
“What evidence supports my hypothesis?”
Ask:
“What evidence would prove my hypothesis wrong?”
Then:
“What alternative hypothesis explains the same evidence?”
“What observation would distinguish between these explanations?”
“Which assumption is carrying most of my conclusion?”
“What data would I expect to see if the opposite were true?”
“What am I interpreting as confirmation that could also be explained another way?”
And perhaps most importantly:
“If you had to destroy my argument, where would you attack it?”
Now AI is performing a very different function: it is no longer primarily an engine for generating answers, it becomes an engine for generating tests.
Attack your decisions before reality does
This may be particularly valuable in business:
Organizations are already full of incentives to confirm existing beliefs.
A strategy has a sponsor, a product has a champion, an acquisition has executives who negotiated it, a forecast has a CFO who presented it, a transformation programme has consultants who designed it.
Once enough organizational capital has been invested in an idea, asking whether the idea is wrong becomes socially and politically expensive.
AI potentially makes the problem worse by making it extremely easy to manufacture sophisticated analytical support for an existing decision.
But it can also do the opposite:
Imagine making falsification part of the decision process.
Before approving the acquisition:
“Construct the strongest possible case that we should walk away.”
Before launching the product:
“Assume this product fails eighteen months from now. What were the most likely reasons?”
Before accepting the forecast:
“Which three assumptions, if wrong, would destroy this forecast?”
Before changing pricing:
“What evidence would demonstrate that pricing is not actually the cause of the sales decline?”
Before committing to a strategy:
“What would we expect to observe six months from now if our fundamental thesis were wrong?”
This is more than asking AI for a generic list of risks.
It is deliberately using the system to attack a decision before reality does.
And that may ultimately be a much more valuable use of artificial intelligence than asking it to validate what management already wants to do.
The Socratic machine meets the Popperian machine
Socrates was irritating for a reason.
He did not make people’s arguments better, he kept asking questions until their arguments broke.
Popper gave that intellectual instinct a methodological form: expose ideas to criticism, make claims vulnerable to evidence, and treat surviving serious attempts at refutation as more informative than accumulating comfortable confirmation.
AI gives us something historically unusual:
We now have access to machines capable of generating arguments, counterarguments, hypotheses and criticism at negligible marginal cost.
That capability can be used in two completely different ways:
We can use AI to make our beliefs more defensible, or we can use it to make our beliefs more vulnerable.
The first feels better, the second is probably more useful.
And this is why blaming AI for amplifying cognitive bias misses part of the point.
AI did not invent confirmation bias, nor invent motivated reasoning.
It did not invent overconfidence, tribalism, bad management or executives falling in love with their own hypotheses.
Those came pre-installed!
AI simply gives them an extraordinarily powerful new interface.
But the same interface can be turned in the opposite direction, and the same machine that can become our private filter bubble can become our permanently available intellectual adversary.
The difference is not necessarily the model, but it is the question we ask it.
Perhaps the most important AI skill, then, will not be prompt engineering in the usual sense.
It will be something much older:
the ability to formulate questions designed to make our own beliefs fail.
Because critical thinking has never meant having fewer opinions.
It means being willing to attack your own opinions with at least as much intelligence as you use to defend them.
So perhaps the important question in the age of AI isn’t:
“Can AI help me prove that I’m right?”
It is:
“Can I use AI to discover that I’m wrong?”
If the answer is no, we may end up surrounded by the most sophisticated analytical tools humanity has ever created, and use them primarily to become better at being wrong.
If you want to know more:
- Tversky, A. and Kahneman, D. (1974). Judgment under uncertainty: Heuristics and Biases. Science, 185(4157), pp.1124–1131.
- Kruger, J. and Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), pp.1121–1134. doi:10.1037/0022-3514.77.6.1121.
- Pariser, E. (2011). The Filter Bubble: What the Internet Is Hiding from You. [online] New York: Penguin Press.
- Nguyen, C.T. (2018). ECHO CHAMBERS AND EPISTEMIC BUBBLES. Episteme, 17(2), pp.141–161. doi:10.1017/epi.2018.32.
- Popper, K.R. (2014). The Logic of Scientific Discovery.
- Popper, K.R. (2006). Conjectures and refutations : the growth of scientific knowledge. London Routledge.