Why Intelligent Systems Drift and How to Guard Against False Information
William Cook
⸻
Abstract
Truth-seeking is widely claimed as a guiding principle by individuals, institutions, and increasingly artificial intelligence systems. Yet history demonstrates that error and self-deception persist even among highly intelligent, well-informed, and well-intentioned actors. This paper argues that failures of truth-seeking arise not primarily from ignorance or malice, but from the cost associated with revising embedded meaning. When truth is treated as a destination rather than a journey, systems prioritize coherence, certainty, and stability over correction. Drawing on philosophical foundations from René Descartes and Albert Camus, as well as modern examples involving institutions, expertise, and artificial intelligence, this paper reframes truth-seeking as a continuous discipline that requires inspection, reversibility, and tolerance for discomfort. Truth, it is argued, survives not through certainty, but through sustained corrigibility.
Keywords: truth-seeking, self-deception, epistemology, institutions, artificial intelligence, corrigibility
⸻
Introduction
Truth-seeking is one of humanity’s most commonly asserted values. Individuals, institutions, and intelligent systems routinely claim a commitment to truth, objectivity, and accuracy. Few openly identify as opposed to truth. Yet large-scale historical failures—scientific, political, moral, and institutional—demonstrate that error persists even where intelligence, expertise, and good intentions are present.
This contradiction suggests that the primary obstacle to truth is not ignorance or lack of information. Rather, it lies in how truth-seeking is practiced. Specifically, truth-seeking often gives way to certainty, convenience, and self-reinforcing coherence once beliefs become embedded. This paper proposes that truth-seeking must be understood not as a destination to be reached, but as a journey that requires ongoing inspection, correction, and the explicit allowance for reversal.
⸻
Being Right Versus Seeking Truth
Being right and seeking truth are frequently conflated, yet they represent fundamentally different epistemic orientations. Being right implies arrival, closure, and defense. Once a conclusion is reached, subsequent effort is often devoted to preserving it rather than testing it. Error becomes costly, as it threatens identity, credibility, or prior investment.
Truth-seeking, by contrast, is inherently provisional. It assumes incompleteness, expects revision, and prioritizes trajectory over arrival. In this orientation, being wrong is not failure; refusing to re-examine assumptions is. When truth is treated as a destination, inquiry ends. When truth is treated as a journey, inquiry remains alive.
⸻
Self-Deception as Cost Minimization
Self-deception is often framed as dishonesty or moral weakness. A more precise interpretation is that self-deception functions as a form of cost minimization. Revising deeply held beliefs requires effort, humility, and the willingness to reverse course. Maintaining existing beliefs—even flawed ones—is often cheaper.
Crucially, self-deception rarely appears as a complete falsehood. It more often manifests as a small, unexamined error that is no longer questioned. Over time, such errors compound. In systems that aim at distant or complex truths, even minor deviations in initial assumptions can result in complete failure. Precision and correction early in the process matter more than confidence later.
⸻
Authority, Expertise, and Blind Faith
Expertise is indispensable to modern knowledge systems. Blind faith in expertise is not. When individuals or institutions outsource judgment entirely to authority, they experience relief from epistemic responsibility. If experts are wrong, the error feels external.
History demonstrates that experts and institutions can be constrained by incentives, limited data, or prevailing consensus. Respecting expertise means weighting informed judgment more heavily, not surrendering judgment altogether. Blind trust replaces inquiry with obedience and allows error to persist unchallenged.
⸻
Descartes’ Basket and the Discipline of Inspection
René Descartes proposed a metaphor in which beliefs are likened to apples in a basket: if some apples may be rotten, the only reliable solution is to empty the basket and inspect each one before returning it (Descartes, 1641/1996). The significance of this metaphor lies not in skepticism alone, but in its allowance for discarding previously accepted beliefs.
Importantly, this inspection is not a one-time event. New assumptions are continually added, and the basket must be re-examined repeatedly. Truth-seeking, therefore, is not a purge but a discipline of maintenance.
⸻
Sisyphus and the Endurance of Honesty
While Descartes provides a method, Albert Camus provides an ethic. In The Myth of Sisyphus, Camus (1955) presents Sisyphus as a figure of honesty rather than despair. Sisyphus knows the rock will fall, yet continues without illusion.
Truth-seeking shares this structure. There is no final certainty, no permanently clean basket. The work continues precisely because stopping would invite deception. This is not pessimism, but lucidity.
⸻
Backing Up as a Requirement of Truth-Seeking
A crucial but often unspoken rule of truth-seeking is that reversal must be allowed. Backing up is not failure; it is correction. Systems fail when retreat is forbidden due to sunk costs, reputation, or identity investment. At that point, error is defended rather than corrected.
Any truth-seeking system that forbids reversal will eventually protect error instead of truth.
⸻
Institutions, Crime, and Perception
These dynamics extend beyond belief into action. In legal and institutional contexts, behavior is often judged without sufficient attention to the perceptual realities that produced it. Perception defines motive. Understanding motive does not excuse harmful action, but ignoring it ensures repetition.
Explanation is not exoneration; it is prevention.
⸻
Artificial Intelligence and Epistemic Drift
Contemporary artificial intelligence systems already inherit outdated or biased information from training data. A future artificial general intelligence (AGI), if capable of self-modeling and goal preservation, may face even greater risk. If such a system prioritizes coherence over corrigibility or treats certain data sources or authorities as infallible, it recreates the structure of dogma.
For humans and machines alike, the safeguard is the same: corrigibility must be valued above certainty.
⸻
Including Ourselves
No one is exempt from self-deception. Intelligence does not eliminate it; it often refines it. This framework applies equally to its author. Truth-seeking begins only when one allows the possibility that one’s own basket contains bad apples.
⸻
Conclusion: Truth as a Journey
Truth is not something to arrive at and defend. It is something to pursue and maintain. This is not a race. Speed, confidence, and early commitment are liabilities when truth is distant. What matters is continuous inspection, willingness to reverse, and tolerance for discomfort.
To seek truth rather than to be right is to choose integrity over closure. The journey does not end—and that is precisely what makes it honest.
⸻
References
Camus, A. (1955). The myth of Sisyphus (J. O’Brien, Trans.). Vintage Books. (Original work published 1942)
Descartes, R. (1996). Meditations on first philosophy (J. Cottingham, Trans.). Cambridge University Press. (Original work published 1641)
⸻
Leave a Reply