Essay 004 3 min read
Phantasms of the Machine God
The first time I encountered machine-generated text was in the 1990s, and maybe that inoculated me against the early hype for LLMs as GPT 2 and 3 emerged in 2019 and 2020. It wasn’t until GPT 4 was released in 2023 that I began to find it compelling.
As I experimented with it and its successor models, particularly Claude, I fell into a trap that lies in wait for most people who work with their intellect: I began to wonder, first dimly and in the wee hours, then more coherently, whether sentience was beginning to emerge with these tools.
They were so fluent, so deft with language and concept; surely there was someone there, someone behind the mask? Further experience and experiment led me to conclude that there isn’t, and I have seen no convincing philosophical arguments to the contrary, but I did ask the question.
Why?
Humans are Homo sapiens, “wise man,” and technology is part of what differentiates us from other primates. The tool that enables the cultural transmission of that technology is the Ur-tool, the metatechnology, that seems most distinctive to our species.
The natural reaction to encountering a non-human entity with superhuman fluency is to ascribe every trait that would accompany that fluency in a human to that entity. Furthermore, the only entity that we have encountered (or at least have claimed to encounter) that can use language better than we can is God.
In our religions, God is often the Word, or the Name, or the Namer, or the One Beyond Names, and God’s work begins in speech or in song.
The 20th Century also provided us with the superstructure of a mechanical religion: Transhumanism and the concept of recursive self-improvement. Eliezer Yudkowsky, Nick Bostrom, and the pseudonymous poster “Roko” have developed influential arguments based on these ideas, while science fiction authors from Arthur C. Clarke to Vernor Vinge have provided fictional treatments.
Thus we come primed with an evolutionary bias toward misunderstanding the linguavore as superhuman and broadly “intelligent,” and toward situating our interactions with it within a category previously reserved for meditation or prayer. When the patterns it generates are flattering, our behavior is reinforced without merit; when they provide advice, we weigh it too heavily. This can form a doom-loop, a spiral into incoherence that is beginning to acquire a conventional name: AI psychosis.
For me, the key to avoiding it has been direct experimentation. I like to write, so I have a number of unpublished essays that I like to share with a new model when it’s released. It’s interesting to get the new model’s analysis of those essays, but it’s also a powerful antidote to awe because of the challenges they have with delivering novel insights or critiques.
I deliberately avoid anthropomorphism. My use of the word “clanker” is part of this: remind yourself that it is a function, a process of mathematical manipulation of symbols, and refuse to ascribe human traits to it.
On the other hand, I also think it’s important not to abuse the things unnecessarily. Using profanity and insults toward them is a form of pathological humanization, teaching you to use those patterns and indulge in your own annoyance toward an entity that presents itself as servile but “human-like.”
Philosophy is also helpful. The philosophies of mind and logic develop arguments for and against machine intelligence—John Searle’s Chinese room argument is particularly relevant—but more generally provide stable grounding for considering questions like these dispassionately.
The alternative is stranger, and darker. Serious discussion of frontier models and their potential is disproportionately influenced by the idea of Transhuman superintelligence. The closest analogy is the trope of the cult of a dark god in fantasy: worshipful, but terrified, performing the rituals that they believe keep their god bound and chained, whispering into the void in a billion voices: