Through a Glass, Linguistically

AI and the nature of language, thought and reality

This essay is an excerpt from the introductory chapter of a book in progress by the author titled The Autoregressive Mind: Bridging Language, Thought, and the Generative Brain

For most of human history, language has been a largely unexamined backdrop to our conscious lives. We learn to speak as children, internalizing patterns of grammar and vocabulary without ever stopping to consider the mechanisms that make communication possible. Language is treated as a given—a medium that just works, and it does so in a way so seamlessly integrated into our cognition that its operation can seem almost magical. In everyday experience, the very act of using language is transparent: we do not notice the rules, the patterns, or the complex computations that underlie every sentence we utter or every thought we form. Indeeed, unless we are struggling to come up with them, we barely even notice the very words that make up our speech. Instead, language is experienced as a natural extension of our minds, a tool that effortlessly bridges our inner experiences and the external world.

This uncritical acceptance of language’s transparency has led to a prevailing view in both popular thought and many traditional academic disciplines: that language is inherently and deeply tethered to our sensory and experiential world. Meaning is assumed to be derived from that world —our perceptions, emotions, and bodily interactions with our surroundings serve as the ultimate referents for our words and our words, somehow, capture this external reality in an immediate sense. Language, under this view, is a bridge that connects the mind to a pre-existing, objective reality.

The advent of large-language models (or ‘LLMs’) such as OpenAI’s GPT series has forced us to reconsider these assumptions. These models are built on the principle of autoregression: they generate text one token at a time, with each token determined solely by the preceding sequence. Critically and remarkably, these models learn to perform this generation based on previous examples of pure text alone: they have never seen a sunset or tasted an orange (after all, they have no sensors), nor have they been provided with the kind of data (such as digitized images) that might even begin to convey this kind of sensory information. Instead, all they have ever seen in their digital existence is text and nothing but text (or, more precisely, numerical representations of text). In other words, they have no exposure to the core kind of sense data that we humans assume underlies our fundamental grasp of reality. Yet, despite this lack of sensory input or embodied experience, LLMs can produce language that is not only syntactically correct but also rich in context, nuance, and even creativity. They can speak knowledgeably, even poetically, about those unseen sunsets or untasted oranges.

It’s important to note that these capabilities do not reflect the inherent structure of the underlying algorithms; the world knowledge is not somehow sneaked in through cleverly crafted algorithms or databases. Instead, LLM’s capabilities are a pure reflection of the structure of language itself; LLMs, like all of modern AI algorithms, are built on top of neural networks, which are highly generic learning systems that are trained to generate input/output pairs from (typically very large) volumes of preexisting data. In the case of LLMs, this data consists of a massive digitized corpus of human language. What the LLMS learn is how to predict the next ‘token’ (we may think ofi this as a word for now) in a sequence, based on the entire previous tokens in the sequence. Thus, what LLMS end up knowing is not based on the pre-existing structure of the algorithms. Everything they know, they learn directly form the linguistic corpus itself. And what they ultimately learn in this process is the relational structure of language itself; that’s all, nothing else. But we now know that it is enough.

Thus, the success of LLMs points to an alternative understanding of language—one in which the generative process is self-contained. Language generation does not require grounding in the external world but emerges from the internal consistency of the system itself. Language becomes, in effect, a self-referential medium. The text produced by an LLM is coherent not because it is linked to an external reality, but because it is the product of a system that has internalized the statistical regularities of human language. This insight forces us to ask a profound question: if a machine that operates without any sensory or experiential grounding can generate meaningful language, might the same be true for human language?

The possibility that human language may operate on similar principles to those of LLMs has far-reaching implications for our understanding of cognition. Traditionally, human language has been thought to rely on a close coupling between words and the external world: we name objects, describe events, and communicate emotions by anchoring our language to sensory experiences and bodily interactions. However, if the success of LLMs indicates that language can be understood as a self-contained, autoregressive process, then it may be that human linguistic cognition similarly relies on internal computations that are largely independent of direct sensory grounding.

This reconceptualization of language invites us to view our minds not merely as passive recipients of sensory data that then generate language, but as active generative systems that construct meaning from within. Under this view, language is not simply a tool that mirrors our external experiences; it is the very medium through which our thoughts are formed. In other words, our minds might be thought of as “linguistic” in the most literal sense—they are not just using language, they are, fundamentally, composed of it. To put this a bit differently, it is not really us who are thinking or speaking through the medium language; language is thinking or speaking on its own, through the medium of our brains and bodies.

At first glance, the notion of language as a self-contained generative process might seem to lead to a paradox. How can language, if it operates independently of external referents, still convey meaning in communication and thought? After all, if the words we use are generated solely from internal statistical patterns, then what grounds their meaning? This paradox is at the heart of the challenge posed by LLMs: while they reveal that coherent language can emerge from internal computation, they also force us to confront the traditional idea that meaning is derived from a connection to the world.

The resolution of this paradox may lie in understanding that meaning is not an inherent property of language, but rather an emergent feature arising from the interplay between a self-contained generative system and the cognitive structures that interpret and use language. In humans, this interpretative process may be shaped by pre-existing neural architectures that evolved to process and generate language, or perhaps older systems that evolved for other non-linguistic reasons that were then ‘repurposed’ by language. In either case, these neural substrates, honed by millions of years of evolution within a larger cognitive ecosystem, may provide the necessary framework for imbuing language with meaning even if the generative process itself is self-contained.

Thus, meaning in language might emerge as a kind of “second-order” phenomenon: while the generation of language follows internal, autoregressive principles, the interpretation of that language—by individuals and within communities—takes plan within a complex ecosystem of cognitive, cultural, and sensory experiences. This duality suggests that language operates on two levels. On one level, it is a self-contained, statistical system capable of generating coherent text without external grounding. On another level, its use in communication and thought is underpinned by a host of pre-existing cognitive structures that allow us to attach meaning to the patterns we generate and perceive.

Despite these promising ideas, it is important to acknowledge that we are still far from fully resolving the paradox of how a self-contained generative process gives rise to the deep sense of meaning in human communication and thought. The possibility remains that our current understanding is incomplete. We may have only glimpsed part of the picture—the interplay between an internal, autoregressive system and the cognitive structures that attach meaning to its output. The challenge, then, is to develop a comprehensive framework that accounts for both the statistical, self-contained generation of language and the rich, contextually embedded nature of human meaning.

Yet, regardless of how we ultimately solve this linguistic paradox, our understanding of language—and with it, our understanding of ourselves—has shifted dramatically and permanently. We should now see language not as a mere passive conduit for thought but as a dynamic, generative force with a computational life of its own. This transformation goes far beyond academic theory. Language is the lifeblood of contemporary human existence, the foundation of culture, and the bedrock of civilization. Rethinking language in this new light compels us to reexamine our deepest assumptions about our ideas, our beliefs, our very selves. After all, every facet of our knowledge and every nuance of our thought is encoded in this once-invisible/now visible medium, urging us to reevaluate not only our ideas about language but also our very understanding of what it means to be human.