Memory Isn't Real (Part 1)

The autoregressive theory of cognition changes everything we thought we knew about knowing

I. The Illusion of Retrieval

Years ago, a family member asked for my opinion on the death penalty. I proceeded to launch into a fairly detailed exposition, citing historical evidence, considering the sociological implications and attempting to navigate the tricky moral landscape. Afterward, I was struck by what had happened; not the position I had taken, but rather a puzzling realization: I hadn't consciously thought much about this issue before. Where did this nuanced, fully formed opinion that came gushing out of me actually come from? How did I have a structured view on a complex topic I had never deliberately contemplated?

The conventional explanation would suggest that I had subconsciously formed these beliefs over time through exposure to various influences, and they were stored somewhere in my mind, waiting to be retrieved when prompted. But recent advances in artificial intelligence suggest a more radical possibility: I didn't have that opinion —or indeed any opinion—until the very moment I was asked about it. In other words, the viewpoint I articulated wasn’t “in” my brain at all; it only came into being as I was responding to my friend’s question. This seemingly counterintuitive viewpoint is not just a fanciful philosophical conjecture; the models that underlie contemporary artificial intelligence—particularly large language models—operate precisely this way. They don't store information that is later retrieved during runtime. Instead, they operate through a process that generates text "just in time," one word after another, more like painting an abstract design than faithfully reproducing an existing scene.

If our minds operate based on similar principles this would challenge our fundamental understanding of how our basic cognition works. We intuitively picture our minds as archives where memories, knowledge, and beliefs are already present, waiting to be accessed when needed. We speak of "storing memories," "retrieving information," "holding beliefs," and "possessing knowledge"—metaphors that frame the mind as a repository of preserved experiences, facts, and convictions. This storage-retrieval paradigm dates back to antiquity; Plato likened memory to a wax tablet recording impressions, while Aristotle compared knowledge acquisition to a seal stamping its image into wax. The metaphors evolved through history—from Locke's tabula rasa upon which experiences write to the modern computational/cognitive models that explicitly compare human cognition to computer storage systems with discrete encoding, storage, and retrieval processes.

These metaphors aren't merely linguistic conveniences; they form the bedrock of how we conceive of ourselves and others as continuous beings with coherent identities. The very notion that I "know," "remember," and "believe" certain things before accessing them is not just a casual assumption—it's baked into our most fundamental concept of selfhood. We define ourselves and others largely in terms of the supposed contents of our minds: the memories we carry, the knowledge we possess, the beliefs we hold. To challenge these metaphors is to challenge our most basic understanding of what it means to be a person with a continuous identity across time.

At a more practical level, assumptions about our basic cognitive architecture inform scientific research about the brain and how it functions—and misfunctions. Much of the psychological, neuroscientific and medical literature is based either implicitly or explicitly on some version of a model— first articulated more than half a century ago— in which there are discrete long-term and short-term memory stores. The same is true in the technology space where most contemporary computer architectures are fundamentally built around these concepts of discrete storage and retrieval—from RAM to hard drives, information is encoded in specified locations and later retrieved through addressing mechanisms. This reciprocal influence between cognitive models and technology has reinforced our intuition that minds, like computers, must contain discrete stored memories. However, as this essay will suggest, the powerful algorithms that are finally demonstrating human-like cognitive abilities use architectures that abandon discrete storage in favor of generative, autoregressive approaches. Indeed, in my own lab I am currently designing a completely memory-free computational model that operates without any discrete storage components—a radical departure from conventional architectures.

The persistence of the storage-retrieval model is not surprising. Our phenomenological experience of remembering and knowing often feels passive and immediate—information seems to simply appear in consciousness, as if retrieved from some internal archive. When asked about a historical date or childhood experience, we don't experience the generative process but only its product, creating the compelling illusion that we've accessed pre-existing information. However, this intuitive storage-retrieval model may fundamentally mischaracterize the nature of cognition. When we "remember" a childhood event, "know" a historical fact, or "hold" a political belief, we may not be accessing static representations but generating them anew through complex predictive processes shaped by prior experience. The apparent stability of our memories, knowledge, and beliefs stems not from faithful preservation but from consistent patterns of generation across similar contexts.

This viewpoint goes far beyond so-called “constructivist” accounts of memory that merely suggest memories can be altered or reconstructed over time. Those accounts still presuppose some form of stored representation that serves as the basis for reconstruction. The autoregressive framework proposed here is more radical: it posits that what is "in there" are not stored representations of experiences, facts, or beliefs —only distributed parameters that shape generative capabilities. These generative processes create the compelling illusion of storage and retrieval while actually operating through principles more akin to pattern completion and sequence prediction. In this view, when we "remember" a childhood event, "know" a historical fact, or "hold" a political belief, we aren't accessing static representations but generating them anew, in the moment, through complex generative processes. These processes are shaped by prior experience of course, but only insofar as they guide future generation of information; the information itself is not contained in the brain until this generation happens. The apparent stability of our memories, knowledge, and beliefs thus stems not from faithful preservation but from consistent patterns of generation across similar contexts.

In the next sections, we will delve into the foundations of the autoregressive model, examining how it operates through principles of sequence prediction rather than storage-retrieval. We'll explore how this framework can reinterpret various memory phenomena, consider the empirical evidence from cognitive science and neurobiology, and draw parallels with modern artificial intelligence systems. By challenging our most basic assumptions about how minds work, we may arrive at a more accurate understanding of cognition—one that sees us not as archives of accumulated experience, but as dynamic, generative systems perpetually creating our sense of self, our memories, and our beliefs in each new moment. This view not only aligns with emerging technologies but may fundamentally transform how we understand ourselves as continuous beings across time.