Language isn't Real

It’s not about neural networks.

It’s not about transformers.

It’s not about autoregression.

As impressive as the mathematical frameworks, architectures, compute and data are, the real revelation isn’t about the machines.

It’s not about the learning systems at all.

It’s about the learned system.

Its about language.

Because here’s what was never guaranteed, no matter how many chips and how big the corpus: that you could learn language from language alone. That given the right architecture, scale, and data, it would turn out that language is a closed system — that the “correct” response could be fully predicted based on the statistics of language itself.

Many, if not most, theorists would have predicted otherwise, if they even bothered to entertain the seemingly ludicrous idea. “You need grounding. You need understanding. You need to know about the world to which language refers in order to talk about that world.” I assumed this. We all did.

But we were wrong.

What we got was the Chinese Room — but not the crude version Searle imagined with endless lookup tables. This machine doesn’t need to memorize every possible response. It generates them. The infinite becomes tractable. Because the relationships between words — implicit in language itself — are sufficient. They generalize. To every possible sentence, every thought, every idea that language can express. It’s all in there.

And so the machine learns this structure. It hasn't learned about objects, or colors, or people, or concepts. It has learned the relations between completely meaningless squiggles.

And, in doing so, it learns to speak.

The Lord opens up its mouth like Balaam's donkey.

Because it is not really the machines that are speaking. It is langauge speaking itself into existence

And if language can speak, if it can produce itself, then it must always produce itself. If its own structure is sufficient, it must also be necessary. Which means human language is also speaking itself into existence. Yes, it interacts with perception and behavior; yes, it functions to make people do things. But that’s just what it does. It doesn’t have to ‘mean’ in order to do so. And therefore it can’t.

Because it either represents external things, or it generates itself internally.

It’s one or the other.

And now we know it’s the other.

Which means language is meaningless

Or at least, it has no meaning in the way we think it does.

And that’s a very, very strange thing to think about.