Was Einstein Just Autocomplete? - Audrius Berzanskis
Audrius Berzanskis
SubscribeSign in
Was Einstein Just Autocomplete?<br>How a New Representation Becomes New Information for a Bounded Mind
Audrius Berzanskis<br>May 29, 2026
Share
Often LLM debate comes down to one question: can these models produce anything genuinely new, or are they only predicting text?<br>The answer depends on what you mean by “new.” In the technical sense, an LLM may not create new information at all. But humans are cognitively limited. We can absorb only a tiny fraction of the information available to us, and even processing that fraction requires substantial time and mental effort. An LLM can process vast amounts of existing data and may surface structures humans have missed or are incapable of constructing on their own.<br>By representation I mean a structured way of organizing information that changes what can be seen or inferred. Language is itself a representation of reality. It maps the world and our thoughts onto symbols. I’m using “language” broadly here: any system of symbols and rules that lets us encode, compress, and transmit complex mental representations, whether spoken, written, or mathematical.<br>A new representation can open up cognitive territory that was closed before, and in that sense it becomes genuinely new information for the mind that encounters it. This isn’t the trivial sense in which any fact you happen not to know is “new.” A representation earns the word because it changes what you can infer next, the way a single law predicts motions you’ve never observed.
Einstein
Before Newton, people knew apples fell and planets moved. What didn’t exist was a representation that compressed those observations into a single coherent law human thought could use effectively. Newton supplied one.<br>Newton was not only a master of representation but also a brilliant experimentalist. He built instruments, performed measurements, and interacted directly with the physical world. His discoveries emerged from a combination of observation and representation.<br>Einstein pushed much further toward the representational side.<br>Einstein worked almost entirely inside language. His breakthroughs came from thought experiments performed not on objects but on representations carried in language. Riding alongside a beam of light. Falling inside an elevator. Two clocks drifting apart as the people holding them move past each other. None of these happened in a lab; they happened inside language. By manipulating representations instead of apparatus, Einstein explored possibilities that had never been seriously considered before and eventually arrived at structures that made relativity thinkable.<br>So here’s the question. If a new representation reorganizes existing information into something human thought couldn’t reach before, is the representation itself a kind of new information? Or was Einstein just doing autocomplete, reshuffling old data into a new arrangement?<br>Calling Einstein autocomplete feels strange. Of all scientists, Einstein is remembered for his willingness to reject prevailing assumptions and depart from conventional thinking. He succeeded precisely because he did not follow the dominant intellectual trajectory of his time.<br>Yet this only pushes the question one level deeper. Where did that tendency come from? Einstein’s insights did not emerge from some transcendent source. The observations, theories, and contradictions were already there. What distinguished Einstein was that he picked up on structures and implications that most of his contemporaries either missed or ignored.<br>Perhaps the real question is not whether Einstein was doing autocomplete, but what selected one continuation over another. What made him follow the unlikely path?
Subscribe
LLMs
A current large language model is perhaps the closest thing we have built to a system that operates primarily within language, much like Einstein did. It isn’t trained by letting it experience the world. It’s trained on the accumulated record of how people have written about the world. Whatever ends up in the model’s parameters is a compressed version of that record.<br>Information about the world is spread across millions of pages. Training compresses that scattered record into the model’s weights, and the corpus is far larger than any human mind could absorb directly.<br>LLMs are enormous nonlinear functions with randomness baked in, so they aren’t confined to replaying training data. Physics repeatedly shows that nonlinear interactions among many components can produce emergent structures that are not obvious from the underlying elements — turbulence, evolution, pattern formation. As a physicist, I am not surprised that a nonlinear system of this scale, with so many interacting components, would produce structures and representations nobody has seen before.<br>A model can produce continuations, some insightful and some hallucinated, that almost certainly never appeared in its training text....