Hot Take: LLM can 'jump' | Yong Zheng-Xin

Hot Take: LLM can 'jump'

An interconnected-knowledge thesis: what it means for scientific breakthroughs, potential safety risks, and continual learning.

Published

8 August 2026

On This Page

I want to lodge my takes on the how LLMs can advance science through major breakthroughs, and what the assumptions behind my takes imply for safety risks and continual learning.

Let me start with my disagreement with the ICML’26 position paper that “LLM can’t jump” by Tom Zahavy.

Preliminaries

In “LLMs can’t jump”, Tom used the General Relativity case study, where Einstein was inspired by his elevator thought experiment while trying to construct a mathematical theory of gravity. Mercury’s anomalous orbit later became an important test of the theory.

Specifically, this required abductive reasoning: inferring the most plausible explanation for the apparent equivalence between acceleration and gravity. Tom explained that “(Einstein) relied on a physical prior. Because the simulated sensory experience of acceleration was indistinguishable from the remembered sensory experience of gravity, Einstein abducted that they must be the same phenomenon. The field inside the box was not a fake inertial effect; it was, by definition, a genuine gravitational field.”

Tom also made his case strong through the following arguments:

There wasn’t a need for a new gravity theory as Newtonian framework was working well, and the anomalous orbit of Mercury could be potentially explained by other factors.

There weren’t axioms available at the time for him to come up with the theory via deduction.

In other words, Tom pushed for the point that, given that LLMs do not have a reliable world model (which is necessary for the physical priors) and there’s no dataset to really derive General Relativity, LLMs wouldn’t have jumped and come up with the breakthrough.

All in all, I believe that Tom and I would probably agree with the following:

LLMs are capable of deduction or induction.

Abduction––specifically relying on sensory experience––was the way of how Einstein came up with The Theory of General Relativity.

However, my main disagreement isn’t whether LLMs could do similar abduction via the same imagination, but whether someone could later stumble upon General Relativity through a different, more deductive path. Surprisingly, this wasn’t surfaced in the ICML 2026 reviews.

Is there another way to derive General Relativity?

I went down a rabbit hole of asking if there could ever be a contemporary discovery on General Relativity.. I came across this article, which suggests that Einstein’s abductive route was not the only possible route to General Relativity.

“Instead of trying to explain the rest of physics in terms of gravity I propose to reverse the problem by changing history. Suppose Einstein never existed…”

— Richard Feynman

Here’s the relevant text as verbatim: “on the basis of the general principles of quantum field theory and of experimental results it is possible to conclude that gravity, as any other force, has to be mediated by exchanges of a virtual particle, which in this case is a massless neutral spin-2 quantum, the graviton. Thus, by constructing a Lorentz invariant quantum field theory of the graviton and by imposing certain consistency requirements, full general relativity should be recovered .”

Curious readers can read upon page 4 in details, where Feynman route starts from special relativity, then adds quantum field theory and empirical properties of gravity. These assumptions do not directly presuppose Einstein’s theory.

Note that my point wasn’t to establish that Feynman would inevitably have rediscovered general relativity from scratch. He already knew Einstein’s theory, and reconstructing a known result is easier than discovering it for the first time. Furthermore, historically, quantum field theory was developed after General Relativity was proposed (but do not depend on General Relativity).

My narrower claim is that, given a sufficiently rich body of later knowledge, general relativity can be recovered without Einstein’s thought experiment. Much of Feynman’s route is deductive, which the kind of reasoning that LLMs already exhibit especially with OpenAI’s Astra models.

Why interconnected knowledge makes “jumping” possible

Going beyond the case study of Tom’s work, I believe the essence is whether LLMs can come up with plausible explanations that enable breakthroughs. I agree with Tom’s argument that Schmidhuber’s perspective of compression, which is “the search for a simple program that concisely explains observations” would not unlock the abductive jump (read: come up with plausible hypothesis).

My deeper belief why LLMs can achieve major scientific breakthroughs is about the structure of knowledge itself . Because facts and axioms are connected via mathematical structures or causal relationships, one factual claim or observation either limits the...