We are in the era of Science Slop - by Jonathan Oppenheim

Post Quantum

SubscribeSign in

We are in the era of Science Slop<br>(and it's exciting)

Jonathan Oppenheim<br>Dec 05, 2025

53

11

Share

Created with irony by Sora<br>It feels like there’s been a sea change over the last six months. Most of my colleagues now use LLMs regularly— for bouncing around ideas, proofreading, even doing calculations. A number of high-profile examples have broken into social media: Scott Aaronson credited GPT-5 with a student-level insight in a recent proof. Tim Gowers wrote that he’s crossed a threshold where LLMs save him more time than they waste, Terence Tao has been all in for AI assisted maths for some time. AI is clearly going to make a lot of great scientists even greater. But it’s also going to bury many genuine breakthroughs under a torrent of slop.<br>The First AI-Generated Physics Paper?

It’s interesting that so far, it’s mostly been mathematicians who’ve been at the front of embracing AI’s help. I suspect this is partly because there is the promise of being able to verify AI output using formal proof systems like Lean—if it type-checks, it’s correct. Physicists have been slower to claim progress.<br>Thanks for reading Post Quantum! Subscribe for free to receive new posts and support my work.

Subscribe

Which brings me to a paper that’s been making the rounds. My colleague Steve Hsu recently announced on X that he’s “published the first research article in theoretical physics in which the main idea came from a LLM —GPT-5 in this case.” Greg Brockman of OpenAI promptly amplified this, and the story has been picked up as a milestone for AI in science, as part OpenAI’s broader push.<br>There’s just one problem: the paper answers a question whose answer we’ve known for 35 years.<br>UPDATE : Upon closer inspection, there’s a more significant problem . Turns out the AI didn’t just reinvent the wheel — it also pointed it in the wrong direction. I’ve updated the post with italics to reflect this. (Dec 7, 2025, 11:35 BST) and posted a manuscript in the arXiv.<br>The paper, which is published in Physics Letters B, claims to determine whether nonlinear modifications to quantum field theory can be made compatible with special relativity, and it does so in a very complicated way. And while I have no reason to doubt the actual math, I’m pretty confident that Steve published this as an example of what an AI could do, rather than as an example of interesting physics. Which is what makes this a cautionary tale.<br>You see, already in 1990, Nicolas Gisin and Joe Polchinski showed that nonlinear modifications to quantum mechanics allow superluminal signalling, and even worse, cause the statistical interpretation of the quantum mechanical density matrix to break down. In the comments I explain the proof, because it’s so beautiful and elegant.<br>Now, the AI does at some point acknowledge the prior work of Gisin and Polchinski, but doesn’t explain what its new formalism and criteria adds. In fact, the criteria that the LLM came up with is not a criteria for determining whether non-linear modifications of quantum mechanics violate relativistic covariance (the subject of the paper). Instead, it presents a criteria for whether non-local modifications to the Hamiltonain violate relativistic covariance, which of course they do. The LLM’s criteria fails to catch local, non-linear modifications to the Hamiltonian, but does catch non-local and non-linear modifications, but only because they are also non-local. Simple put, the criteria the LLM comes up with has nothing to do with non-linear modifications to quantum theory . I’ve posted some details in the comments, but it’s interesting that the LLM’s criteria looks reasonable at first glance, and only falls apart with more detailed scrutiny, which matches my experience the times I’ve tried to use them.<br>The LLM almost makes a contribution —namely, it considers a claimed loophole to the Gisin-Polchinski no-go theorem, due to Kaplan and Rajendran. Here, there could be some benefit to a field-theoretic analysis of the sort GPT performs, but the AI makes little attempt to seriously engage with Kaplan and Rajendran whose claimed loophole lies outside the scope of the methods it uses. The AI doesn’t end up refuting Kaplan and Rajendran but rather makes a perfunctory gesture toward one. Typical LLM vagueness.<br>Now, Steve has done valuable work on designing systems to determine when LLM output can be trusted—he knows better than most that these systems make errors. And I have no reason to believe the paper is *wrong*. It’s just... unnecessary. The AI suggested using the Tomonaga-Schwinger formalism to determine when non-linearities violate relativity, and this produces technically correct mathematics, but the criteria it comes up with has nothing to do with the actual problem it claims to solve.<br>Nor did it have the scientific taste to say: “Actually, this research direction was answered decades ago. What specifically are...