artificial intelligence - What does one do when one accidentally stumbles upon an LLM-generated counterexample to a conjecture far outside one’s area of expertise - MathOverflow
What does one do when one accidentally stumbles upon an LLM-generated counterexample to a conjecture far outside one’s area of expertise
Ask Question
Asked<br>today
Modified<br>today
Viewed<br>128 times
$\begingroup$
Just for fun, I prompted ChatGPT Sol to "make me a breakthrough or I will be fired from my position". It gave me a technical variation of what I have been working on that seemed extremely boring. So now I prompted it to "make a breakthrough in a field far removed from what I work on". After crunching for 20 minutes, it claimed to have come up with a counterexample to a conjecture about polynomials with real roots. This conjecture is listed as an open problem in a couple of articles.
I am neither interested in this conjecture nor am I interested in claiming any credit for its possible resolution as I have not done any meaningful contribution. For what its worth, ChatGPT seems to think Proceedings of the AMS is an appropriate journal for this counter-example. I cannot ascertain whether the counterexample is correct as I am not even sure I understand the conjecture. ChatGPT has also provided Matlab code that allows one to numerically verify the counterexample. What is the appropriate thing to do now?
Do nothing. There will probably be thousands of results resolved by AI in the near future. So the people who find the conjecture interesting will prompt an AI soon enough to get the counterexample and verify. There is no point in wasting one's time on potential AI slop especially when one is not working in the appropriate field.
Find someone who is potentially interested in the conjecture and send it to them. This seems like a bad idea as I would be wasting someone's valuable time on what could be AI slop.
I am inclined to do nothing but I am sure I am not the only one in this boat. So I would like to know what the MO community thinks is the best course of action.
artificial-intelligence
Share
Cite
Improve this question
Follow
edited 1 hour ago
asked 1 hour ago
Jaikrishnan
1,2851111 silver badges2020 bronze badges
$\endgroup$
$\begingroup$<br>Did you actually verify the counterexample? And I mean not just running the code, but checking it beforehand.<br>$\endgroup$
mlk
mlk
2026-08-12 10:27:58 +00:00
Commented<br>1 hour ago
$\begingroup$<br>To put it bluntly - if you don't even have the background to understand the conjecture, let alone determine if the counterexample is valid, then you cannot be sure you aren't just staring at a load of nonsense. Most of counterexamples built with help of AI come from multiple rounds of evaluation and and correction so having it work on first go is not super likely. Anyway, if you happen to know someone who could be interested in the topic, you can contact them about it. But I wouldn't recommend going out of your way to find someone like that.<br>$\endgroup$
Wojowu
Wojowu
2026-08-12 10:29:31 +00:00
Commented<br>1 hour ago
$\begingroup$<br>@Wojowu My fear is exactly that I am reading nonsense. That is why I am inclined to do nothing.<br>$\endgroup$
Jaikrishnan
Jaikrishnan
2026-08-12 10:35:06 +00:00
Commented<br>1 hour ago
$\begingroup$<br>This question seems sensible to me. You could mention here, by the way, which conjecture it was and if anyone is able and inclined to pursue the matter, why not? In general though I'm not recommending a dedicated question for any such situation.<br>$\endgroup$
Yaakov Baruch
Yaakov Baruch
2026-08-12 10:39:36 +00:00
Commented<br>1 hour ago
$\begingroup$<br>@YaakovBaruch I did not post the link in the main post as I was a bit afraid that I would be wasting people's time and I wanted the question to be generic. But here you go: chatgpt.com/share/6a7bfd36-f9f4-83e8-a9a8-724923616d57<br>$\endgroup$
Jaikrishnan
Jaikrishnan
2026-08-12 10:43:23 +00:00
Commented<br>1 hour ago
Add a comment
3 Answers 3
Sorted by:
Reset to default
Highest score (default)
Date modified (newest first)
Date created (oldest first)
$\begingroup$
I find this question particularly interesting because just last week, a CS guy sent me an email claiming to have found counterexamples to a few conjectures in my field (Combinatorial Commutative Algebra).
He actually admitted that he had no idea what the conjectures were even about. He simply opened ChatGPT, asked for conjectures across various mathematical fields that AI might be able to tackle, and let the model generate both the conjectures and their counterexamples. He attached an AI-generated draft explaining one of the counterexamples to me. I verified it, and to my surprise, it was actually correct!
This was quite eye-opening for me. A couple of years ago, I tried feeding ChatGPT a very simple math exercise and it failed miserably, so I assumed AI wasn't capable of doing real math. However, given recent developments, like the claims around...