AI agents can't yet do open-ended AI research

SubscribeSign in

AI agents can't yet do open-ended AI research<br>Early evidence from two case studies

Sayash Kapoor and Arvind Narayanan<br>Aug 05, 2026

43

Share

The goal of leading AI labs is recursive self-improvement (RSI): the automation of AI research using AI agents. RSI also underpins forecasts of explosive AI progress. How can we assess if we are close to this milestone?<br>One way is to use benchmarks that test if agents can conduct AI research. Given the AI community’s focus on benchmarks, they have been the dominant way to evaluate progress towards RSI. Over the last year, many such evaluations have found that agents are now able to make progress on tasks where success is easily verifiable, prompting speculation that we are on the verge of RSI.<br>But while these evaluations are helpful, they are limited to narrow, verifiable tasks. AI research can be much more open-ended. Success is often not immediately clear or verifiable, and to make progress, researchers need to test promising hypotheses, backtrack, or consider new or unconventional approaches. How can we evaluate agents’ ability to conduct open-ended AI research? We take our first step towards answering this question in a new paper.<br>We partnered with the authors of two unpublished AI papers and asked them to draft their papers’ main research questions. We then tasked frontier AI agents with conducting research to answer these questions, and gave them thousands of dollars of API credits and compute, and six days of wall-clock time. The original authors reviewed the agents’ papers.<br>The authors unambiguously rejected both agent papers. To better understand these results, our team spent over a hundred hours analyzing the agents’ logs. Our main takeaways:<br>The agents lacked the judgment for conducting open-ended research. While the agents proposed directions the expert reviewers found impressive, they quickly rejected their proposed directions based on low-quality or synthetic data.

The agents lacked awareness about the resources available to them. Both runs ended with less than 50% of the API budget spent and with hours left before the deadline, even though the agents could monitor their usage and were encouraged to spend down their budgets.

The agents did not creatively respond to feedback. Despite the agents’ own AI self-reviews surfacing many of the issues that the expert reviewers later raised, the agents did not creatively address these concerns. When faced with negative feedback they responded by adding caveats to existing findings, and doubled down on unpromising research directions.

The agents did not effectively backtrack. They retired their most ambitious research targets within the first day of the experiment, and neither agent fundamentally shifted its approach after that point.

The agents did not follow concrete instructions. They ignored explicit rules about how much time to spend on exploration, how often to get reviews from AI self-review tools, and strict limits on paper length.

We have wanted to evaluate AI’s ability to conduct open-ended research for two years, ever since we released a benchmark to study if agents could be used to improve reproducibility. But we wanted to get our method right. The idea behind our method was suggested by some of the UK AISI coauthors of the paper and refined by our core team at Princeton.<br>We call these “shadow evaluations” since the agent shadows the original study. In addition to the two of us, the core team comprises Peter Kirgis, Andrew Schwartz, and Stephan Rabanser. The full author list is at the end of this essay.<br>Shadow evaluations have important advantages: they allow us to test agents on results they haven’t been trained on and can’t access online. They also allow experts who have spent months answering the questions to evaluate agents’ outputs.1<br>But shadow evaluations also have inherent limitations. Expert reviewers know that the paper is AI-generated, and they might prefer the approach they took over the one that the agent took. Because we are conducting in-depth evaluations of each paper, the sample size is small (in our study, we used just two papers). And these evaluations necessarily involve a lot of researcher flexibility in design, execution, and interpretation.<br>In fact, we are known for a particular position in the debate on recursive self-improvement and superintelligence. This could influence how we conduct the research. We have a detailed section in the paper on our potential biases and how we address them. We sought out a team of collaborators who don’t all share our priors, and we explicitly surface the disagreements that resulted.2 For future evaluations, we are interested in having “adversarial collaborators” as part of the core team.<br>Implications for explosive AI progress<br>Our results suggest that conducting open-ended research remains challenging for frontier AI agents. Still, these findings are tentative, and we are...