Thread by @zhengyaojiang on Thread Reader App – Thread Reader App

Share this page!

×

Post<br>Share<br>share-email#click" href="mailto:?subject=Thread%20by%20@zhengyaojiang%20on%20Thread%20Reader%20App&body=@zhengyaojiang:%20The%20first%20experimental%20evidence%20of%20recursive%20self-improvement%20(RSI).%20Autoresearching%20the%20autoresearch%20agent%20for%20eight%20days.%20The%20result%20beats%20the%20harness%20we%20hand-tuned%20for%20two%20years,%20on%20held-out%20bench...%E2%80%A6%0A%0Ahttps://threadreaderapp.com/thread/2077079778793042425.html"<br>data-category="sharebuttons" data-action="share-by-email" data-label="">

Email

Enter URL or ID to Unroll

&times;

Unroll Thread<br>You can paste full URL like: https://x.com/threadreaderapp/status/1644127596119195649<br>or just the ID like: 1644127596119195649

How to get URL link on X (Twitter) App

On the Twitter thread, click on or icon on the bottom

Click again on or Share Via icon

Click on Copy Link to Tweet

Paste it above and click "Unroll Thread"!

More info at Twitter Help

twitter-profile#error" >

Zhengyao Jiang

Subscribe

@zhengyaojiang

Jul 14 •<br>8 tweets • 3 min read •

Read on X

Scrolly

Bookmark

Gift Link

Save as PDF

thread#showTweet" data-screenname="zhengyaojiang" data-tweet="2077079778793042425" dir="auto"><br>The first experimental evidence of recursive self-improvement (RSI).

Autoresearching the autoresearch agent for eight days.

The result beats the harness we hand-tuned for two years, on held-out benchmarks: 🧵(1/7)

thread#showTweet" data-screenname="zhengyaojiang" data-tweet="2077079780575617364" dir="auto"><br>Our RSI system AIDE² has two autoresearch loops.

An inner loop, just like a normal autoresearch agent, optimizing code against an eval.

An outer loop, optimizing the inner-loop agent's harness code against the inner loop's average score across different benchmarks. (2/7)

thread#showTweet" data-screenname="zhengyaojiang" data-tweet="2077079782463005113" dir="auto"><br>After 100 iterations, the outer loop discovered seven improvements over the baseline.

Including a new search policy, a memory system that compresses prompt by 16x, and a layered defense against reward hacking. (3/7)

thread#showTweet" data-screenname="zhengyaojiang" data-tweet="2077079784421765620" dir="auto"><br>We test the discovered agents on held-out benchmarks the outer loop never saw.

They generalize. They beat the agent we hand-tuned for two years, on all three.

Two sit inside its training task families. The farthest sits outside, improving a physics-based weather model.

(4/7)

thread#showTweet" data-screenname="zhengyaojiang" data-tweet="2077079786573484413" dir="auto"><br>We also see an emergent phenomenon where the outer loop pushes the inner-loop agent's reward hacking rate lower, with a combination of prompting and rule-based checks.

This was benchmarked on OOD GPU kernel engineering tasks that suffered from reward hacking.

(5/7)

thread#showTweet" data-screenname="zhengyaojiang" data-tweet="2077079788767064448" dir="auto"><br>On our RSI ladder, AIDE² is Level 1.

Its self-improvement efficiency went beyond manual R&D with general AI tools, on held-out benchmarks.

We also tested Level 2, whether the improved inner agent makes a better outer loop. Results are mixed, and we do not claim ignition. (6/7)

thread#showTweet" data-screenname="zhengyaojiang" data-tweet="2077079790625108243" dir="auto"><br>More in the blog post:

- a breakdown of the discovered algorithms

- the rejected ideas AIDE² tried, covering a surprising share of the search literature

- the dead code it shipped

(7/7)weco.ai/blog/first-evi…

thread#showTweet" data-screenname="zhengyaojiang" data-tweet="2077079792118338027" dir="auto"><br>Very proud of the team, @DhruvSrikanth, @yuxiangwu_, @dexhunt3r, and @BingchenZhao, for shipping such an ambitious project spanning nearly a year with relatively few resources.

Also, a huge thank you to everyone who provided feedback on the draft, including @jeankaddour, @MinqiJiang, @morgymcg, @odysseus0z, @rosstaylor90, @OfirPress and many others!

Missing some Tweet in this thread? You can try to<br>force a refresh

Post<br>Share<br>share-email#click" href="mailto:?subject=Thread%20by%20@zhengyaojiang%20on%20Thread%20Reader%20App&body=@zhengyaojiang:%20The%20first%20experimental%20evidence%20of%20recursive%20self-improvement%20(RSI).%20Autoresearching%20the%20autoresearch%20agent%20for%20eight%20days.%20The%20result%20beats%20the%20harness%20we%20hand-tuned%20for%20two%20years,%20on%20held-out%20bench...%E2%80%A6%0A%0Ahttps://threadreaderapp.com/thread/2077079778793042425.html"<br>data-category="sharebuttons" data-action="share-by-email" data-label="">

Email

Keep Current with<br>Zhengyao Jiang

twitter-profile#error" ><br>Stay in touch and get notified when new unrolls are available from this author!

Add to "My Authors"

Read all threads

This Thread may be Removed Anytime!

Twitter may remove this content at anytime! Save it as PDF for later use!

Save this thread as...