Inside ARC-AGI-3: The Benchmark Built to Measure Real Intelligence | by Alan Scott Encinas | Jul, 2026 | MediumSitemapOpen in appSign up<br>Sign in

Medium Logo

Get app<br>Write

Search

Sign up<br>Sign in

What this actually is<br>Why it’s the field’s uncomfortable mirror<br>The interactive leap<br>Why anyone should care<br>The bet I’m making<br>Where I’m standing right now

Inside ARC-AGI-3: The Benchmark Built to Measure Real Intelligence

Alan Scott Encinas

5 min read·<br>4 hours ago

Listen

Share

Hand a seven-year-old a video game they have never seen. No manual, no tutorial. Within a few minutes they have worked out which button does what, what kills them, and what they are supposed to be chasing. Nobody explained the rules. They learned them by poking at the world and watching what happened.<br>That casual, almost invisible act is one of the hardest unsolved problems in artificial intelligence. I’m competing to build a machine that can do it.<br>This is a new thread in my builder’s log, and a second competition running alongside the hyperspectral one I have been writing about. Completely different game. Same deal: you get it as it happens, from behind a delay, with the parts that are still my edge kept dark.<br>What this actually is<br>The competition is the ARC Prize 2026, built on a benchmark called ARC-AGI. To understand why it matters, you have to know what it was built to embarrass.<br>ARC stands for the Abstraction and Reasoning Corpus, and it came from François Chollet, the researcher who also built Keras, one of the tools that put deep learning in everyone’s hands. In 2019 he wrote a paper called On the Measure of Intelligence with an argument that has aged into a quiet prophecy. Most of what we call AI progress, he said, is not intelligence at all. It is skill. A model that has seen ten million chess games is not smart at chess, it is well-rehearsed. Real intelligence is not how much you know. It is how efficiently you handle something you were never prepared for.<br>So he designed a test that you cannot rehearse for. ARC is a set of little puzzles where every single one follows a rule you have never seen, and you get only a handful of examples before you have to generalize. You cannot memorize your way through it, because nothing repeats. The only thing that helps is the raw ability to look at a few examples of a brand new pattern and grasp what is going on.<br>Why it’s the field’s uncomfortable mirror<br>Here is the part that keeps ARC in the headlines. Humans, including children, solve these puzzles comfortably. The most powerful AI systems on earth, for years, scored close to nothing.<br>That gap is the whole point. The same models that pass the bar exam, write working code, and draft a passable essay would sit down in front of ARC and fall apart, because passing the bar exam is a memory-and-pattern feat and ARC is not. It is the benchmark that refuses to be impressed by scale. Every time the models get bigger and quietly start to crack one version, the benchmark gets sharpened into a harder one, and the gap reopens. ARC is the mirror the field keeps walking past, the standing evidence that “knows almost everything” and “can actually think” are not the same sentence.<br>The ARC Prize exists to do something about that. It is an open competition, with real money behind it, deliberately built to drag the field’s attention away from “make the model bigger” and toward “make the model adapt.” It rewards efficiency and generalization, the things scale alone does not buy.<br>The interactive leap<br>Earlier versions of ARC were static. Look at a few examples, produce one answer, done. ARC-AGI-3, the one I’m competing in, makes it interactive, and that changes everything.<br>Now the AI is not staring at a snapshot. It is dropped inside a small game it has never encountered and has to win. It is not told the goal, the controls, or the rules. It has to take an action, watch what the world does in response, form a guess, test the guess, and slowly build a working theory of a place nobody described to it. It is the difference between reasoning about a photograph and figuring out a machine by pressing its buttons. That is far closer to how a real mind learns, by doing, by consequence, and it is far harder for a computer.<br>And you still cannot study for it. The games it gets graded on are hidden. There is no dataset to scrape, no test to pretrain on. The only thing that helps is learning on the fly, in a world you have never seen, from a standing start.<br>Why anyone should care<br>Strip away the contest and here is the stakes. Today’s AI is dazzling inside the conditions it was trained on and brittle the moment reality steps outside them. It is a brilliant student who has memorized every past exam and panics at a question phrased a new way.<br>The skill ARC measures, getting your footing in a situation nobody prepared you for, is the exact thing standing between the narrow tools we have now and machines that can handle the truly unfamiliar. A robot in a kitchen it has...