How To Choose A Subproblem - by Mike Winer - Millicosm
Millicosm
SubscribeSign in
How To Choose A Subproblem<br>And why the best subproblem is always 'make a picture'
Mike Winer<br>Jul 20, 2026
Share
The biggest weakness I’ve noticed in mentoring junior researchers is ‘problem-solving strategy.’ They have a chestful of medals in math olympiads, they aced every class at MIT. But they’ve also never worked on a problem that took more than 10 hours. Now we want them to produce something over the course of 10 weeks, though for all we know, the problem can be done in 10 days, or perhaps more like 10 years.<br>Promising young researchers struggle to solve 400-hour problems, but if they’d been handed a sequence of 400 1-hour problems they’d be tearing through them. Nobody can be expected to write down a 400-step plan. The thing to learn, then, is how to peel the next 20-hour problem off of a 400-hour problem, and then peel 1-hour problems off of that. These are important strategic decisions, and making them well requires both practice at the skill and data about the specific problem. I can’t get you data about your problem1, but I can help you build the skills.<br>So in order to save you from a monster problem, I created…
This advice is mostly aimed at fairly junior people doing some sort of theoretical work, like theoretical physics or computer science or AI alignment. Okay, fine, it’s specifically aimed at the people I’m mentoring at work. But the rest of you should be able to get something out of it, even if you don’t recognize every single example.<br>How To Find Subproblems
A standard format for a problem is to answer some question about some system. What is the entropy of this black hole? What’s the expected number of 10-cycles in a random regular graph? How can you prevent a powerful AI from escaping the data center? Given this sort of problem, there are three natural ways to make a subproblem:<br>Ask a simpler question
Ask about a simpler system
Ask what the fuck is even going on
Suppose someone asks you to adapt ARC’s cumulant propagation algorithm to work for transformers (if that means nothing to you, reader, just know that we want some kind of algorithm to work on some sort of machine learning thing). There are three natural ways you could try to simplify it.<br>You can start small . There are two time-honored ways of finding a good special case. You can take the simplest case you don’t already understand. The cumulant propagation algorithm is complicated; why don’t we forget about all this cumulant stuff and just deal with means? If you’re a string theorist trying to ask a question about scattering, try just the tree-level amplitudes. If you’re trying to prove something can be done in O(N^2) time, first check that there’s a polynomial-time algorithm.
They say the first step is admitting you have a problem…<br>You could also work on a toy problem . The transformer architecture is complicated. Why not focus on a simpler architecture until you understand how the algorithm works there? Then you can build your way back up.<br>The goal for toy problems is to find the very simplest system which has the thing you actually care about. I spent a lot of time studying the physics of the liquid-glass transition. Window glass is typically made mostly of silicon and oxygen with various impurities thrown in. But studying hypothetical glass-like substances with just one type of atom was simpler, so that’s what I did.<br>A physicist’s favorite technique to find toy models is taking limits of the original problem. Why not set the width to infinity? Or the depth? Or the number of tokens?2
So complicated! Why not work on something simpler first?<br>A final way to approach a problem is fact-finding . Learn basic things about cumulant propagation and about transformers. Maybe that means reading other people’s papers. That’s fine! ‘What does this paper say’ is a very important question to be able to answer, even if you can’t always publish your results. Or you might learn less-basic things. What are the asymptotics of the cumulants in a transformer? What is the distribution of weights in a softmax?<br>One extremely important variant of fact-finding is fact-processing . The information already exists, but you need to shape it into something your brain can better understand. Maybe that means translating from the language of math to the language of economics or physics. But, in my experience, the single most important thing you can do to process information is 🚨PUT IT IN A PICTURE🚨 . Your brain has a lot more machinery for visual processing than vector calculus. Put that gray matter to work!
If you want to understand an idea… make a picture!<br>These moves can and should be repeated, creating a fractal structure. In fact, if you look up, you’ll see that the problem your advisor handed you was a sub-sub-sub-problem of something much larger like aligning super-intelligent AI or figuring out a Theory of Everything. Much of the day-to-day work I do is...