Demand Forecasts: The Best You Can Do Is Guess — Julien Reszka

← Blog

Frank Knight drew the line in 1921: risk can be measured, uncertainty can only be judged. Demand for anything new sits on the wrong side of that line.

40% Self-reported forecast accuracy for new-to-the-world products one year after launch, the lowest of any product category studied Kahn, K.B., 'Solving the Problems of New Product Forecasting', Journal of Product Innovation Management, March 2002

In 1921 the economist Frank Knight drew a line that most planning advice still ignores. Risk is a probability you can measure and insure against. Uncertainty is a probability nobody can measure at all, and it can only be handled by judgment, which is another word for guessing well. Demand for anything genuinely new sits on the uncertain side of that line.

Kenneth Kahn's 2002 study of new product forecasts measured exactly how uncertain, by asking companies how accurate their own forecasts turned out to be a year after launch:

Cost reduction and product improvement forecasts land closest to Knight's risk category, with 72 and 65 percent accuracy

New to the world products, the ones with no comparable history to lean on, average only 40 percent accuracy

Coca-Cola ran the most expensive demand study in the industry's history to avoid exactly this trap. Before launching New Coke in 1985 the company spent two years and 4 million dollars on 200,000 blind taste tests, and the new formula won, 53 percent to 47. Real demand disagreed. Coca-Cola fielded 400,000 angry calls and letters and pulled New Coke from shelves within three months. More research bought a better guess. It did not buy a plan.

Iridium ran the same experiment with money instead of taste. Motorola originally projected close to 5 million subscribers for its satellite phone network. By its 1998 business plan that had already been cut to 500,000 for the following year. Actual subscribers by August 1999 were 20,000. Development had cost more than 5 billion dollars by then, Iridium defaulted on 1.5 billion in loans, and the company filed for bankruptcy that same month. New Coke was wrong about taste. Iridium was wrong about a market that did not exist yet at anywhere near the size anyone had guessed, and no amount of engineering could substitute for demand nobody could actually measure in advance.

The fix is not a better forecast. It is running many cheap bets instead of one expensive one, and finding out fast which is actually winning. That trick is older than any modern product team. Frederick Taylor's stopwatch studies at Bethlehem Steel around 1899 tested different shovel loads against each other and timed every one, finding that 21 pounds per load beat every size his workers had been using by habit, and output on the job rose three to four times. The stopwatch did not predict the right shovel size in advance. It measured several real attempts against each other fast enough to tell which one was winning. The Wright brothers ran the same play with a wind tunnel, testing more than two hundred wing shapes against each other before they ever built a real aircraft.

Myth: With enough market research, you can know how much demand a new product will have before you launch it.The Coca-Cola Company, New Coke market research and consumer response, 1985

Before you present a demand number as a fact, ask whether it has real history behind it. If it does not, run several cheap versions instead of one expensive one, and benchmark them against each other fast rather than trusting your confidence in a single forecast.

Post on X

Products I recommend

Both free to try, no login needed. Results saved locally in your browser.

Trying to decide between several options instead of guessing which one wins? Put them side by side and let AI compare them on the criteria that actually matter to you.<br>Compare your options free

Stuck on a problem or chasing a goal? Describe it and AI will check whether you are reasoning from first principles or just repeating assumptions.<br>Audit your axioms

Discussion

Have you ever seen a demand forecast presented with total confidence for something that had never actually been tried before?

Post on X

Your name *

Your email * (never published)

City, country (optional)

Tell me about your situation *

Send<br>All comments are manually moderated by the author.

Subscribe to get new posts by email →

See also

Your Exit Route Determines Your Strategy

Know When to Stop, Pivot, or Double Down

They're Required to Tell You How to Beat Them

Browse all figures →<br>Browse all myths →