Validation Is the New Bottleneck (Also, the Old One)
SubscribeSign in
Validation Is the New Bottleneck (Also, the Old One)<br>Early-stage startups have limited options for rapid validation. That makes validation timelines the bottleneck of product development.
Pawel Brodzinski<br>Jul 29, 2026
Share
We just went over the metrics dashboard with our client. They predominantly focus on the acquisition funnel, which matters for the product now, but we read the temperature for conversion and retention, too. Finally, we have a compound metric, a.k.a. the North Star metric, that covers a range of customer behaviors.<br>Then, we have a whole list of product experiments queued up.<br>The most obvious path forward is to prioritize those experiments, try the simplest prototypes, and see how they affect the metrics. If readings go up, we double down on the idea. If they’re meh or go south, we roll back and pick another thing from the list.<br>Except it’s not that simple.
Subscribe
The Metric Liquidity
In finance, we talk about market liquidity. If the market is liquid, it means we can easily buy and sell the asset. What follows, the prices of these transactions are frequently and continuously updated to reflect how the “market” values the asset.<br>In a similar manner, we can consider metric liquidity. When Amazon deploys a feature to production, they know almost instantly whether it helps or harms their KPIs. Their metrics are liquid.<br>With our client, it’s not the case. Mere weeks after the initial release, the traction is, well, I’d love to use the word “modest,” but it’s not even that. Week-to-week readings may bounce up or down by as much as 100%. And it’s not because something significant has changed about the product. It’s because of how few data points we’re dealing with.<br>To stick with the financial market analogy, it’s as if transactions were few and far between. In such a case, the market is not liquid, and we can’t infer much from the most recent price. It might have been a one-off thing. Or it happened too long ago to tell much about the current status of the asset.<br>By the same token, metrics that are not liquid enough will tell us little about how well a product is doing. Add a game-changing feature, and if there isn’t anyone to notice, the metrics won’t budge.<br>Timelines of Validation
Fundamentally, the more liquid a metric is, the faster the lead time for experiment validation. Amazon can automatically roll back underperforming features and keep those that positively impact their KPIs. Anthropic can embrace the spaghetti strategy to see what sticks with their users. Netflix famously A/B tests the shit out of everything before settling on what’s the new norm.<br>For contrast, imagine a fledgling startup with a handful of early users. Unless the idea under scrutiny truly blows up, which happens but is extremely rare, it’s gonna be a long wait till the metrics show traction.<br>Those 3 new users who used the product a week after we released a new feature may suggest a 38% engagement increase thanks to the new idea. Or it can be a natural fluctuation that tells literally nothing. How do we know? We wait. If the change is real, the traction growth over time will compound and outweigh the natural engagement flux. We will know. Eventually.
The more liquid the metric, the more options we have<br>Metric liquidity defines validation timelines and available strategies. As much as we’d always like to have short experiment feedback lead times, without enough data liquidity, it’s not an option.<br>Why Long Validation Lead Times Suck
There’s an important reason to push for short feedback loops whenever we experiment. OK, there are many. Like the fact that humans don’t like uncertainty. We don’t like it all. We actually prefer the certainty of receiving an electric shock to a 50/50 chance of getting away without the painful experience. Curious creatures we are, indeed.<br>Electric shocks aside, long validation lead times suck in product development because they serve as a pace-maker. We either wait for the outcomes of one experiment before trying another, or we risk the outcomes of different attempts interfering with each other. In the latter case, odds are we’ll have the right change and never know it, buried under lots of other, not necessarily helpful, stuff.
Even more crucially, the outcomes of earlier experiments inform later decisions. It’s the old Build-Measure-Learn cycle from Lean Startup. Skip the “learn” part, and you drive in the fog. You can make one turn or another, but it’s mostly guesswork. Good luck trying to navigate like that, I guess.<br>When you run a startup, you’re playing against the odds already. No need to make your situation even more miserable by making your experiments random. So yes, long validation lead times suck big time (to use a technical term).<br>“We Wait” Should Be a Default Strategy at the Earliest Stage
Unless we subscribe to the notion that building a successful startup can be predesigned (which I called...