Pitfalls of Estimating Parameters from Aggregates

May 27, 2026

Pitfalls of Estimating Parameters from Aggregates

One of the most common mistakes in data analysis is treating observed aggregates as if they are the parameters themselves. Observed data is not the parameter — it is the result of a process governed by unknown parameters. Let’s start with a simple example.

The Coin Toss Analogy<br>You flip a coin 20 times and observe 8 heads. The data consists of:<br>n = 20 (number of trials)<br>y = 8 (number of heads)<br>The parameter of interest is p, the unknown true probability of landing heads.

A naive person might say “the probability is 8/20 = 0.4.” But this is not the true parameter — it is merely a statistic computed from the data. The actual p remains unknown.

A more principled approach clearly separates the parameter from the data:<br>Prior: p ~ Beta(5, 5)<br>Likelihood: y | p, n ~ Binomial(n, p)

We then infer the posterior distribution of p given the observed data.

The Same Mistake in Marketing Analytics

This exact error appears frequently when calculating bids for user acquisition campaigns, especially when optimizing for ROAS (Return on Ad Spend).

A common formula for setting a bid might look like this:

For simplicity, let’s assume the target ROAS is 100% (i.e., break-even). The formula simplifies to:

The Naive (and Wrong) Approach

People often plug in observed aggregates directly:<br>revenue = 500<br>installs = 100<br>impressions = 10_000

ARPU = revenue / installs # 5.0<br>IPM = (installs / impressions) * 1000 # 10.0

bid = ARPU * IPM # 50<br>This looks reasonable — until you realize it’s the same mistake as treating 8/20 as the true probability p. You’re using sample statistics as if they were the true parameters.

Two Critical Issues

Parameters vs. Statistics<br>The observed ARPU and IPM are noisy estimates, not the true values. They carry sampling error, especially when based on small cohorts.

Expectation of Product ≠ Product of Expectations<br>Even if your estimates of ARPU and IPM were perfect, this would still be wrong:

ARPU and IPM are often correlated (high-quality users may have both higher conversion rates and higher LTV). Ignoring this correlation leads to systematically biased bids.

A Better Approach: Model the Parameters Properly

Instead of plugging in point estimates, we should treat ARPU and IPM as unknown random variables and model them explicitly:

# Priors<br>ARPU ~ Gamma(1, 1) # Weakly informative prior<br>IPM ~ Beta(1, 100) * 1000 # Prior for install rate scaled to IPM

# Observation models

# sensible choice but outside the scope of this article<br>sigma_arpu ~ HalfNormal(1)<br>y_arpu | ARPU, n ~ Normal(ARPU, sigma_arpu / sqrt(n))

sigma_ipm ~ HalfNormal(1)<br>y_ipm | IPM, n ~ Normal(IPM, sigma_ipm / sqrt(n))

# Bid is the posterior expectation of the product<br>bid = E[ARPU × IPM | data]<br>This framework naturally accounts for uncertainty, sample size, and potential correlation between ARPU and IPM (which can be modeled explicitly with a multivariate prior or copula if needed).

Key Takeaway

Never confuse the map with the territory. Aggregated metrics are the map — noisy observations of an underlying reality. Treating them as ground truth leads to overconfident and often biased decisions.

The next time you’re calculating bids, LTVs, or any other key parameter from cohort data, ask yourself:<br>Am I estimating the parameter, or am I just doing arithmetic on the data?

About Alexander Petrov

I build products for fun and profit. web page

Subscribe to get future posts via email (or grab the RSS feed)

Subscribe