Choosing the right restaurant with Multi Arm bandits
Summary
Whenever I go to a new place and have to choose a place to eat, or whenever I am looking for new earphones, I end up stuck in the same dilemma : should i choose the option with the most reviews ? Or the one with the best rating ?
Let’s imagine you are in a new city and look for a place to have dinner tonight. You quickly look up on the web and come up with two options:
Option A: A hidden gem sushi spot with a 5.0-star rating… but only 3 reviews
Option B: A popular local chain with a 4.3-star rating… but 2,500 reviews
Which one would you choose ?
If we were presented with average ratings only, we would choose A. Fortunately (or not), ratings are often coming with number of reviews, and this is where our brains get confused. Your human intuition says: “Wait, 3 reviews could just be the owner’s mom and two friends. 4.3 stars across 2,500 people is social validation.”
We make choices everyday, and most of the choices we make can be summarized as a tension between two different forces: exploration and exploitation.
“Exploration” is about taking risks and exploring less well-known options.
“Exploitation” is about following the general consensus or options which seem safer.
This trade-off is everywhere:
traders managing a portfolio and trying to maximize their returns.
Websites trying to choose which products they should display to their users to maximize conversion.
Netflix deciding which movies you will most likely watch.
So going back to our restaurants, and the critical question: how do we choose ? Or if we rephrase: how do we make the best decision ?
In bayesian statistics, the best choice is often linked to something we call minimizing regret. We will come back to this notion later. But for now, we would like a simple solution to balance reviews and ratings to make the best choice that takes into account both factors.
Why averages lie
There is a simple formula to balance ratings and reviews, used by the famous IMDB database (used to rank movies).
$$<br>\text{Bayesian Rating} = \frac{(R \times v) + (C \times m)}{v + m}<br>$$
Where:<br>- $R$ = The raw average rating of the specific item/restaurant.<br>- $v$ = The number of reviews it has.<br>- $m$ = A threshold number of reviews (e.g., 50 reviews to be taken seriously).<br>- $C$ = The average rating across the entire website (e.g., 4.0 stars).
When $v$ is tiny (e.g., 2 reviews): $m$ dominates the denominator, and $C \times m$ dominates the numerator. The rating gets dragged straight toward the platform average ($C$).When $v$ is massive (e.g., 5,000 reviews): $v$ completely overpowers $m$, the $C \times m$ term becomes negligible, and the score simplifies to $R$ (the raw average).
However, while simple, this solution has several drawbacks:
This formula assumes ratings follow a normal distribution, which is not the case. In most cases, ratings are either binary (click/don’t click, like/dislike), or multinomial (rating from 1-5, poor/average/good/excellent, etc). You cannot give a 3.542525 rating to a restaurant. So this formula unfortunately does not work in our case.
The other problem is that averages can reflect very different realities. Think about:
A restaurant with twenty 5-star reviews and twenty 1-star reviews averages out to 3.0 stars.
A restaurant with forty 3-star reviews also averages out to 3.0 stars.
Both restaurants show an average rating of 3.0, but to a human consumer they are wildly different. The first is a highly controversial, risky gamble; the second is the definition of an “average” restaurant, on which people seem to agree.
Finally, in the real world of e-commerce and Google reviews, people rarely leave 3-star reviews. They usually leave a 5-star review if they are thrilled, or a 1-star review if their order was broken or late. This creates a bimodal (camel-like) distribution.
Thinking “Bayesian”
To model this problem, we can use a Bayesian framework. Bayesian statistics are actually very close to how the human brain learns and takes decisions. A child quickly learns that putting its hand in the fire will burn. It does not need to repeat the experiment 1000 times. Let’s describe this situation in a Bayesian framework, in which we want to learn the probability that i harm myself given fire:
Prior: there is a flame, but i don’t know if it hurts or not.
Observation: i put my hand in the fire, it burns
Posterior: i update my knowledge. I know the probability i get burned by a flame is stronger now.
In our restaurant case, the prior would be: “They usually leave a 5-star review if they are thrilled, or a 1-star review if their order was broken or late”. We then gather data (also called evidence) to update our prior belief. Either it strengethens our belief, or it goes against it. We end up with a posterior distribution that shows the final prior + data update.
The...