Which country voted the best at Eurovision? - Lalit Maganti
Eurovision was on yesterday. I’ve never been interested much in the musical side<br>but the weird political dynamics of Eurovision voting have always fascinated me;<br>I tune in each year just for them and somewhat snarky commentary of Graham<br>Norton, the UK commentator.<br>As I was watching the jury votes come in, a question popped into my head: Which<br>country has voted the best in Eurovision? That is, which country was best at<br>picking the eventual top 10 and in the right order?<br>Strangely enough, while there’s<br>plenty<br>of<br>work on voting blocs and bilateral biases at<br>Eurovision, most of it asks who votes for whom; I wanted to ask who votes<br>accurately. I couldn’t find anyone asking the question that way, so I decided<br>to do some data analysis myself.<br>The metric<br>To begin to answer this question, I first needed to formalize what “best” even<br>means. That is, some mathematical notion of “good” and “bad”.<br>The simplest measure I could come up with is “of the 10 countries you gave<br>points to, how many were in the actual top 10?”. The problem is that it can’t<br>tell apart a voter who gave the eventual winner 12 from one who gave them 1.<br>That feels wrong: an allocation is a ranking, not just a set, so position<br>ought to matter.<br>To weight position, I needed some scheme for how much position 1 matters<br>compared to position 10. Turns out Eurovision itself has already answered this<br>question: the 12, 10, 8, 7, 6, 5, 4, 3, 2, 1 it hands out is exactly such a<br>scheme. So I just reused it.<br>So I tried the next simplest thing: for each country, I have two numbers: what<br>the voter awarded it, and how many points it actually received at the end.<br>Multiply them together, sum across all countries, and divide by the maximum<br>value that sum could possibly take if the voter had ranked everything perfectly<br>(awarded 12 to the actual winner, 10 to second place, and so on). The result is<br>between 0 and 1, where 1 is a perfect vote.<br>A worked example. The actual jury top 5 in 2023, by points received, was Sweden<br>(340), Israel (177), Italy (176), Finland (150), Estonia (146). Imagine a<br>voter’s jury panel awarded (12 → Italy, 10 → Sweden, 8 → Estonia, 7 → Israel, 6<br>→ Finland).<br>Here’s the voter’s score:<br>Voter awardedCountry× Actual points=12Italy× 1762,11210Sweden× 3403,4008Estonia× 1461,1687Israel× 1771,2396Finland× 150900Sum 8,819 And here’s the perfect score for this voter, what they’d have got if they had<br>awarded the 12 to the actual top scorer, the 10 to second, and so on:<br>Voter awardedCountry× Actual points=12Sweden× 3404,08010Israel× 1771,7708Italy× 1761,4087Finland× 1501,0506Estonia× 146876Sum 9,184 So this voter’s score is 8,819 / 9,184 = 0.96 . Close to perfect: the same<br>five countries are there, just slightly out of order.<br>Is it the right metric?<br>Obviously this is not the only metric I could have used and there are many<br>other candidates out there: least-squares distance to the perfect vote, NDCG@10<br>(a standard ranking measure used in search), and Pearson correlation.<br>I went with the one above because I wanted the simplest thing which still<br>worked. Least-squares is the only “simpler” alternative I can think of, but its<br>scores end up squashed into a narrow range. The best voter across nine contests<br>only reaches around 0.42 out of a theoretical max of 1.0, which is the highest<br>score in the data but doesn’t read like a good number. The metric above sits<br>more naturally on a 0-to-1 scale: the best voters land in the 0.8s, the worst in<br>the 0.6s, and 1.0 still means a perfect vote.<br>In any case, I did run all the metrics listed above and they mostly agreed,<br>with the same countries clustered at the top and bottom regardless of which one<br>I used. But the headline #1 did shift between the top three or four countries<br>depending on which metric I picked.<br>The answer<br>I picked 2016 as the start because that’s when Eurovision split each country’s<br>contribution into separate jury and televote slates, each awarding their own 12,<br>10, 8, … points. Older contests used a single combined vote, so the data<br>doesn’t decompose the same way.<br>Across these nine contests, here’s the full table:<br>RankCountryScore1Spain0.8152Lithuania0.8103Belgium0.8044Germany0.7985Netherlands0.7976Malta0.7927Sweden0.7908Austria0.7899Iceland0.78810Norway0.78311Israel0.77812Latvia0.77713Finland0.77714Moldova0.77415Poland0.77316Romania0.77117Estonia0.77018San Marino0.77019Ireland0.76220Armenia0.76221Cyprus0.76122Denmark0.75723Albania0.75424Slovenia0.75125Australia0.74926Azerbaijan0.74627Serbia0.74528Portugal0.74329Switzerland0.73930Czechia0.73531Croatia0.73332United Kingdom0.72333France0.71234Georgia0.70735North Macedonia0.70636Greece0.69137Bulgaria0.68838Ukraine0.67539Italy0.66640Montenegro0.625Spain edges it, but only barely. The top seven are within 0.025 of each other,<br>narrower than the...