Lies, damned lies, and Brexit statistics

https://www.theguardian.com/news/datablog/2016/may/16/lies-damned-lies-and-brexit-statistics

Version 0 of 1.

Both sides of the EU referendum campaign are bombarding us with charts, facts and numbers, intending to convince us of their arguments. But we all know the famous adage “Well, you can prove anything with facts.”

What are we to make of claims about the impact on the economy of leaving the EU, or the figures about how much we spend on membership, and where else it could be spent? Suggestions that future trade deals away from the EU will be a better deal for the UK, or worse than what we have now?

Looking at a rather frivilous claim last week gives us some insight into just how easy it is to twist even a small set of figures to make the case for and against Brexit.

On Friday the Leave.EU campaign sent a tweet saying “Watching #Eurovision on Saturday? More countries outside the EU have won than those inside #LoveEurope”

It’s hardly the most important issue at stake, but I couldn’t believe it could possibly be true by any measure, so I decided to investigate. Here are the different ways I found to prove it was wrong. And to also prove that at the same time, it was indeed totally correct.

Have more countries from outside the EU won the Eurovision Song Contest than countries within the EU?

No.

16 current member states have won Eurovision: Austria, Belgium, Denmark, Estonia, Finland, France, Germany, Greece, Ireland, Italy, Latvia, Luxembourg, Netherlands, Spain, Sweden and the United Kingdom have all had at least one night of Eurovision glory.

Ten countries who have never been in the EU have been crowned Eurovision champions: Azerbaijan, Israel, Monaco, Norway, Russia, Serbia, Switzerland, Turkey, Ukraine and Yugoslavia. And one of those doesn’t exist anymore.

What about number of victories by each country? Maybe non-EU countries win Eurovision more frequently?

No.

Countries who are currently EU member states have won the contest a total of 48 times since it started. Countries who have never been members of the EU have only won it sixteen times.

But surely it makes a difference if people were actually members of the EU when they won it?

Good point. Not every country that is currently an EU member state won the contest while being a member of the EU.

The UK won twice in the sixties before joining, and Baltic states Estonia and Latvia both won in the early 2000s before becoming EU members in 2004. The Netherlands even won the contest in 1957 before the European Economic Community came into being.

There are 13 occasions when a country that is now in the EU won the contest before it joined the European Union. This makes the total score a lot closer. 35 wins to 29 wins.

But still in favour of the European Union members winning Eurovision.

Is this all about 1969?

For reasons I can’t fathom, in 1969 the Eurovision contest result was a four-way tie, and they didn’t have a penalty shoot-out to decide which of Spain, France, the Netherlands and the UK’s Lulu were truly the winners. The title was split four ways. Boom-bang-a-bang indeed.

However, reducing the 1969 victory for those four countries to only represent 0.25 of a victory each still doesn’t quite shift the balance in favour of non-EU countries.

But it hasn’t always been called the European Union, has it?

Maybe this is the key. The European Union came into being as a result of the Maastricht Treaty and started to exist on 1 November 1993. So to see if the Leave.EU tweet is correct we should maybe discount all contests before 1994.

This still doesn’t do it though. Since 1994 Eurovision has been won twelve times by an EU member state. And, including the 2016 victory by Ukraine, it has been won ten times by a non-EU member state.

OK, but since 1994, how many different countries have won it?

Finally a set of numbers that validate the tweet.

Since 1994, Austria, Denmark, Finland, Germany, Greece, Ireland, Sweden and the United Kingdom have won the Eurovision song contest while being members of the European Union.

And since 1994, Azerbaijan, Israel, Norway, Russia, Serbia, Turkey and Ukraine have won the Eurovision Contest having never been members of the EU.

That makes it 8-7 in favour of being a member.

However, you’ll recall I mentioned Estonia and Latvia earlier? Who both won before joining the EU? If you add them into this tally, their victories swing it for the Brexit camp.

But that wasn’t even the methodology that Leave.EU were quoting

No. It is more complicated than that. They were linking through to a City AM piece by Brian Monteith, and his article states “For it is a fact that more countries outside the EU (18) have won Eurovision than countries that are EU members (13)”

To get those numbers, Austria, Denmark, Ireland, Sweden and the UK count for each side, as they have won the contest before they joined and then again as an EU member state. Latvia, Estonia and Spain count as non-EU because their victories all came before they joined the Union. And then there are the ten countries who have won but who have never been part of the EU.

At last, I have found the chart where non-EU success at Eurovision looks significantly bigger than EU success.

What does this tell us about the use of statistics in the EU referendum?

None of the charts in this article are wildly inaccurate. It is only Eurovision, and definitely a bad case of correlation ≠ causation, but I’ve just outlined seven different ways to examine the same objective set of data, which you could then use to make the point you wanted to about membership of the EU.

And this tactic is widespread in the EU referendum campaign.

There will be a lot more data-backed claims during the course of the referendum campaign. Many of them rather more serious than whether membership of the European Union helps or hinders winning Eurovision.

But those Eurovision statistics are a warning.

We should be judging statistical claims on both sides, not just by the numbers, but by thinking carefully about who is making the claim, and why they’ve chosen to interpret those statistics in that particular way.

Note: The Leave.EU tweet was made on Friday afternoon, before the 2016 contest was won by Ukraine. I have opted to include Ukraine’s second victory in the figures above, as it made no material difference to the conclusions.