Sunday, 22 May 2016

Why that Express splash about '12m Turks' is so mathematically illiterate

"12M TURKS SAY THEY'LL COME TO THE UK," shouted the Sunday Express today. The story was based, they said, on an exclusive opinion poll carried out in Turkey, asking people whether they would want to come to the UK if we stay in the EU, and Turkey joins.

My first thought was: well, that sounds like kind of a lot. Twelve million. The entire population of London and Greater Manchester combined.

My second thought was: I hope they have included a link to the full opinion poll so I can judge the figures for myself.

They did. It is here. And it proves the 12 million figure is utterly unjustified, and the Express has (presumably unwittingly) made some dreadful errors with the statistics.

The 12 million figure is based on a simple calculation. Take the percentage of people who answered 'yes' to the pollsters' question about coming to the UK (15.8%). Apply that to the total population of Turkey (80 million or so). Hey presto, you get about 12 million.

And that's fine.

But let's look at the question that was asked:


Notice the word 'consider'. Not 'would you intend to move', or 'would you probably move', but 'would you consider'. Personally I consider doing a lot of things I'm unlikely to end up doing. I've considered moving to New York. I've considered going vegetarian. When I was buying a car, I considered very many models. 

I only ended up buying one.

But the real problem is that other bit: 'or any member of your family'.

This skews the figures completely and renders the 12 million claim utterly null and void.

To understand the point, imagine if Turkey had a population of just 100, neatly divided into 10 separate families.

Now imagine that each of those families had just ONE member who would consider moving to the UK.

It doesn't take a maths genius to realise that the number of people who would consider moving to the UK is 10, and that these 10 make up 10% of the entire population.

But what if we applied Express logic to this group of people?

Let's imagine pollsters choose the first person in each row and ask them the question: "Would you, or any member of your family, consider moving to Britain?"

Now, NONE of the people polled would themselves consider moving to the UK. But ALL of them have a family member who would.

So they would all have to answer 'yes' to the question as put.

We would end up with 10 out of 10 people, 100%, saying 'yes'.

In fact, whichever one of the 100 people you polled, they would all have to say 'yes'. Even though 90% have no intention of even considering moving to the UK.

Applying Express logic, and generalising these figures to the population as a whole, we'd end up with a headline saying that EVERY SINGLE PERSON IN TURKEY was going to come to the UK. And yet clearly only 10 out of 100 were even considering it.

Look at it this way: I imagine as near as dammit 100% of Turks have at least one family member who has considered NOT moving to the UK if Turkey enters the EU. Maybe they are considering moving somewhere else; maybe they are at least considering staying in Turkey.

You wouldn't therefore try to construct a splash saying "ABSOLUTELY NO TURKS WILL ENTER THE UK IF WE MAKE THEM PART OF EUROPE".

That would be equally stupid.

It is, in short, a terrible way to phrase the question if your honest intent is to find out the number of people who are genuinely likely move to the UK in the event of Turkey joining the EU.

(A final note: quite a lot of people on Twitter are criticising the poll for 'only' asking 2,600 people for their opinion. Well, it's a poll. That's how polls work. It's a decent sample size with a tolerable margin of error. There are many things wrong with the poll, but that isn't one of them.)