Five signs that someone is trying to mislead you with statistics

Be on your guard with these five clues to look out for that might mean somebody is trying to lead you up the garden path with numbers. They come from Anthony Reuben’s new book Statistical – Ten easy ways to avoid being misled by numbers.

When you hear the unemployment figures each month you might think it’s a precise count of all the people in the country who do not have a job but would like one. But it isn’t – it’s based on a survey. It’s a seriously big survey, covering 40,000 households with 100,000 individuals every three months, but it’s still a survey, which means there’s a margin of error.

For example, the figures announced in May were that unemployment had fallen by 65,000, give or take 74,000. As that second number is bigger than the first one, we can’t say for sure that unemployment has fallen at all, which means that the result is described as not being statistically significant. And statistically significant changes in unemployment are pretty unusual – we haven’t had one since 2015. So be a bit wary next time you hear that unemployment has fallen or risen.

Read more:

• Five weird facts about maths
• Can maths defeat terrorism?

We’re always hearing that something is costing the economy billions of pounds, whether it’s snow, flooding, people not getting enough sleep or employees pretending to be ill so they can stay at home and watch a sporting event. It’s almost always nonsense and let’s use the example of snow to explain why.

When we talk about things being good or bad for the economy we are referring to their impact on gross domestic product (GDP), which is a measure of the total amount produced in the economy. Some things about heavy snow are definitely bad for the economy, for example, if you are running a car factory and you have to shut down production because your staff or parts can’t reach you then you will lose money. On the other hand, if councils are having to buy grit and pay people to spread it on the roads then that’s a boost to GDP.

So this isn’t a simple calculation, and it’s definitely not one that can be made the day after the snow storm.

“Up to” is a weasel expression that means that the figure you are being given is the maximum possible, but it’s often used in situations in which that is not the number you are looking for.

So when you are walking down the High Street and you see a sign saying “Always up to 60 per cent off”, it is promising you that nothing will ever be discounted by more than 60 per cent. It would not need to reduce any prices at all to be able to make that promise. You’ll notice that such signs always have the 60 per cent in about 500 point and the “up to” in about eight point text.

This isn’t just a problem for sales shopping. Recent election manifestos have promised “up to 10,000 more mental health professionals” and protection for “up to one million acres of accessible green space”. Neither of those should be difficult promises to keep, because the parties involved would not be breaking them unless they employed more than 10,000 mental health staff or protected more than a million acres.

Listen to the Science Focus Podcast:

• What happens when maths goes horribly, horribly wrong? – Matt Parker
• What's the deal with algorithms? – Hannah Fry

A good sign that there are dodgy statistics being used is when the figures seem too precise. For example, I saw a sign on the Tube for a dating website that claimed it had led to 144,000 Britons being in relationships, adding “that’s 2,208 Tube carriages of people”. Given that 144,000 was clearly a rounded number I wondered why they had gone for 2,208, given that would be 65.2 people in each carriage.

Once you start looking out for them you’ll see them all over the place. There was a report that suggested bots on social media had boosted Donald Trump’s election results by 3.23 percentage points. The mechanism between content being shared on Twitter and people altering their votes as a result seems dubious at best, but suggesting it could be accurate to within 0.01 percentage points is definitely stretching the possible accuracy of the claim.

The Simpsons took advantage of the way that oddly specific numbers sound wrong when they had their “138^th Episode Spectacular”.

Beware of situations in which you are only given the percentage change for a figure and not the absolute number – what I call “lonely percentages”.

A good example of this is the sale of vinyl records. Vinyl sales peaked in the 1970s when shipments reached about 90 million a year. By 1994 sales were down to 1.5 million a year, reaching 200,000 in 2007. But then the recovery began, with sales reaching 4.1 million in 2017. Now you could say that vinyl sales had fallen by about 99% and then risen by 1900%, which would make it sound like we had reached peak vinyl, but when you look at the actual figures you will see that would be misleading.

When you’re only being given percentage figures it should be setting off alarm bells, whether they’re record sales, trade data or crime statistics. And remember that percentage changes always sound bigger on the way up than on the way down, because nothing can fall by more than 100%.

Follow Science Focus onTwitter,Facebook, Instagramand Flipboard