Statistics don't lie, but they sure can be misused by Mike Woloch Seventy-three percent of all statistics are made up, including this one.
And how do you know when they're not?
You can't get away from statistics of one kind or another in everything from news articles to advertising. Anyone can lend an air of greater legitimacy to their argument by quoting a few vague numbers. Magically, the weight of what the numbers represent is supposed to be convincing. But when we don't know where the numbers came from, how they came about, or even what the exact numbers are, do we feel more convinced? Should we?
For instance, for this column, I took a random poll and found that 50 per cent of respondents don't like hockey. That same 50 percent of respondents were all bald men. Should you draw a conclusion here? Or would you rather wait until I told you that I interviewed all of two people.
Often, great care is taken to ensure that important or interesting statistics are presented well. The typical report on an opinion poll on politics, for instance, includes information on when it was collected, how many people were surveyed, and even a figure on the statistical accuracy of the survey. The last number is meaningless to anyone with a background in statistics, but the presence of this reference lends weight to the numbers. But too often such vital information is not provided. It bothered me greatly to see a letter to the editor in this newspaper that said, "The majority of single mothers and their children live below the line of poverty in Canada."
For a statement that was supposed to catch our attention and support the argument, I found it to be so vague that its use undermined the article itself.
I can only assume that the statement was based on a current and correct statistic. But the weight we give the statement relies heavily upon the exact percentage of women living below the poverty line, the context of the statistic, where it came from, who did the research, when it was done, etc.
For instance, exactly how much is "the majority"? Recall the last Quebec referendum, and note that 50.5 per cent was a majority, in the same way that 99 per cent is also a majority. A world of difference exists between these values.
The complete statement I would have preferred to see is "According to some source, in 1997, X per cent of single mothers..." While a definition of "line of poverty" would be nice in a longer article, it's not necessary because its context in the media is well known.
So many people are suspicious of the casual use of statistics to support statements, from exaggerated advertising claims to blatant misrepresentations of the truth. Ever since Mark Twain wrote that there are "Lies, damned lies, and statistics", we've known that numbers can be manipulated, and carefully doctored statistics have been used to support everything from poor fiscal policy to racism.
In my work as a consulting engineer, I rely upon statistics and their proper use in every design I do. Standards and recommended practices are based upon meticulous research, unbiased testing, careful recording, and the application of statistical methods to reduce a mass of data to a useful, appreciable figure. It's the good reputation of various groups and organizations that gives credibility to the numbers, and I cannot use these statistics without giving the necessary background information every single time.
The use of statistics is an intimate part of everything from medical research to the quality control processes that ensure that everything from cars to light bulbs to toilet paper are made as they were intended.
But for all of their benefit and necessity, it only takes a few careless statements to undermine the credibility of all statistics, if only because of a little vagueness - a reluctance to take the time to find the reference or write down the long and often boring details.
I may not have the numbers on hand, or the time to look them up and verify them, but a cited reference always looks better, more credible, more professional.
Conversely, I can't easily agree with someone who uses statistics unprofessionally, no matter how much everything else makes sense.
© 1998, Mike Woloch
Go back to Mike's Community Editorial Board Page