Pollsters looking for the opinion of the American public obviously can’t survey everyone. But by surveying a smaller, random sample of the population they estimate the views of the whole population.
The larger the sample, the more accurate the estimate is likely to be, but the results of every survey also include a second estimate as to the accuracy of the results.
Margin Of Error
Along with the primary result of the survey, you often hear what is called “the margin of error.”
Imagine the following announcement: “Gallup poll finds seventy-five percent of Americans, plus or minus five percent, prefer hamburgers to hot dogs.” Headlines often ignore the “plus or minus five percent,” but that margin of error tells you how exact the results are.
In this case, we learn that somewhere between seventy and eighty percent of Americans prefer hamburgers.
How confident can the surveyor be about an estimated margin of error? To measure this, surveys also have what is called a “confidence level”, which is the likelihood that the estimate between seventy and eighty percent in this case is really accurate.
Most surveys have a ninety five percent confidence level. In other words, there’s a five percent chance — or a one in twenty possibility — that the overall public opinion doesn’t even fall within the estimated margin of error.
For Future Reference
Confidence levels and margins of error are based on the size of the sample in comparison to the size of the whole population. They don’t take into account the possibility of inaccuracies in the survey itself.
So, next time you survey results, look for the margin of error. And remember there’s still at least a one-in-twenty chance that the real public opinion doesn’t even lie within that margin of error.