People who want to know the future have many tools. There are crystal balls, fortune cookies, horoscopes, and just a bit more credible, public opinion polls. The 2020 election featured a spectacular failure of the polling industry.
RealClearPolitics catalogues all major (non-candidate) polls. They also average the results of all recent polls, which effectively increases the sample size and reduces the margin of error. Their national polling average was Biden +7.2, while the popular vote was Biden +4.5, an error of 2.7%. The results in many state polls were even worse. In Ohio, the error favored Biden by 7.2%. In Iowa, it was 6.2%. In Wisconsin, it was 6%. Overall, Trump outperformed the polls in 33 states. Curiously, Biden did overperform the polls in a few states, including Minnesota.
A review of how polling works is in order. It is neither possible nor practical to ask everyone in a population their opinion. Thus a pollster seeks a sample of the population and tries to infer the views of the whole population based on the sample. For this to work, the sample should be randomly selected, that is, every possible sample should be equally likely to be picked. A randomly selected sample is unlikely to match the population exactly, but there are well-understood mathematical laws that describe how far from correct the results are likely to be. This kind of error is known as sampling error--error caused by a sample not matching a population. The margin of error is the margin on either side of the estimate that we can be 95% confident contains the true value.
In practice, however, the sample a pollster obtains is not random. Nobody can be forced to participate in a poll, and if even one person declines, the sample is not random. It used to be the case that most people answered the phone and talked to pollsters. But over time, response rates declined due to telemarketers, robocalls, answering machines, caller ID, and cell phones. Now response rates typically range between 1% and 5% of people called. The sample a pollster obtains is usually wildly unrepresentative of the population.
How do pollsters deal with this? They ask respondents various demographic information (race, sex, political party, education). Then they weight the results so they match the presumed demographic breakdown of the electorate. But they don't actually know this breakdown. Pollsters make an educated guess based on demographics of past elections (which can be estimated, but not known exactly) and their beliefs about what the electorate will look like.
Essentially this makes the poll itself an educated guess. Educated guesses are often close to accurate, and they are more accurate than the sort of wishful thinking that predominates among political ideologues. But educated guesses can be wrong, sometimes wildly so.
This cycle, it appears that many Trump supporters didn't answer their phones, or refused to participate in polls. This skewed the samples, even with the adjustments that pollsters made. But why did Trump supporters refuse to talk to pollsters? Some may simply hate the media. Others may be concerned about admitting their views, even in a supposedly anonymous poll.
This phenomenon is known as social desirability bias. One previous example of this is the Bradley effect, in which voters were supposedly more likely to say they would vote for a black candidate than to actually do so. This effect remains controversial, however.
Researchers try to account for social desirability bias in various ways. One way is to ask people what their friends or neighbors think about the election. This is the methodology used by Trafalgar, a pollster who found much better results for President Trump than other pollsters. The problem of social desirability bias applies to issue polls, as well.
Polls can be useful when appropriate precautions are taken, but other indicators of public sentiment should not be ignored.
No comments:
Post a Comment