Few would argue that it hasn’t been damaged by a number of factors in recent years. With that in mind, I read a May speech by Scott Keeter, director of survey research for the Pew Research Center, delivered in his capacity as president of the American Association for Public Opinion Research.
The foundation of statistically sound survey research is that every member of a population has an equal chance of being included. Yet survey professionals have gone from expecting 30 to 50 percent response rates to single digits. So whose voice might be missing and can surveys still produce representative results?
Keeter explores the importance of other emerging methodologies. “Whether we are talking about opt-in internet panels, which have been around for a while, or the non-survey methods such as automated content coding of social media, the integration of data from what has been called the ‘internet of things,’ and from so-called big data more generally, these have drawn interest and resources away from traditional surveys. “
Researchers have never had so much access to so much information about opinions, attitudes and behaviors as they do today – data that can be linked to surveys for richer interpretations and more robust findings. However, such social data has not received the level of scrutiny that has long characterized survey research.
If we must edge away from the probability model to explore the new frontier, we are right to worry about whether our findings are representative, valid and reliable. Bad data can drive out good, especially when people don’t distinguish between fact-checked reporting and the on-the-fly opinion postings.
“Information not only informs policy making, but serves as a political weapon,” says Keeter. “Perhaps it always has, but I have a sense that bad information, whether it’s junk science, economics or polling data, is now more widespread.”
Frank Ovaitt is President and CEO of the Institute for Public Relations.