How Xbox, Big Data & Statistical Analysis Can Measure Public Opinion
Could the Xbox gaming platform and Big Data hold the key to generating accurate measures of public opinion, such as election polling? A team of statistical scientists think so.
By Jeffrey Myers, American Statistical Association.
In an experiment, the statistical scientists showed that with proper statistical adjustment, non-representative polling, such as those conducted through the Xbox gaming community and other online surveys, can predict elections and measure public opinion on a broad range of social, economic and cultural issues. The experiment’s result is also a major advance in the Big Data era in which the amount of data is overwhelming, but drawing insightful conclusions from the diverse and non-random data sources continues to be challenging.
The weaknesses of non-representative or “convenience” sampling were dramatically exposed after a mail-in survey conducted in 1936 by Literary Digest incorrectly predicted a landslide victory for Republican Alf Landon over President Franklin Roosevelt in that year’s presidential election.
Since then, election forecasts and other public-opinion polls traditionally have been based on representative polls, in which randomly sampled individuals are asked for whom they intend to vote or their opinion on an issue. However, the recent election defeat of Rep. Eric Cantor helped reveal the challenges of achieving representative sampling in public-opinion polling, primary of which are high nonresponse rates and high cost. Low response rates likely mean the people who participate in a phone survey might not be representative of the targeted population, such as registered or likely voters, which could compromise the survey’s findings. As a result, the statistical benefits of representative sampling have diminished.
In their experiment, the statistical scientists created election forecasts from a novel and highly non-representative survey dataset: a series of 45 daily voter-intention polls for the 2012 presidential election conducted on the Xbox gaming platform. After applying proven statistical techniques to the Xbox dataset, the team obtained election estimates consistent with forecasts made by leading poll analysts, which were based on aggregating hundreds of traditional polls conducted during the election cycle.
These findings combined with significant cost-savings in collecting survey data via today’s new online technologies, such as the Xbox gaming platform and web-based surveys, could make non-representative sampling the new norm.
The research was conducted by Andrew Gelman and Wei Wang of the Columbia University Department of Statistics and Sharad Goel and David Rothschild of Microsoft Research. Click here to read the research paper.
Jeffrey Myers is Public Relations Coordinator for American Statistical Association.
Related:
In an experiment, the statistical scientists showed that with proper statistical adjustment, non-representative polling, such as those conducted through the Xbox gaming community and other online surveys, can predict elections and measure public opinion on a broad range of social, economic and cultural issues. The experiment’s result is also a major advance in the Big Data era in which the amount of data is overwhelming, but drawing insightful conclusions from the diverse and non-random data sources continues to be challenging.The weaknesses of non-representative or “convenience” sampling were dramatically exposed after a mail-in survey conducted in 1936 by Literary Digest incorrectly predicted a landslide victory for Republican Alf Landon over President Franklin Roosevelt in that year’s presidential election.
Since then, election forecasts and other public-opinion polls traditionally have been based on representative polls, in which randomly sampled individuals are asked for whom they intend to vote or their opinion on an issue. However, the recent election defeat of Rep. Eric Cantor helped reveal the challenges of achieving representative sampling in public-opinion polling, primary of which are high nonresponse rates and high cost. Low response rates likely mean the people who participate in a phone survey might not be representative of the targeted population, such as registered or likely voters, which could compromise the survey’s findings. As a result, the statistical benefits of representative sampling have diminished.
In their experiment, the statistical scientists created election forecasts from a novel and highly non-representative survey dataset: a series of 45 daily voter-intention polls for the 2012 presidential election conducted on the Xbox gaming platform. After applying proven statistical techniques to the Xbox dataset, the team obtained election estimates consistent with forecasts made by leading poll analysts, which were based on aggregating hundreds of traditional polls conducted during the election cycle.These findings combined with significant cost-savings in collecting survey data via today’s new online technologies, such as the Xbox gaming platform and web-based surveys, could make non-representative sampling the new norm.
The research was conducted by Andrew Gelman and Wei Wang of the Columbia University Department of Statistics and Sharad Goel and David Rothschild of Microsoft Research. Click here to read the research paper.
Jeffrey Myers is Public Relations Coordinator for American Statistical Association.
Related: