KDnuggets Home » Polls » What was the largest dataset you analyzed / data mined? (May 2012)

What was the largest dataset you analyzed / data mined?


 
  
What was the largest database / dataset you analyzed? [347 votes]
Largest dataset analyzed in 2012 vs 2011

We note that more than twice as many analysts participated in 2012 poll (347 people) than in a similar 2011 Poll: Largest Dataset analyzed/ data mined, (148 people).

Among significant differences, we observe that the peak of the distribution shifted from 1.1-10 GB in 2011 to 11-100 GB in 2012. A smaller percentage have reported the largest dataset in 1.1-10 GB range in 2012 (16.4%) as compared to 2011 (19.7%), and a larger percentage have reported the largest dataset of 11-100 GB range (21.6%) as compared to 2011 (17.7%).
Note: In 2011 the maximum range given in the poll was 1 PB and over, so no direct comparison for Petabyte range is possible between 2011 and 2012 results.

The median answer in 2012 can be estimated to be in 20-40 GB range, compared to 10-20 GB range median answer in 2011.

The percentage of analysts with experience in the upper range of datasets (over 100 GB) has remained around 35% - same as in 2011, but this could be due to larger participation in 2012 poll. In 2010, about 32% of respondents worked with 100GB and larger DB.

Regional breakdown (below) shows that almost every region now has a median in 11-100 GB range.

Region (voters)Largest Dataset Analyzed (median)% analyzed TB+ data
US/Canada (154) 11-100 GB  24.0%
Europe (106) 11-100 GB  18.4%
Asia (46) 11-100 GB  17.4%
Latin America (21) 11-100 GB  23.8%
AU/New Zealand (10) 11-100 GB  20.0%
Africa/Middle East (10) 1-10 GB  20.0%

KDnuggets Home » Polls » What was the largest dataset you analyzed / data mined? (May 2012)