Prev
IBM Researchers Develop Energy Efficient Method to Analyze the Quality of Data at Record Speeds
Nine Terabytes of Data Validated in Less than 20 Minutes; Greater predictability with more accuracy
SEATTLE and ZURICH, - 25 Feb 2010: IBM (NYSE: IBM) Research today unveiled a breakthrough method based on a mathematical algorithm that reduces the computational complexity, costs, and energy usage for analyzing the quality of massive amounts of data by two orders of magnitude. This new method will greatly help enterprises extract and use the data more quickly and efficiently to develop more accurate and predictive models.
In a record-breaking experiment, IBM researchers used the fourth most powerful supercomputer in the world -- a Blue Gene/P system at the Forschungszentrum Julich in Germany -- to validate nine terabytes of data (nine million million or a number with 12 zeros) in less than 20 minutes, without compromising accuracy. Ordinarily, using the same system, this would take more than a day. Additionally, the process used just one percent of the energy that would typically be required.
...
"Determining how typical or how statistically relevant the data is, helps us to measure the quality of the overall analysis and reveals flaws in the model or hidden relations in the data," explains Dr. Costas Bekas of IBM Research - Zurich. "Efficient analysis of huge data sets requires the development of a new generation of mathematical techniques that target at both reducing computational complexity and at the same time allow for their efficient deployment on modern massively parallel resources."
Read more.