INFORMATION overload is a headache for individuals and a huge challenge for businesses. Companies are swimming, if not drowning, in wave after wave of data — from increasingly sophisticated computer tracking of shipments, sales, suppliers and customers, as well as e-mail, Web traffic and social-network comments. These Internet-era technologies, by one estimate, are doubling the quantity of business data every 1.2 years.

Yet the data explosion is also an enormous opportunity. In a modern economy, information should be the prime asset — the raw material of new products and services, smarter decisions, competitive advantage for companies, and greater growth and productivity. Is there any real evidence of a ‘data payoff’ across the corporate world? It has taken a while, but new research led by Mr Erik Brynjolfsson, an economist at the Sloan School of Management at the Massachusetts Institute of Technology, suggests that the beginnings are now visible.

Mr Brynjolfsson and his colleagues, Mr Lorin Hitt, a professor at the Wharton School of the University of Pennsylvania, and Mr Heekyung Kim, a graduate student at MIT, studied 179 large companies. Those that adopted ‘data-driven decision making’ achieved productivity that was five to six per cent higher than could be explained by other factors, including how much the companies invested in technology, the researchers said.

In the study, based on a survey and follow-up interviews, data-driven decision making was defined not only by collecting data, but also by how it is used — or not — in making crucial decisions, like whether to create a new product or service. The central distinction, according to Mr Brynjolfsson, is between decisions based mainly on ‘data and analysis’ and on the traditional management arts of ‘experience and intuition.’ A five per cent increase in output and productivity, he says, is significant enough to separate winners from losers in most industries. The companies that are guided by data analysis, Mr Brynjolfsson says, are ‘harbingers of a trend in how managers make decisions.’ “And it has huge implications for competitiveness and growth,” he adds.

The research is not yet published, but it was presented at an academic conference this month. The conclusion that companies that rely heavily on data analysis are likely to outperform others is not new. Notably, Mr Thomas H. Davenport, a professor of information technology and management at Babson College, has made that point, and his most recent book, with Ms Jeanne G. Harris and Robert Morison, is ‘Analytics at Work — Smarter Decisions, Better Results’ (Harvard Business Press, 2010).
And companies like Google, whose search and advertising business is based on exploiting and organizing online information, are testimony to the power of intelligent data sifting.

But the new research appears to be broader and to apply economic measurement to the impact of data-led decision making in a way not done before. “To the best of our knowledge,” Mr Brynjolfsson says, “this is the first quantitative evidence of the anecdotes we’re been hearing about.” Mr Brynjolfsson emphasises that the spread of such decision making is just getting started, even though the data surge began at least a decade ago. That pattern is familiar in history. The productivity payoff from a new technology comes only when people adopt new management skills and new ways of working.