By clicking “Accept All Cookies,” you agree to the storing of cookies on your device to enhance site navigation and analyze site usage.

Skip to main content

The World is Big Data’s Laboratory

June 19, 2015

Big data is beneficial, or course, but “the radical innovation is in big experimentation; the unprecedented ability to experiment with social systems at scale,” according to Sinan Aral (pictured), who heads the Social Analytics and Large Scale Experimentation research programs at the MIT IDE. And he is among those leading the charge into this new field of study.

At a May 20 CIO Symposium panel on The Future (and Potential) of Large-Scale Digital Experiments, Aral explained how large-scale experiments can make big data’s impact even more significant than it is today. For example, giant companies such as Yahoo, Nike, Facebook and Google can monitor the habits and data of ten- to 100 million people at a time. At that scale—and in real-time—huge data sets can be analyzed and dissected very differently than in most of today’s R&D environments. As a result,

We are “on the brink of a revolution in our understanding of human behavior” that will help businesses and society better determine the benefits, opportunities and challenges of online transactions, according to Aral.

Professor Karim Lakhani, of Harvard Business School, said that if data scientists are not doing these types of experiments, they are “probably not getting correct data analysis.” Worse, businesses are “leaving money on the table” and missing opportunities that can be found in the data, he said. Lakhani encouraged anyone involved in data analytics to “move beyond lab models” to investigate crowdsourcing and how innovation is often spurred in teams and competitions.

Aral also raised the ethical issues of such deep data monitoring and analysis. He noted that some people see this Big Brother-type of oversight “kind of creepy and a 1984-ish invasion of privacy.”

Lakhani, noted that the field of experimentation is still very new and “we don’t have norms yet.” Financial regulation and penalties for misuse are helpful, but you have to “penalize correctly. Norms are evolving as we amass more data.”

For more on Aral’s work see these blogs. He is also an organizer of the MIT IDE code@mit, the Conference on Digital Experimentation, set for October 16, 2015.