By clicking “Accept All Cookies,” you agree to the storing of cookies on your device to enhance site navigation and analyze site usage.

Skip to main content

Market Experimentation Meets Big Data

September 13, 2016


By Paula Klein

Business leaders are continuously seeking out new and improved ways to drive decisions and meet consumer needs. Many companies are conducting experiments with online users every minute every minute to glean insights, improve engagement and heighten transaction volume, for instance

In particular, digital platform companies such as Yahoo, Nike, Facebook and Google can monitor the habits and data of ten- to 100 million people at a time. At that scale—and in real-time—huge data sets can be analyzed and dissected very differently than in most of today’s R&D environments. Prof. Sinan Aral, Professor of Management at the MIT Initiative on the Digital Economy, described some of these experiments and their benefits at the 2015 CIO Symposium.

At the same time, many businesses find large-scale experimentation unattainable. At this year’s May 18 CIO Symposium, Prof. Aral discussed new ways to achieve these goals: using big data analytics to substitute for marketing experimentation in business environments. Each approach can reap rewards, he said, but since it’s not always easy, affordable or feasible to do large-scale experimentation, social analytics using big data offers good alternatives.

 “Experimentation is very robust, but can be very narrow and difficult to do,” Aral told attendees. “If you have a big digital platform, a/b testing is easy; but if you have a product that’s not digital, or it’s not legitimate to randomly test strategies or approaches on users, you can choose a big data solution

As an example, Aral cited a large fitness company that analyzed data about consumer running habits in order to create engagement and peer-to-peer demand for products and services. To measure exercise frequency, the company tracked global data on 14 million runners over five years. It also added a social network component to encourage new data input. That data then was correlated to weather data to establish patterns and to determine the influence of outside factors and peers on running activity.

In another case, The New York Times collected data from tweets and site clicks to see how word-of-mouth recommendations drive readership, and what that means to monetization and paywall design.

For the full details on these two cases, watch Prof. Aral’s presentation here.

For more on Prof. Aral’s work, view details here.