Experimentation in business

In order to understand what works and what doesn't in business, it's best to test and measure--to experiment--and apply what you learn. Thomas Davenport's 2009 HBR article "How To Design Smart Business Experiments" is an excellent primer on some of the challenges as well as the benefits of conducting sound experiments.
Davenport summarizes his findings as follows:
  • Too many business innovations are launched on a wing and a prayer—despite the fact that it’s now reasonable to expect truly valid tests.
  • Companies that equip managers to perform small-scale yet rigorous experiments don’t only save themselves from expensive mistakes—they also make it more likely that great ideas will see the light of day.
  • With a small investment in training, readily available software, and the right encouragement, an organization can build a “test and learn” capability.

To these, I would add the following comments:
  • At the end of the day, little else than controlled, replicated, consistent experimentation can establish causality between two observables. This challenges not only the businesses that do not a have a “test & learn” culture, but also those who attempt to measure what they do, but rely mainly on non-causal ex-post estimations. Rigorous experimentation can tell you when you are wrong, and if you are right, it is a good hygiene factor.
  • The approach of the article is very pragmatic when stating that experimentation mainly serves to test tactical changes / decisions. Based on Mars’ experience with marketing science and “laws of growth,” I would argue that replicable, rigorously established patterns can definitely serve as the foundation for the marketing strategy of a company. While the tactical improvements are the main short term outcome of a good experimentation culture and practice, it can be a critical enabler for scientifically grounded and effective strategic shifts.
  • Last but not least, the idea that "With a small investment in training, readily available software, and the right encouragement, an organization can build a “test and learn” capability” can have far reaching consequences on an organization like Mars. Mars has historically been a decentralized, locally driven, entrepreneurial company. This has arguably been a critical enabler for Mars’ success so far, especially in period of fast economic growth and distribution expansion. Now that growth is expected to be close to flat in mature markets, that population aging and nutrition concerns are effecting a slow decline in the per capita calorie consumption, we are more than ever in need of the creative power of entrepreneurship to sustain our growth. In the increasingly complex, multi-channel, heterogeneous, information-overloaded business ecosystem, it will probably have to be a very well informed entrepreneurship. The opportunity for decentralized learning that is offered by a small scale rigorous experimentation capability is a critical enabler for today’s and tomorrow’s entrepreneurship, probably as important as the freedom of action granted by decentralized decision making. 
And I would recommend this great book about the topic, which is a very easy read and takes us through the disruption from rigorous experimentation to business, manufacturing, sociology, government, education, health, etc :-)
-- Yassine El Ouarzazi

Weekly Content Briefing, December 23rd

Weekly briefing -- interesting content from many sources