A rare insight into Amazon’s experimental culture
In our talk about why some of your projects take ten times too long, we explained the importance of speed.
If a company can change quickly, it can win. If it can’t, it dies.
All the conversion know-how in the world is worth nothing if a company is mired in bureaucracy. World-changing ideas aren’t enough; you need actually-changing pixels. If you wonder why we keep writing about how to make things happen, it’s because it’s the most effective skill in conversion.
One of the amazing benefits of A/B-testing is that it allows you to postpone decisions until after the idea has gone live. By which time, your decision-making is no longer needed. You can just measure whether it worked. The whole “pontification stage” of the project—beloved by many traditional marketers—goes out the window.
(Of course, it helps to have great strategy, amazing ideas and deep intuition in the first place.)
No company understands this better than Amazon. Amazon’s staff members aren’t mired in big-company bureaucracy. In fact, their CEO, Jeff Bezos, in his letter to shareholders, speaks eloquently about it. That’s how deep experimentation runs in Amazon’s culture.
We urge you to read the following quote from the report (emphasis ours):
One area where I think we are especially distinctive is failure. I believe we are the best place in the world to fail (we have plenty of practice!), and failure and invention are inseparable twins. To invent you have to experiment, and if you know in advance that it’s going to work, it’s not an experiment. Most large organizations embrace the idea of invention, but are not willing to suffer the string of failed experiments necessary to get there.
Outsized returns often come from betting against conventional wisdom, and conventional wisdom is usually right. Given a ten percent chance of a 100 times payoff, you should take that bet every time. But you’re still going to be wrong nine times out of ten. We all know that if you swing for the fences, you’re going to strike out a lot, but you’re also going to hit some home runs. The difference between baseball and business, however, is that baseball has a truncated outcome distribution. When you swing, no matter how well you connect with the ball, the most runs you can get is four. In business, every once in a while, when you step up to the plate, you can score 1,000 runs. This long-tailed distribution of returns is why it’s important to be bold. Big winners pay for so many experiments.
Here’s another great quote from the same report:
Some decisions are consequential and irreversible or nearly irreversible—one-way doors—and these decisions must be made methodically, carefully, slowly, with great deliberation and consultation. If you walk through and don’t like what you see on the other side, you can’t get back to where you were before. We can call these Type 1 decisions.
But most decisions aren’t like that—they are changeable, reversible—they’re two-way doors. If you’ve made a suboptimal Type 2 decision, you don’t have to live with the consequences for that long. You can reopen the door and go back through. Type 2 decisions can and should be made quickly by high-judgment individuals or small groups.
As organizations get larger, there seems to be a tendency to use the heavy-weight Type 1 decision-making process on most decisions, including many Type 2 decisions. The end result of this is slowness, unthoughtful risk aversion, failure to experiment sufficiently, and consequently diminished invention.
Does your company treat Type 2 decisions like they are irreversible? If you wanted to test a new offer, would you require three sets of signatures?
If your company’s culture is too bureaucratic, maybe you could use those quotes from Jeff Bezos as the starting point for a discussion. If your company treats Type 2 decisions like Type 1 decisions, explore why.
© 2019 Implementra Limited trading as Conversion Rate Experts. All rights reserved.