Bruce Springsteen and the critical question to ask before choosing A/B testing software

Published: May 2023

In Making Websites Win—our Amazon #1 bestselling book—we set out a powerful principle:

Principle 2: The top companies carry out experiments on their websites.

The theory behind running experiments is simple enough, but the huge range and variety of modern testing tools can be a gift to procrastination. Let’s fix that step by step.

In this article, we’ll share:

The two things you need to run an experiment.
The lesson of Bruce Springsteen’s guitar.
The critical question to ask before choosing A/B testing software.

The two things you need to run an experiment

As you may know, an A/B test tells you which version of your page generates the most conversions. To run an experiment requires the following:

A Content Creator. Someone to create a new web page to test against the current version.
A Scorekeeper. Software to track the conversion rates and declare a winner (once sufficient data is collected).

The software is an extremely valuable part of the process. Without something to keep score, we’d never be sure which version of the page was the winner.

The number of testing tools has exploded in recent years, each offering a unique mix of features, price, and support. But before you dive in, it’s worth remembering the following:

The single most important thing you can do to accelerate your growth is to commit to a culture of experimentation.

Companies like Amazon and Facebook don’t try to guess what works. They relentlessly test their content daily, compound their ‘wins,’ and benefit from The Power Law of CRO.

The lesson of Bruce Springsteen’s Guitar

In his memoir, Born to Run, Bruce Springsteen describes the moment in 1973 when he first picked up his legendary guitar.

“I strapped on my new guitar, a 1950s mutt with a Telecaster body and an Esquire neck, I’d purchased at Phil Petillo’s guitar shop for one hundred and eighty five dollars… it became the guitar that I’d play for the next 40 years. It was the best deal of my life.”

To state the obvious, The Boss can’t make music without a guitar, but it isn’t the guitar that makes him The Boss. What matters is the music he plays.

Photo of Bruce Springsteen playing his 1950s mutt with a Telecaster body and an Esquire neck. — Image of a content creator setting up an experiment on a yellow testing tool.

The same principle applies to experiments. It isn’t the tools that generate wins; it’s the content that’s created.

Others agree. Roger Longbotham, renowned data scientist, and former Senior Statistician at Amazon, said…

“What you place upon the statistical framework is what ultimately determines the attractiveness of your test results.”

(And if you want authoritative advice on what to test, our free book is based on sixteen years of insight and experience.)

The simple (but critical) question

Please don’t misunderstand us: we think testing software is essential for any web business. We use (and recommend) many of these amazing tools every day.

But it’s easy to fall prey to the illusion that modern software creates winning pages out of thin air. That’s not true.

Worse still, we see smart people procrastinate as they seek a ‘perfect package’ that rarely exists. That’s why there is a critical question to ask when choosing A/B testing software:

What’s the quickest way we can start testing (today)?

Many of the solutions on the market offer free trials, monthly billing, or dedicated support. Choose one that meets your needs, and get started.

To summarize:

Commit to a culture of experimentation.
Focus on the content you will test.
Choose A/B testing software that you can start using today.

However powerful these tools have become, they are still the guitar.

You are The Boss.

How much did you like this article?

How the world’s best companies build unbeatable focus

“The results that we’ve seen have been monumental,” says DiamondBack Covers, a direct‑to‑consumer manufacturing company

What’s your goal today?

1. Hire us to grow your company

We’ve generated hundreds of millions for our clients, using our unique CRE Methodology™. To discover how we can help grow your business:

Read our case studies, client success stories, and video testimonials.
Learn about us, and our unique values, beliefs and quirks.
Visit our “Services” page to see the process by which we assess whether we’re a good fit for each other.
Schedule your FREE website strategy session with one of our renowned experts.

Schedule your FREE strategy session

2. Learn how to do conversion

Download a free copy of our Amazon #1 best-selling book, Making Websites Win, recommended by Google, Facebook, Microsoft, Moz, Econsultancy, and many more industry leaders. You’ll also be subscribed to our email newsletter and notified whenever we publish new articles or have something interesting to share.

Browse hundreds of articles, containing an amazing number of useful tools and techniques. Many readers tell us they have doubled their sales by following the advice in these articles.

Download a free copy of our best-selling book