This article provides a detailed account of A/B Testing in marketing. It provides more information, detail, and context to the article CHAPTER 1: Conversion Rate Optimization Secrets, one of four articles that lay out the Frictionless Commerce Conversion Rate Optimization philosophy.

If you care at all about optimizing your site, you must understand the role of A/B testing in marketing. If you don’t, you’re lighting money on fire.

But where do you start? How do you decide what is important? What is worthwhile testing? And what is just busy work that won’t make a speck of difference in the long run?

We wholeheartedly believe in the power of A/B testing in marketing. It is one of the core principles here at Frictionless Commerce. Of course, we believe that how and where you perform these tests plays a large role in your testing success.

So, if you’re a marketer, especially a marketer who believes in the power of storytelling, you will love this guide.

A Simple, Clear, Easy-To-Remember Definition of A/B Testing in Marketing

There are heaps of definitions of what A/B Testing in marketing is. If we’re not talking about the same thing, this article won’t help you. 

In my experience, the best definitions are simple, clear, and easy to remember. Here’s my definition of A/B Testing in marketing:

A/B Testing is the technique of comparing two versions of a web page to see which one performs better.

That’s it.

You can make it more complicated, and probably a tiny bit more accurate, but that’s A/B testing in a nutshell.

Let’s move on to the good stuff.

Why Marketers Need to Care About A/B Testing

If you’re in marketing, A/B testing is part of the job description. But have you stopped to think why it’s more important now than ever? Have you, truly, deeply thought through what is happening in the eCommerce customer acquisition ecosystem?

I’ve run thousands of A/B tests on hundreds of pages over the last twelve years. Let me see if I can explain this highly technical subject in a few paragraphs.

The State of Digital Ad Spend

The increase in digital advertising spending has been astounding. 2021 was a record year for ad spending, with more growth expected in 2022.

Digital ad spending in the U.S. alone is at $284.3 billion.

 And it’s only going up.

With such massive numbers the marketer who can get the highest ad traffic conversion rate will be the top dog. And to do that you need to A/B test like crazy.

It Ain’t Getting Cheaper

The COVID-19 pandemic threw a giant wrench into everyone’s plan at the start of 2020. Marketing and advertising were no exceptions. That’s why, when you look at Cost Per Click benchmarks over the last two years, you might get a little cross-eyed.

Despite the drastic drop in spending and cost at the beginning of the pandemic, you see costs creeping back up by March 2021:

Even with the drop-off of 2020, the trend is clear: it’s getting more expensive to drive traffic to your site.

Apple Is Calling the Shots

Complicating matters a little lot is Apple’s iOS 14 update.

With the update to iOS 14, Apple has restricted the marketing tracking on their ecosystem. For context, there were 1.8 billion active devices as of January 2022.

And then there’s the fact that iOS users are more affluent. The median iPhone app user earns $85,000 per year, which is 40 percent more than the median Android phone user with an annual income of $61,000. That makes iOS users super attractive to marketers.

What Does iOS 14 Mean for Marketers?

Apple has made it very hard for marketers to use standard marketing tactics on iPhone users.

Retargeting, showing a personalized ad to a past site visitor, no longer works. With iOS 14, marketers can no longer track past visitors. And if you can’t track them, you can’t send them retargeting ads.

That’s a problem because marketers know that returning visitors have a higher conversion rate. So retargeting tactics have an excellent return. Now Apple is limiting that feature.

There are also limitations on the number of site behavior triggers the marketer can track. Once again, fewer data to use.

To dig deeper and fully understand iOS implications, read this excellent article.

Marketers can no longer rely on tracking the future behavior of the shopper. This means the only clear conversion signals available are sales that happen on the first visit generated by the ad click. 

The marketer needs to convert the visitor on the first visit.

How?

As it turns out, this is really, really hard. But not impossible.

Your best bet is to have a deep understanding of buyer psychology. But even then, you can’t just rely on guessing. That isn’t a problem because you have a tool at your disposal to guide you: A/B Testing.

The thing is…

Most Marketers Aren’t Using A/B Testing Correctly

Think about how most marketers do A/B testing. 

If a marketer wanted to optimize the page for the Dyson Supersonic hair dryer, they would typically consider this sort of A/B testing:

Or maybe testing these elements:

I’m not saying these aren’t valid A/B tests, but I am suggesting other tests are more likely to get you your year-end raise.

Why listen to me for A/B testing in marketing ideas? My name is Rishi Rawat, I’m a marketer, and I have spent the last 13 years breaking my head, learning why online shoppers behave the way they do. I know a thing or two because I’ve A/B tested a thing or two.

What Do Most Marketers Focus On?

So, what’s the pattern here? Give most marketers a page to optimize and they’re going to focus on the design.

There are many reasons not to focus on design-based A/B testing. The biggest reason is this: design-based tests get a lot more scrutiny from leadership teams. 

Leadership teams feel they need their fingerprints on design decisions. This slows down the speed of A/B testing, and speed is one of the biggest advantages of A/B testing in marketing.

Design tests are also more expensive and more complicated to develop. 

What About Video?

You may be wondering, “Ok, but what about video? Isn’t video incredibly impactful?” If that’s a question floating in your head, read this post: Words Kick Video’s Ass.

What Should You Focus On?

You’re constructing your pages to convert. To sell. As an eCommerce business, you are looking to convert those first-time visitors into buyers. Conversions are what matters. How do you impact conversion?

Unbounce analyzed 40,000 landing pages. They found that the conversion impact of copy is twice that of design. (source).

Words kick video’s ass. And it’s not even close.

Focus on your sales pitch. Think about the actual words used on the page. 

Focus on your copy.

Conversion Copywriting

Conversion copywriting is the practice of using words to influence shoppers to make a purchase. Don’t make the mistake of thinking copywriting is the domain of people with a degree in English. Knowledge of language has little to do with it. I, for example, am an engineer.

I look at conversion copywriting as a sales function. The marketer uses conversion copywriting to construct a sales pitch designed to get the buyer from interest to purchase.

Once we switched from the throw-spaghetti-at-the-wall-and-see-what-sticks conversion rate optimization strategy to the laser-focused-on-conversion-copywriting strategy, our clients saw their results skyrocket. 

Not convinced? I get it. It took me a long time to come around too. Don’t take my word for it. Instead, take a look at this case study.

Want to learn more? Conversion copywriting is so important that it is one of the four foundations of the Frictionless Commerce philosophy. To learn more, you can read all about it here.

Start Your Marketing A/B Tests

Now that you know what to test, here’s a quick overview of how to test.

First, construct your sales pitch. Second, create a new version of an existing product page based on the selling angles you identified. Third, create a test in VWO. Run your new product page against your current product page. Let it run as long as is needed to declare a winner.

Once the test is complete, look at the difference in conversion rates. Did you get a significant bump in conversion rates? Continue to run tests until you are satisfied.

A huge benefit of testing copy is how quickly you can iterate. We routinely create 10, 12, or even 15 different concepts before we find the winner.

Next Steps

Let’s now look at how most agencies do conversion rate optimization.