The Power of Amazon A/B Testing: Unlocking E-Commerce Success

Did you know you could increase your sales by up to 25% just by implementing Amazon A/B testing?

Consistent optimization of your Amazon product listings could boost traffic and help increase both conversion rates and sales. 

In this article, we’ll explore the impact of A/B testing for Amazon sellers and how to make use of this free Amazon tool. 

What is A/B Testing?

A/B testing – also known as split testing – is a systematic approach used in marketing to compare 2 versions of something to discover which one performs better.  

Usually the A version is the original and the B version is the variation with the changes made to it.

Here’s an example:

Split testing can be done by testing changes to all kinds of things:

• Colors 
• Copy 
• Images 
• Icons 
• Placement of elements (such as in design) 

…and much more. 

Do I need A/B Testing?

If you want to increase traffic to your listings and improve conversions, then yes, you absolutely need A/B testing.

A/B testing is all about optimization. 

First, you can work on optimizing for click-through rate (CTR). 

The more you optimize your product title and main image on Amazon, the more customers you’ll get clicking through to your product listings from the search results page (SERP).  

Once you have customers clicking through to your listing, you can work on improving your conversion rate (CVR).  

The more optimized your content on the product page – bullet points, description, A+ content and Brand Story – the more conversions you’ll have. 

Once you’ve established a high click-through rate that results in a high conversion rate, Amazon will notice – and this will increase your overall ranking.  

Keep in mind, though, that a higher conversion rate doesn’t necessarily mean a better conversion rate. You could actually bring conversion rate down if you are getting people to click through on your listing but you’re not driving the right type of customer to your page.  

A higher ranking leads to more traffic as Amazon sees your products and listings as high-quality converters. 

More traffic leads to more sales… 

More sales lead to more reviews, which improves your listing quality. When customers see a product with reviews, this works as social proof and helps them trust the brand. 

This all continues in a self-sustaining cycle that’s known as the Amazon flywheel which you can read more about in this post. 

So yeah, optimizing your listings is good for all sorts of things – growing your brand, increasing conversions, increasing traffic and ranking as well as sales. 

Does Amazon Allow A/B Testing?

Yes, Amazon allows A/B, or split testing.  

In Seller Central, A/B tests are known as “Experiments” and can be found under the main menu by clicking on “Brands” and then “Manage Experiments”. 

Who are Experiments available to on Amazon?

Amazon’s “Manage Your Experiments Tool” is available to brand-registered sellers. If you’re brand-registered and want to start A/B testing your product listings, but you don’t see the “Manage Experiments” option, you can click on “Build Your Brand”.  

This will take you to a page with a lot of helpful resources and programs to help: 

• Grow your audience 
• Increase sales 
• Build brand loyalty 
• Protect your brand 

In the “Increase Sales” section, select “Manage Your Experiments”.

What Types of A/B Testing can you do on Amazon?

The “Manager Your Experiments” tool allows you to split test most facets of the individual product listing: 

• Product Title 
• Bullet Points 
• Product Description 
• Images 
• A+ Content 
• Brand Story (the newest addition for testing in Amazon’s experiments) 

How to do A/B testing on Amazon Seller Central?

1. From the Manager Your Experiments tool, select which type of experiment you want to run. 

2. A list of your ASINs will appear, with their eligibility status. Select the ASIN you want to run a split test on.

⚠️ Important: You can only run ONE test on an ASIN at a time. For example, if you’re testing 2 images, you can’t also test 2 different sets of bullet points. 

3. Toggle “View Settings” to adjust the title, hypothesis, duration, start date and whether you want to automatically publish the winning version of your experiment. 

4. Give your experiment a descriptive title so that when you look back on the results, you have some context. 

5. Enter an optional hypothesis. This should be specific and based on a well-defined assumption. This will also help you when you look at the results once the split test is over. 

For example:  

• “Adding descriptive detail in the product title will increase the number of customers.” 
• “Adding a lifestyle image of the product will increase the number of sales” 

6. Set the duration of the A/B test: 

• The recommended duration “to significance” is a good bet because it will collect data until there is a version of your content that’s performing much better, and the experiment will end.  
• Toggle “start my experiment sooner if possible”, unless you plan to run a test on a product that’s yet to be published – then run it at a later date. 
• Toggle “automatically publish the winning version” if you want the listing to automatically change based on the winner – this is optional 

7. Enter or select your “B Version”. 

The “A Version” will always appear as the original. The “B Version” is what you’ll be testing. 

Example for a product title split test: 

Example for an image split test: 

8. Schedule the experiment. Now’s the time to wait for the data to be collected. 

9. Analyze the results. We’ll go into more detail about this in the next section. 

Analyzing A/B Test Results

While the experiment is going on in Amazon, you can go in and see how they’re doing. 

Once the experiment is over, you’ll be able to view results for the A and B versions like: 

• Sample size – this will be as close to 50%-50% as possible 
• Conversion 
• Units sold 
• Sales 
• Projected 1-year impact of winning content 

Some results will have an obvious winner, like this one: 

Some will be a little less:

Amazon also shows you the estimated lift in sales from the a/b test. Here’s an example: 

You’ll also see an estimate of the impact potential over a year:

Others won’t have as big of an impact, but that’s okay. It’s better to continue testing to see what works. 

Amazon A/B Testing Examples

You might be wondering, what kind of tests should I be running? 

Since your title and main image are the first things that people can see from the search results page (that you can at least control), you can start with one of those. 

Main Image A/B Testing Examples

Before you start split testing your main image, make a search for your product using one of the keywords to find.  

Try to find patterns and see how you could make your product stand out from the rest. 

To create a convincing or engaging main image, you can experiment with changing the size of the image of the product like here: 

Including a bit of the product outside of the packaging versus just the packaging: 

Testing whether or not the packaging adds value to the image: 

In the example above, perhaps the brand Blueland played around with different main image ideas:

• With and without the blue tin box
• With and without the envelope packaging
• With and without the laundry tablets visible

Secondary Image Amazon A/B Testing Examples

With the secondary images, you can try testing how simple product shots work against lifestyle images:

You can also test to see whether images with people or lifestyle images work better: 

You can read more about what kinds of images you should include in your image reel in this article.

Tips for Better Amazon A/B Testing

A/B testing on Amazon can be a powerful tool for optimizing your listings. How can you get the most out of it? Here are some tips: 

Take your time 

Don’t draw conclusions too quickly. Testing over an appropriate duration accounts for changes in customer behavior and makes sure you’re not basing your results on inconsistent or irregular data. This is why it’s good practice to let Amazon decide when they’ve gathered enough data. 

Test ONE thing at a time 

All successful split tests test just one variable at a time. If you’re testing multiple variables at a time, you may not be able to correctly pinpoint what’s working and what’s not. With just 2 options, you’ll have the results that show the clear winner.  

Always be testing 

Don’t just do one test and leave it. After you’ve tested the main image for an ASIN, test the product title. Once you’ve done that, test the bullet points or the description. Keep running tests to refine results and increase optimization. 

Don’t get discouraged by negative results 

If you run a test and see very negative results, don’t worry. This is a great learning lesson to draw insights to inform future tests and improvements. 

Key Takeaways

A/B testing on Amazon is an opportunity for optimization that should not be ignored. 

The results offer: 

• Data-driven decisions that optimize sales, traffic and conversions
• Continuous optimization 
• A customer-centric approach 

So what are you waiting for? Start split testing your Amazon listings now.