A/B testing is a multivariate test in web analytics that involves two variants to determine which version of a web page yields better results. Two versions of a web page are randomly assigned to web visitors, statistical analysis is performed, and the results of that analysis determines which variation of a web page is better.
A/B testing (also known as bucket testing or split testing) refers to a process that involves comparing different versions of a web page in order to find out which version is performing or converting better. Though the name A/B might suggest that only two pages at a time can be compared, in reality, you can compare as many pages as you would like. The web pages are compared by showing two different variants (lets say A and B) to visitors that have similar qualities and habits at the same time. The test results of A/B testing will reveal the page that produces the better results and converts visitors more successfully, and that page is considered the better option.
Running an A/B test that compares a direct variation of a page against one that is currently being used allows you to learn what time of changes, if any, you should make to your webpage in order to improve its effectiveness. By using AB testing, the guesswork that surrounds conversion optimization and website optimization can be eliminated. It allows you to make decisions that are based on data rather than guesses or projections. Through measuring the impact that changes have on metrics, you can be sure that the changes you make will yield positive results.
How Does A/B Testing Work?
The process of conducting an A/B test is pretty straightforward. You start by taking a specific webpage and making changes to it to create a second (or third, or fourth, or however many you wish) version of it. The changes you make can be as simple as altering the headline or moving a button, or it can be as complex as a complete overhaul of the design of the page.
Fulcrum-Tip: Once the changes have been made, a portion of your traffic is shown the original version of the page (this is the control page) and another portion is shown the page that has been modified (this is the variation page).
As visitors are shown either the control or variation of the page, their engagement with each one is measured and collected through an analytics dashboard. That information is then analyzed via a statistical engine and you can see if changing the page provide to provide a more positive experience, if it created a negative experience, or if there was no effect on user behavior at all.
What can be Tested?
Virtually any element on your website that has an effect on the behavior of your visitors can be A/B tested. Some examples of elements that can be tested include:
Headlines
Sub headlines
Testimonials
Text used in a call to action
Buttons used for call to action
Text used in paragraphs
Links
Social proof
Images
Mentions in the media
Badges
The Process of A/B Testing
The process of conducting an A/B test includes the following:
Gather data. Analytics can provide great insight into where you can start optimizing. Starting with high traffic areas on your site will allow for much quicker data collection. Pages to consider include those that have high traffic, yet have low conversion rates and/or high drop-off rates that you want to improve.
Determine conversion goals. Conversion goals are the metric that you will be using to find out if the variation of a web page is more successful than the original version of the page. Conversion goals can vary and can include anything from users clicking on a button or a link, to signing up for an email list, or to making a purchase.
Develop a hypothesis. After youve established a goal, you can start generating ideas for your A/B testing, as well as hypotheses regarding why you think variations will perform better than the existing version of a web page. Once youve created a list of ideas, start prioritizing them in terms of the impact you expect they will make and how difficult the changes will be to implement.
Develop variations. Using whichever A/B testing tools you have chosen, make the changes that you desire to a web page. These changes could include changing the color and location of a button, making navigation elements more or less visible, or changing the arrangement of elements on the page. It could also be something that is completely customized. Most reputable A/B testing tools will offer a visual editor which makes it easier to make the changes. After you make your changes, conduct quality assurance to ensure that the changes work as you expect them to.
Run the experiment. Get your experiment up and running. Site visitors will be randomly assigned either the control or the variation page. The interaction of users with each experience will be measured, counted, and compared to see which one performs better.
Assess the results. After the experiment is complete, analyze the results. The A/B testing software you are using will show the data from the experiment, as well as the differences between how the versions of your web page performed, allowing you to see if there is a difference that would be considered statistically significant.
Whatever the outcome of your experiment may be, make sure that you use your experience as a way to improve your web page and improve the experience of your users.
A/B Testing and Search Engine Optimization
Major search engines, including Google, not only allow A/B testing, but they encourage it. In fact, search engines have stated that conducting A/B test or other types of multivariate tests will not do any damage to the ranking of your website. However, with that said, it is possible to put your page ranking at risk if you use A/B testing tools too frequently, especially if you are using them for the purpose of cloaking.
In order to avoid damaging your page rank, make sure to follow these best practices, which are presented by Google:
Dont cloak. Cloaking refers to the practice of showing search engines content that is different than an average visitor to a website would see. If you conduct cloaking, your site can end up being demoted or even completely removed from search engine results.
Only run experiments for as long as they are necessary. Running A/B tests for a longer period of time than is actually necessary might be viewed as an attempt to trick search engines. This is particularly true if you are showing one variation of your web page to a large amount of users. According to Google, you should update your site and remove all test variations as soon as your A/B test is completed. Furthermore, you should not run tests longer than you need to.
Opt for 302 redirects, not 301s. If you running a test that redirects the original URL of a web page to a variation URL, make sure you use a 302 redirect instead of a 301. A 302 redirect is temporary, while a 301 is permanent. Using the 302 redirect will let search engines know that a redirect it only temporary and that they should retain the original URL for the web page indeed instead of the test URL.
By following these best practices offered by Google, you can ensure that conducting A/B tests wont have a negative impact on the ranking of your site.
The Benefits of A/B Testing
Through A/B testing, you can clearly see which variation of a web page customers interacted with more and had a better experience with. With that information, you can improve your strategy to create a more appealing web page that your visitors will enjoy interacting with, which can improve your conversions, your page ranking, and the overall success of your business.
In addition to these benefits, heres a look at some of the other notable benefits that A/B testing can provide:
Better engagement with content. The information gathered from A/B testing can help you improve the structure of your content. When your content is better designed and structured, customer engagement with that content will improve.
Lower bounce rates. High bounce rates are a real bummer. Through A/B testing, you can significantly reduce your bounce rate and help keep your visitors on your site long enough so that you can provide them with valuable information, which can ultimately lead to more conversions.
Better conversion rates. A/B testing is the simplest and most effective way to transform a web page and turn it into a site that gets more conversions. Through test results, you can see which elements your visitors interact with more and make the necessary changes to your web page in order to encourage more engagement with the page, which will lead to more conversions.
Better content. Content is one of the most important if not the most important element of a website. Through A/B testing, you can create better content for your site, which can improve the experience of your users and improve the overall ranking and success of your site.
These are just some of the benefits that A/B testing provides. In short, this type of testing provides valuable information that you can use to improve your website, and thus improve your page ranking and your overall success.
Why A/B Testing Works
Through A/B testing, you can see which content on your website leads visitors to:
Engage more with your site
Spend more time on your site
Click through your site to additional pages
Take certain actions, such as signing up for mailing lists and making purchases
In other words, A/B testing allows you to see which version of a web page leads your targeted audience to take the actions that you want them to take. For example, if your goal is to have more people sign up for your mailing list, through A/B testing, you can try out different designs or content for your call to action and find out which one leads to the most amount of signups.
Fulcrum-Tip: Instead of having to guess or predict what the visitors to your site might want or how they might engage, you can test your theories with A/B testing and see what visitors actually do when they are provided with different options.
Bottom Line
A/B testing is a valuable tool for web analytics. It can drastically improve the success of a website and lead to more conversions, and ultimately help you reach the goals that you have set out to achieve.