A/B Testing Tools Overview
What is A/B Testing?
A/B or split testing divides web traffic among multiple versions of a page and provides information on which version performs better at achieving a desired outcome such as signups or purchases.
The testing process aims to optimize a website’s ability to convert visitors into leads or customers. The goal is generally to increase revenue without spending more money to acquire traffic.
A/B Testing Tools Features & Capabilities
All A/B testing tools include the following features:
Splitting traffic among the different variations being tested
Calculating conversion rates
Measuring statistical likelihood that one variation will consistently outperform the other
Basic A/B testing tools may require custom design and code. Some advanced tools offer a WYSIWYG interface for creating variations. This is without needing to use HTML or CSS, which works well for non-technical users.
More sophisticated tools run targeted campaigns. These tools display different versions based on demographics, device type or referral source. They can also perform multi-page campaigns. This means testing one element (like a banner) on multiple pages during a session. These advanced tools help create a personalized user experience, with the goal of maximizing conversions.
A/B Test Best Practices
It is generally best to change one element of a page at a time so that any change in outcome can be attributed. With this approach, you’ll know which element results in better conversion.
Elements to test include:
Some split testing tools allow multivariate testing. This allows you to change multiple elements on one page, creating countless combinations. An analyst can then measure the impact of each variable on conversion.
Split testing should run long enough to reach statistical significance - at least 95% confidence. The amount of time needed varies depending on the amount of traffic to the page. Because they involve many versions, multivariate tests require more time and traffic to reach statistical significance.
A/B Testing and Conversion Rate Optimization (CRO)
A/B testing is often part of conversion rate optimization. CRO refers to the process of continually improving a website’s effectiveness at reaching its goal. Usually, the goal is to turn visitors into customers or leads.
It is usually a cyclical workflow, involving multiple steps:
Monitoring website visitor behavior
Gathering insight and visitor feedback
Testing a hypothesis for improving conversions
Key technologies in the CRO workflow include:
In-page web analytics (heat mapping, visitor recordings, etc.)
A/B testing tools
Getting pricing information for a specific A/B testing tool typically requires asking for a quote. You can find basic plans for as little as $50 per month. The more sophisticated tools can run up to $18,000 per year.