Advantages and Disadvantages of AB Testing
Marketers can increase their conversion rates dramatically by AB split testing alternative content designs. The name comes from the fact that two different versions — “A” and “B” — of a landing page or email are tested simultaneously to work out which is most optimised for conversion.
“Split testing” then refers to the random assignment of new visitors to the version of the page they see. In other words, the traffic is split, and both versions are shown in parallel throughout the data collection period in equal proportions.
A/B split testing advantages
A/B split tests have several advantages, including:
Useful in low-data rate tests: If your landing page has only a few conversions per day, you simply can’t use a more advanced tuning method.
Ease of implementation: Many software packages support simple split tests. You even may be able to collect the data you need with your existing Web analytics tools.
Ease of test design: Split tests don’t have to be carefully designed or balanced. You simply decide how many versions you want to test and then split the available traffic evenly among them.
Ease of analysis: Only very simple statistical tests are needed to determine the winners. All you have to do is compare the baseline version to each challenger to see if you’ve reached your desired statistical confidence level.
Flexibility in defining the variable values: The ability to mix and match allows you to test a range of evolutionary and revolutionary alternatives in one test, without being constrained by the more granular definition of variables in a multivariate test.
A/B split testing disadvantages
A/B split tests also have their drawbacks, including these:
Limited number of recipes: While you may want to test dozens of elements on your landing pages, because of the limited scope of split testing, you have to test your ideas one at a time.
Inefficient data collection: Conducting multiple split tests back to back is the most wasteful kind of data collection. None of the information from a previous test can be reused to draw conclusions about the other variables you may want to test in the future.
Despite these drawbacks, if you’re new to landing page optimisation, A/B split testing is definitely the way to start. Because of its simplicity and the availability of free tools, it’s easy to begin down the road of improving your conversion rate.