Why creating A/B/n tests on your digital business is a must

2018/03/27 07:52

ab test

Back in the 1990s when Google, Facebook or even Myspace didn’t exist, businesses could afford to ignore the World Wide Web. Only a few nerds using dial-up modems were online to check them out. Today, with the advance of mobile technology, everything is changed. We use our mobile phones to connect with friends, order pizza, make travel reservations and so on. The World Wide Web is our favorite channel for social interaction and our most important information source. Also, the web is highly influential. It is estimated that up to 90% of Americans perform an online research before making an offline purchase.

Unfortunately, many websites couldn’t keep up with web’s explosive growth and we are now in an uncommon situation: the web continues to grow but the performance and use experience of many websites are below their potential. Making it simple, you can read this as your online business is getting fewer actions that it deserves! It’s a very bold claim but the fact is that there is no website that could not be improved.

Consider the mighty Google who is permanently improving its algorithms and website design. The same holds true for the most important websites, from Facebook to Amazon. All these companies are all well known for making continuous improvements to their websites through testing.

Today, there are only a few websites where design and content are based on results from controlled testing. Judging by the similarities in design, it looks like most websites are based on copying ideas from the competition. This approach leads to poorly performing solutions being accepted as “best practices” because of their commonality and, obviously, you’ll miss something essential by following the “best practice”. You’ll miss taking into consideration the unique environment of your business, its specific goals, and target audience. Your website should uniquely communicate the value proposition of your service or product. It should make it very easy for visitors to take the actions they desire. Every time new functionalities are added to your website there’s something you must always remember to do beforehand: Testing. Let’s explore why controlled testing is required to continuously improve your conversion rate.


What Is Controlled Testing

Controlled testing implies structuring the test with a valid control that is run synchronously with challenger variations. In a controlled website test, the control group sees the unaltered web page that is being tested for improvements. The challenger variations are tested against the control, and all other factors are kept, as much possible, unchanged.


What Is A/B/n Testing


In A/B/n testing, also referred as split tests, to each of the visitors is assigned to see one of several challenger variations, numbered from A to n. The testing process follows the following steps:

  1. Each visitor must enter into the experiment unaware of it and must be randomly assigned to control or one of the challenger variations.
  2. To ensure valid test results, the visitor must see same page variation on each visit.
  3. Upon test goal completion, a conversion is recorded for the variation assigned to the visitor.
  4. The testing tool calculates which one of the variations delivers the best conversion rate.
  5. The test ends when statistical significance is reached, meaning that the test will run until it is unlikely that chance is responsible for the results.


Selecting a Tool for A/B/n Testing

Even if it’s possible to run A/B/n testing without a designed tool, this approach is far from being optimal. Selecting valid samples is just one the advantages that a specially designed tool bring to the table. Most of the testing tools share at least the following additional functionalities:

  • Incoming traffic split between variations.
  • Make sure that each visitor sees the same page variation each time they return by setting cookies or similar mechanisms.
  • Provide easy to understand reports for conversion rates, with data for each variation
  • Show the statistical significance of each variation, to help decide when tests should end and a winning page could be declared.

There are many tools available to help you perform A/B/n tests. True to our A/B/n testing approach we will weight the best solutions available on the market to help you decide which one is best for your business.

Google Optimize

  • Provides not only A/B/n testing but also multivariate and redirect.
  • Deep integration with other Google tools: Analytics, AdSense, AdWords.
  • Advanced reporting capabilities.
  • Results are auto-loaded to Google Analytics.
  • Native Chrome extensions for A/B/n tests creation.
  • Advanced targeting (URL, audience, geolocation, etc.).


  • Provides not only A/B/n testing but also multivariate and redirect.
  • A large number of integrations available out of the box, including Drupal.
  • Advanced reporting capabilities.
  • Real-time and easy to use the editor to create tests and goals.
  • Complex targeting options (single page/user type/whols site).
  • Tests scheduling.
  • Multi-page tests.

Visual Website Optimizer

  • Provides not only A/B/n testing but also multivariate and redirect.
  • Integration with Google Analytics.
  • Multi-page tests.
  • Third-party integration with heatmap apps like Crazyegg.
  • User-friendly and visual editor.

Which one should you use? OK, the answer for that is… it depends. If you are already a heavy user of Google tools like Analytics then Google Optimize should be your first choice. On the other hand, if you are looking for a more customizable and developer-friendly solution you could choose Optimizely or Visual Website Optimizer.


Are your A/B Tests giving you the expected results of your strategies?  Tell us your experience with this useful digital marketing tool.