How to do A/B testing: A step-by-step guide for beginners

Sep 19, 2019·7 min read

This article will guide you through the main stages of A/B testing, starting with building a hypothesis up to implementing changes. to your website or app. These founded improvements are doomed to succeed!


So, you have a website. It works well, attracts visitors, and once in a while you even manage to sell something. But suddenly, you’re struck by a genius idea for how to make this website even better! You decide to change several buttons, rewrite the text, and add photos in a new style. You’re convinced all this will help you sell more!

How would you act in this situation? If you change everything immediately and all at once, you may risk your existing success. And, even if you are convinced your vision will improve your site’s design, your clients might completely disagree.

Should you leave everything as it is simply because your website is already generating some revenue? Of course not. Changes are good, but they need to be well-founded changes. How do you prove a change is necessary? With an A/B test.

Why is A/B testing important?

A/B testing is a great tool that enables you to test multiple variations of an app’s design or functionality and implement solutions that promise the best results. There’s no guessing or speculation—when you A/B test, you work with pure facts. You literally ask your real users their opinions. And since your users are the ones who give you money (and not your designer, external consultant or you personally)—you can make choices that are certain to improve website visits, conversions, and whatever else is important to your project. You can find out more about the potential as well as different types of A/B testing in our article.

A complete lifecycle of an A/B test

Even though an A/B test can seem intuitive and easy, the reality is a bit different. A/B tests require extensive preparation, detailed work and careful analysis. So, let’s take a closer look at the different phases of successful and result-oriented A/B testing.

Prepare for A/B testing

Careful preparation is probably the most important part of a test’s lifecycle.

Step 1. Learn more about your target audience.

Sometimes, your assumed target audience and actual audience are two completely different things. Proper research into your users is the first step to any qualitative website changes. Start by defining your buyer persona—the portrait of your ideal customer. List all their relevant information: age, education, place of occupation, family, current job, average yearly income, pets, music preferences, tattoos, etc. The more exact your buyer persona description, the better you will understand your users and their needs and expectations. When you know more about your users, you’ll be able to choose and test website element variations that are more likely to work for your user.

For example, you don’t need to test a pink landing page background color if you know most of your website visitors are men ages 35-50.

Step 2. Measure the current success of your website or app

If you aim to improve conversions with website improvements, you should know your current results. Check your site’s data via Google Analytics. What’s that? You don’t use this service for your website? Well, that is probably one of the biggest mistakes you’re making. Setting up a Google Analytics account to track your website or application stats will result in a better understanding of your users’ needs.

Google Analytics is an essential tool for tracking website and application statistics. In our case, conversions matter most. With Google Analytics, you can set and track sales, page view or file download goals. After documenting your current results, you’ll be able to compare them with variation results and figure out which website or application changes are worth it.

Let’s imagine, you offer your visitors a chance to subscribe to your blog. For some reason, you’re convinced that too few users subscribe, and your blog can achieve better results. You open your Google Analytics and, Oh my God! The stats tell you that 85 % of users click the Subscribe button but never complete the subscription form! This is a clear sign that something has gone wrong and the form requires immediate intervention.

Step 3. Build a hypothesis

Your hypothesis is an assumption of effect any change in your website elements will have on conversions. Implementing A/B testing is not just blind experimenting for the sake of experimenting. You need a clear aim to drive and focus your efforts.

Say your website visitors don’t want to subscribe to your blog’s updates. You research this topic and find out that one possible reason for this is an overly complicated subscription form. You assume deleting some fields from the form will improve the situation. But which fields cause the most confusion? There are several candidates to experiment with: name, age, phone number, company name, company size, etc.

But before removing them, you should consider the importance of these fields to your business. Do you need to ask your potential subscribers for their ages? This general age information is already provided by Google Analytics. What about phone numbers? Are they essential to your company, or you can give them up to improve conversions?

Are you ready to start A/B testing your web app?

Discuss all the nuances with our business analyst to achieve a better result!

Contact us

Run A/B testing

After careful preparation, it’s time to start testing.

Step 4. Create variations based on your hypothesis

If you only want to test one variation, then create one copy of the webpage with that variation. To test several variations, several copies are required. Depending on the A/B testing tool you choose, you can either build page copies with HTML and CSS or make visual changes with a WYSIWYG (what-you-see-is-what-you-get) editor. You can also try out multivariate testing and implement several changes on one page.

Let’s return to our subscription form example. You might decide to create several versions of the subscription form, one with each of the questionable fields removed. The test will show you which fields most irritated your potential subscribers and convinced them to leave the page.

You can also opt to completely redesign the subscription form, changing the required fields, their order, the background picture, and more. If you choose this multivariate approach, you likely won’t be able to identify the primary reason for low conversions, but you’ll probably still find the most efficient combination of elements for your subscription form.

Step 5. Calculate the time needed for credible A/B test results

The time required for collecting credible A/B test results depends on several factors. Current website traffic, the number of variations, the current conversion rate, and variation performance are all factors that impact a test run’s length. Multivariate tests usually require even more time to produce credible results.

The time needed to run a test can be precisely calculated, and the current results gathered when preparing for your test will help you make this calculation. You can use the Optimizely Sample Size Calculator to estimate the number of visits your website needs to produce credible results. Then, using your current traffic, you can determine the optimal time needed for your A/B test.

Step 6. Run the A/B test for the estimated time

When you run an A/B/n test, you divide your traffic evenly between the tested variations. If you are comparing your current version with three variations, then 25% of your visitors should see the current version of the website, and 25% will access each other version.

Technical implementation of A/B testing will require some specific tools like Optimizely or VWO. Their features help you make the most of every A/B test. Google also offers two fantastic A/B testing tools: Google Content Experiments and Google Optimize. Their functionalities are quite different, so we have done some research and compared the two in our article.

Analyze the results

Step 7. Analyze the result of testing

As you analyze your A/B test results, you may be surprised. Sometimes, your hypothesis doesn’t prove correct and your users think completely different than you expect them to. But this is actually good news! It means the test has prevented you from implementing a change that could have negatively affected your conversion rate!

Though, more often, you’ll find that your intuition is correct and the variations work better than the current website. In this case, you can fearlessly implement the changes that work with much less risk.

Step 8. Implement changes

This is the most exciting part of the entire A/B testing process. After long preparation, careful execution and precise analysis, you are ready to implement the winning variation.

This phase is also typically the most predictable one, since we already know what to expect from the website update and can anticipate improvement to conversion rates.


The results of A/B tests can be predictable or completely unexpected, but they are nonetheless approved by your real users. As a result, you can implement A/B tested changes without fear. And who knows, maybe changing a few lines of code will bring you millions of dollars and worldwide recognition, just as this happened to Microsoft - a case that we have described in our comprehensive introduction to A/B testing.

Do you want to implement changes to your web app but are not ready to risk current conversions?

Choose the wise approach and discuss the possibilities of A/B testing in your case

Contact us
Reviews: 0
Rate us 5 stars!
Thanks for reading!

Describe your product idea and we will start working on it within 24 hours.

Contact us