A/B testing with Google Analytics Content Experiments

Introduction

The Google Web Optimizer has recently been incorporated into Google Analytics and is now called Google Analytics Content Experiments (GACE). This blog post will show how we’ve set up A/B testing in GACE for our company homepage. Our goal for the experiment is to promote CollabSpot Boards to the visitors of our homepage. We’ll also show what the result of the experiment looks like and how you can proceed to other experiments.

Variation pages and experiment goals

Since we’ll be doing A/B testing for the CollabSpot homepage, then we need to have a variation of the original webpage that’s present in a separate URL. In our case the URL for the original version is at http://www.collabspot.com, while the one for the variation is at http://www.collabspot.com/home. We will input these two URLs once we start setting up the experiment within Google Analytics.

A variation webpage is not very useful unless we have a goal to consider for the experiment. A goal is simply an answer to the question “What do we want the visitors to do when they get in our homepage?”. The goal for our experiment is to promote CollabSpot Boards by making our homepage visitors to go to the CollabSpot Boards website. To consider this goal in our experiment, we made the CollabSpot Boards section in the original homepage more noticeable by putting a border around it. Alternatively, the variation page doesn’t have a border around the CollabSpot Boards section. We’re convinced to make only subtle changes in our variations so we can easily correlate those changes with the result of the experiment. It will be harder to pinpoint what factors lead to the winner of the experiment if there were so many changes implemented at the same time.

One can set up goals in GACE by clicking on “Admin” on the top right of the Analytics interface, selecting the “Profiles” tab, and then clicking on the “Goals” tab. You can create various goals for your experiments, but the simplest type of goal is the URL destination. Since our goal was to promote CollabSpot Boards, then the URL destination must be http://boards.collabspot.com. If a visitor lands on this URL during a visit to a certain variation of the homepage, then that variation scores a point for the experiment.

Setting up the experiment

You need a Google Analytics account to start using GACE, and your own webpage as a subject for A/B testing. In our case, the test subject is our company homepage http://www.collabspot.com. Log in to your Google Analytics account and on the left panel, click on “Content” then click on “Experiments”. From there, setting up an experiment is pretty much straightforward by simply following the instructions. We’ve named our experiment “www.collabspot.com to cs boards” to easily identify the goal of the experiment. The only potentially tricky part in the set up is putting the experiment codes in your website’s HTML source. If you have total control over the codes or if you have a webmaster to do this for you, then this part of the setup will be a no-brainer. But if you’re using a CMS like WordPress, then you may have to resort to GACE plugins (see below) to facilitate in your setup.

How to access the Content Experiments in the Google Analytics panel.

The last step allows you to start the experiment. Upon doing so, you can let GACE do its thing and wait for some time until an experiment winner is declared. Note that you cannot change the goals once you start an experiment. You can only change the URLs for the variation pages and the percentage of site visitors you want to involve in the experiment.

Results

The view when a winner is found in an experiment.

The experiment will automatically end once a winner has been found. This happens when there are enough visitors in the homepage who have participated in the experiment. We can easily check this by going to the GACE interface, seeing the experiment status as “Ended”, and the details as “Winner found”. In our case, the winner of the experiment is the original version, where we’ve put a border around the CollabSpot Boards section. Its visitors displayed a higher percentage of going to the CollabSpot Boards website. Since we now know this, we can iterate on the winning version and try out other experiments with other goals.

Content Experiments plugin for WordPress

Since GACE is relatively new, there are not many plugins available yet. But WordPress users will be happy to know that there exists a plugin to easily set up the experiments. You can find it in this link: http://wordpress.org/extend/plugins/google-content-experiments/. After installing the plugin, a checkbox will be present in the bottom part of every page editor. Checking this box will bring out a text area where you should put the experiment codes given during the setup in GACE.

A sample input via the GACE WordPress plugin.

So there you have it. I hope you’ll find the Google Analytics Content Experiments a valuable tool in optimizing your website for various goals. Its power lies in not having to rely with your gut feeling on what design works better for certain goals by applying a quantitative approach in determining which layouts or changes will work and which won’t.