VKI Studios is now Cardinal Path! www.CardinalPath.com
Learn more about Cardinal Path

Google Website Optimizer and Google Analytics - Interpreting the Data

Previously, I wrote a blog post on how to integrate GWO data with GA using custom variables. The ultimate goal of the integration was to look at metrics other than just conversion rate. For example, using the e-commerce reports you may find that one of the losing variations generated significantly more revenue than the winning combination. Or, you could segment your visitors by variation and geographic location. Perhaps one variation does better in one state or region versus another. If you do implement my GA/GWO integration method, here are some of the reports I would suggest you look at in GA. One thing to keep in mind when comparing the data from the two tools is that the data will never match exactly.

First off, let's look at the GWO results

As you can see, our "buttons" combination had almost a 15% lift from the original. That's quite a feat, and our client was very happy about that. Now, let's see what the GA data provides us.

GA Reports

The first report I would recommend looking at is the Custom Variables report. To get to this report, click on "Visitors" then "Custom Variables" in the navigation.

The beauty of this report is that it will give you a great high-level overview of how your variations perform across a variety of metrics. Already, we can see some interesting data. It looks like visitors to variation 1 have a longer time-on-site, but view less pages.

If you are running a conversion test that will affect your online revenue, like our test did, you will definitely want to check out the E-commerce tab. This tab will give you many e-commerce metrics segmented by the variation numbers. You'll be able to compare revenue generated by the different variations, average order value, and other e-commerce related metrics.

During our testing period, our buttons variation generated a 10% increase in revenue over a two week period. If the revenue from the original variation was $100,000 in that two week period, our buttons variation would have added $10,000 in revenue. If we extrapolate that 10% increase out to a one year period, that's $260,000 extra in revenue! Not bad from a relatively simple button test.

Advanced Segmentation

For those of you who love to use advanced segments, you can also segment your visitors based on what variation people saw, and apply the segment to any of the reports.

Following is a screenshot of how I created my segments. To create your own segment, you will need to use the key and value dimensions that correspond to the slot you are storing your GWO data in. In my case, I used slot 1.

The values that you will be able to select may need a little bit of explaining if you don't know the GWO cookie format very well. If you are running an A/B test, the values that will appear will be a single number. Zero represents the original and one represents the "B" page. If you had more than one variation (ie. an A/B/C/D test), then numbers two and three would represent the "C" and "D" pages respectively. In a multivariate test, the format will look like "x-x-x" in a 3 section multivariate test. Each "x" will be a number that corresponds to the variation of the section the visitor saw. Again, the number is zero indexed, so zero represents the original, one represents the first variation, two the second variation etc.

Questions To Ask

Now that you know where to look for your data. The next step is to formulate some questions about the data and really dive deep into the data. Some questions you might want to find answers for are:

  1. What average order value does each variation bring in?
  2. Do my variations drive my visitors to complete other goals on my website, other than my conversion goal?
  3. How do different geographical regions respond to my variations? Do visitors from country A convert more when seeing the original, while visitors from country B react more favorably to variation 2?
  4. How many days or visits does it take before a visitor converts for each variation.

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
Great Post
# Posted By Nathan | 5/4/10 3:46 PM
Yes, I've had the same experience.

We customised GA to give us greater insight into what the GWO traffic was doing, and it basically gave us a completely different picture to what GWO was saying.

What we found was that the poorer "converting" option was actually generating three times as much revenue.

I think most people are really unaware of how misleading the GWO chart can be.
# Posted By Mark | 9/8/10 11:18 PM
Thanks for sharing screenshots, that really helps!
# Posted By Rishi Rawat | 11/30/10 3:33 AM
Hi I just wanna say Im a newbie when it comes to GA, so Ive found this very useful and Ive started using it on my website http://www.etsy.com/shop/CharmingBeadz. So thanks so much!
# Posted By darings | 1/27/11 6:39 AM
Our Company has been collecting GA data for a long time but never knew how best to utilize the information. As a complete newby to the GA platform, after reading this article and a very useful book by Mary E. Tyler, Jerri Ledford from http://bytesland.com/view/Google-Analytics I attended 2 day Google Analytics Seminarwas and was able to not only follow the pace of the training but really gain a valuable insight into what GA is designed to achieve. I have walked away with the confidence to set up our goals and funnels and really dig deeper into our visitor data. I also attended the Google Website Optimizer 1 day Seminar and found it very interesting as we have just launched a new website and are launching a whole new stream of E-Commerce. The knowledge I gained will be invaluable in making sure we get the fundamentals rights and create a user-friendly interface for our customers. All in all I highly recommend such trainings for anyone using GA and Optimizer.
# Posted By Basyl Hunter | 3/3/11 2:16 AM
In response to question 3, I've been noticing dramatic differences in the level of response based upon geographic region. I'm collaborating on a website that is international in nature (http://mythicscribes.com/), and we're finding that visitors from Europe and Australia respond very differently to certain variations than do visitors from North America. We wouldn't have noticed this if we hadn't taken the time to analyze the data.
# Posted By Gerald Frain | 4/29/11 4:02 PM
What you are saying all very well. But small webmasteers do not have the time to spend on tweeking analytics. They may be better off spending the time to create more contents
# Posted By quotesom | 5/26/11 2:52 AM
.