Are the results of your A/B tests integrated with your analytics tool?
If you’re like most people in the audience of this presentation by Dan McGaw at the Growth Marketing Conference in San Francisco, California, the answer is no.
You should be. And this presentation explains why.
When you think of A/B testing, you likely think of either Optimizely or Visual Website Optimizer. In this (NSFW—language!) presentation, Dan McGaw explains that these tools can do two things very well. They make creating variations on your website easy, and they ensure that groups of users see the same thing so you can do an accurate A/B test. The problem, however, is that they can not measure your conversion like a funnel report can.
If you’ve been in analytics very long, you know that using funnel reports is critical to your success. Increasing signup conversions is great, but if it doesn’t lead to more money, what is the point of doing it? Dan will walk you through two stories where a successful A/B test created a decrease in another key metric.
First, you’ll learn about Dan’s own A/B test, which increased conversions dramatically. Everyone was excited, until they looked at the Lifetime Value of that cohort three months later and found that it had dropped by 30%.
Second, you’ll learn about Segment.io (a great company to help you integrate your A/B testing with your analytics tool, incidentially). They tested the language on a single button, and their A/B testing tool declared “Get Started” the winner over “Create a Free Account.” Most companies would have made the change immediately. But Segment.io decided to look beyond the single metric, and ultimately saved themselves a lot of money.
Mixpanel, Kissmetrics, and Google Analytics all integrate cleanly with most A/B testing tools. Dan McGaw explains why it’s so important to take the time to do it, and then take the time to look carefully at your entire funnel before declaring an A/B test a success.
A/B testing is powerful, but only if you are doing it correctly. If you become completely focused on your test’s impact on a single metric, like signup conversions, you can miss its impact on other potentially more important metrics like lifetime value, churn, and revenue.
If you want to increase your bottom line, watch this presentation to learn more about integrating your A/B testing tool with your analytics tool. You can also download the slides to get a better look at the detailed case studies that Dan presents. Use better data to get a better bottom line.
