“Designing an experiment is like gambling with the devil: Only a random strategy can defeat all his betting systems.” (R. A. Fisher)
These days just about everyone does some form of A/B testing to optimize pages. But asRonald Fisher knew, A/B testing is loaded with traps, and the only way to avoid them is through careful use of randomization and statistics.
There are plenty of free tools out there that make A/B testing easy and accessible, but not all tools are created equal. Google’s Website Optimizer is one of the most complete and polished, designed to gather events directly from your site and display results in a nice report. GWO is a great way to get started with A/B testing and is how we ran our first few tests at Thumbtack.
Pretty soon, however, we started to outgrow to limited event gathering and reporting features of GWO. As data-driven decision making runs strong in our DNA, we decided to develop our own event tracking and reporting system. One component of this system was an A/B test report inspired by GWO.
As the months have gone by, we’ve come to realize just how valuable this tool is, and naturally we wanted to share it with the world. As such, we’ve packaged the code up into a nice, reusable Javascript library with a demo app that lets you enter your test results and get a handy report. It’s simple to use, runs entirely in the browser, lets you pass links to others, and uses some nifty statistics under the hood.
So give Abba a spin, check out the source, and let us know what you think! The demo page has a full FAQ where you can find plenty of details about how to interpret the report and how everything works under the hood.
