Ultimate A/B Testing resources and how I used them

Posted August 4, 2009 by Brian Cray

Reading time: About 2 minutes

Comments10

Definition: A/B testing allows you to test variations of content simultaneously to see which performs best.

Update – Aug 3rd, 2010: I just released phpA/B for A/B testing with PHP. Try it out today!

A/B testing overviews

An Introduction to A/B testing by Jesse Farmer; Jesse writes a clear introduction to A/B testing, and writes more on the subject at his blog
How To Increase Site Performance Through A/B Split Testing on UX Booth by John Hyde of Site Doublers; A nice simple introduction written in UX Booth style
Putting A/B testing in its place by Jakob Nielson; Jakob argues that A/B testing has many benefits, and qualitative tests should supplement
How to improve A/B testing on ClickZ; Simple tips on how to improve the value of your A/B tests
Effective A/B testing by Ben Tilly; A slideshow on what is A/B testing

Using Javascript to run A/B tests

For flexiblity use Genetify, a stand alone Javascript file that allows you to do testing on variations in content and CSS.
For ease-of-implementation use Google Website Optimizer, a more turnkey solution than Genetify. Just plug stuff into Google Website Optimizer and it does almost everything for you.

Google Website Optimizer tutorials

Video tutorials of Google Website Optimizer provided by Google
Advanced Website Optimizer tips on the official blog by Trevor Claiborne
Google Website Optimizer tricks blogged by Eric Vasilik; Also includes Google Analytics tricks, too!
How to do A/B Testing in WordPress on Think Vitamin by Ryan Carson; A screencast for using Google Website Optimizer specifically on WordPress

Genetify tutorials

Genefity walkthru & demo walks you through how Genetify works and allows you to safely create example tests in real-time
Genetic A/B Testing with JavaScript by John Resig; John uses his javascript mastery to take Genetify to its potential

New A/B tests on Brian Cray.com

I’ve been wanting to run an A/B test on Brian Cray.com for a while. This morning I setup some tests in my blog post footer using Google Website Optimizer to test which words influenced the most action.

Here are the tests:

Variations of “Add comments”

Add comments (original)
Respond
New comments

Variations of “Subscribe”

Subscribe (original)
RSS

Variations of “Share”

Share (original)
Bookmark
Save

Variations of “Follow me”

Follow me (original)
@briancray
Twitter

What will be the results?

What do you think the results will be? Have you run similar A/B tests? You should share your experiences with others in the comments!

About the author

Brian Cray is a Columbus, Ohio-based web entrepreneur & consultant. View some of Brian’s work in his portfolio and learn how to make kick ass websites by reading his blog.

Brian Cray: Measurement-driven, user-focused web entrepreneur & consultant