A/B Testing, As Explained with Animated GIFs

A/B Testing, As Explained with Animated GIFs


At ShareProgress, we’re big fans of using data science in our work. But when sharing our excitement with others, we’ve discovered that not everyone knows what we’re talking about.

What did you say?

To help clear up some of that confusion, we’ve decided to create a few posts explaining some key concepts of data science. First up: the A/B Test!

I'm so freakin' excitedI’m so freakin’ excited!

Here’s how it works: Let’s say you have a bunch of people you want to invite to your birthday party.

Party!I don’t know if the Olive Garden will be able to seat us all together.

You have two ideas for an invitation email: one about birthday cake and one about presents.

That's my stapler Merry... birthday!

Instead of just going with the one you like best, you randomly choose a few people and send them the cake email, and then you randomly choose a few more and send them the presents email.

My email is better than your emailCake vs. Presents

You wait a while, and then you see which people have RSVP’d for the event. If more people RSVP’d from the cake email, you send that to everyone else who you’re inviting. If more people RSVP’d from the presents email, you send that one instead.

Which one will win??

If you’re only inviting a few friends, you might not get enough RSVPs to be sure which email is better. But if you invite hundreds or thousands of people, there’s a good chance you’ll discover that a lot more respond to one version over the other. In that case, you’ll know with confidence which one is the best choice.

Were you expecting someone else?Everybody loves cake!

For nonprofits and companies, the same approach can be applied to the emails they send: asking supporters to make a donation; selling subscriptions to online services; telling people they should upgrade to the newest smartphone.

Is that the new iPhone?“I hear this one comes with a 40 megapixel camera, an 8″ screen, and the ability to time travel four minutes into the past.”

A/B Testing can actually be used for asking people to take any type of action, as long as you can send different messages to small, random groups and determine whether those people did or did not take the action in response.

ATTACK! / Please engage, if you'd be so kind.

It’s possible run an A/B Test with more than just two options, using different groups for each one.

Hey, Jim / Need your help, Jim / This is important, Jim / ...

And it can even be used for differences beyond just language, like how information is presented or what imagery is shown.

A vs. B

That’s it! Now you know about A/B Testing — you can visit the links below for more information, and check back soon for more data science concepts explained through animated GIFs.

Pedro for President

Helpful Links:

Written By

Jim Pugh