Imagine this: you’ve been kidnapped by the villain of your latest progressive campaign (Fracking! Opposition to same-sex marriage! The Trans-Pacific Partnership! Fill-in-the-blank!). Let’s go with fracking–a force so evil that most people can’t even totally explain how it works.
Fracking has tied you to a wooden chair and strapped a bomb to your chest. Things are looking bleak. Fracking twirls his moustache devilishly. His crude oil throne sparkles behind him. The timer on the bomb shows five minutes, and it’s ticking down towards zero. Fracking cackles as the seconds tick down.
“There’s only one way to stop this bomb, you progressive do-gooder,” Fracking says. “You have to come up with five A/B tests for new best practices in the next five minutes, or you’re toast. Which is ridiculous! There’s no way you’ll come up with that many tests! It takes some organizations years to come up with one A/B test!” Fracking cackles again and sits on his throne of crude oil to wait for your response.
Five A/B testing ideas in five minutes? It’s gonna be tough… But at least you’ve got one idea.
1. Focusing on the person vs. Focusing on the Issue
“Okay, I have a test! In an email, what if you kept everything else the same, but in one version you tried focusing on the person who was receiving the email and in the other version you only talked about the issue? You could focus on the recipient by adding phrases like ‘I thought you’d be interested…’ or ‘You’ll care about this…’ In the other version, you would exclusively describe the issue and why it was important, without mentioning the person who was receiving it.”
Fracking narrows his eyes at you. “Fine. I’ll give you that.” You have four minutes left, and four more tests to explain. This might be possible after all…
2. Sign this Petition vs. Add Your Name
“I know another one!” You’re feeling confident now. “This is simple. For a petition that you want your supporters to share on Facebook, you could test the same Facebook headline, but put ‘Sign This Petition’ in front of one version, and ‘Add Your Name’ in front of the second version. Then you’d wait to see which version your supporters responded to.”
It almost looks like Fracking’s throne has shrunk a little after you announce your second A/B testing idea, but you can’t quite be sure. He waves his hand at you. “Three more to go.” The clock is ticking, and you’re starting to sweat a little.
3. Description vs. No Description
“Can I do another Facebook one?”
“What if you just had the link to the petition without any description at all? And then for other version, you included a description of what the petition was.”
Fracking lets out a dramatic sigh. His throne is definitely getting smaller–you can see it shrinking now, as you watch.
4. Short Email vs. Long Email
“This one’s for email. So what if you tested one shared email that was just a sentence or two long, and one email that was double that length?”
“Ha!” Fracking exclaims, “But what if people weren’t responding to the length, but were instead responding to the message of the additional sentences in the longer version? That’s a confounding variable!” You can almost feel the gears in your head spinning as you try to come up with a way around the problem.
“You could run the same test on multiple different campaigns! Then you would see whether the results were generalizable, or if they just applied to that one campaign.”
Fracking lets out a tiny shriek, and shakes an oily fist at the sky. “Fine! But I’m deducting a minute from your time.”
5. Urgent vs. Regular Headline
“This one works for any social platform! You could do it for a tweet, or a Facebook headline, or the subject line of an email to your friends. You’d just do one version with your regular headline, and then one version with the word ‘URGENT’ in front, and then the same headline. So easy!”
Fracking screams, and his whole body begins melting into the ground before your eyes. “HOW?!” he screams as he melts. “How did you do it?!”
You shrug, and the rope around you falls off.
Maybe the secret all along was that coming up with quick, well-designed A/B testing ideas just isn’t that hard.