Top 4 A/B Testing Myths BUSTED

Top 4 A/B Testing Myths BUSTED

Best Practices

It’s no secret that we’re big fans of A/B testing around here.

But maybe you’re not so sure about A/B testing, or you have a coworker with some questions about it? We’ve listed four common myths about A/B testing, and what the truth really is for each one.


Myth #1: You and your supporters like exactly the same stuff.

You like the environment. Your 80-year-old supporter Dolores likes the environment. You guys have a lot in common already! So maybe she’s been retired for 20 years, and lives in Fort Meyers, Florida, and subscribe to Birder’s World. And maybe you’re 24, and live in Brooklyn, and get most of your news from Vice’s Snapshot stories. But if you think something’s funny, she’ll definitely think it’s funny too, and then share it with their Facebook friends. You do both like the environment, after all.

The Truth: Okay, even if you can guess what will make Dolores share on social media, what about your thousands of other potential supporters? It’s tough to know what other people will respond to. But A/B testing means that you can actually see what types of messaging your supporters prefer. Then you can use that information to create even better messages to test in the future.


Myth #2: Only big organizations with huge digital teams are doing it.

Yeah, there are organizations that are using A/B testing to improve their messaging, but it’s only possible if you have a huge budget and a ton of staff members who can invest the time and money that you need.

The Truth: It’s true that big organizations like the AFL-CIO, Planned Parenthood, and Greenpeace are doing A/B testing on their social media sharing with ShareProgress. But smaller organizations like Change Kansas, Americans for Tax Fairness, and the Story of Stuff Project have also run tons of A/B tests using ShareProgress tools. A/B testing lets those smaller orgs figure out what kind of messaging works best for their supporters, and expand the reach and effectiveness of their communication–all without paying for ads.


Myth #3: Your engagement rates are already good, so there’s no room for improvement.

If you already have a high social engagement rate, there’s a limit to how much you can improve it. Sometimes you’re like, “What if more of our supporters were using their position as trusted messengers to share our message with their extensive social networks?”

And then you’re like, “Nah, our social share rates are already great.”

The Truth: Even if you have a good social share rate, you can still expand your reach through testing. There’s no limit to how much your content could be shared. If your social sharing rates are already good, you have a better chance of making you content go viral by giving it an extra boost through testing.


Myth #4: A/B testing is complicated, and requires a lot of statistical experience.

The Truth: You don’t need to be an expert in statistical significance to get started with A/B testing. ShareProgress makes A/B testing simple, with no advance knowledge needed. You type in a couple of ideas for Facebook headlines, click save, and voila, you’re running an A/B test! You can start simple with just a couple tests, and build up to a robust testing program. We’ve even written instructions on how to create a culture of testing at your organization. There are tons of different ways to jump into A/B testing the different components of your website and campaigns–it’s time to get started!


Written By

Anna Schmitz