Marketing is full of decisions. Which headline should go on the postcard? Which photo belongs on the flyer? Should the offer be 15% off or a free gift? You can go with your gut, or you can let your customers tell you. That second option is what A/B testing is all about.
What is A/B testing?
A/B testing, also called split testing, means comparing two versions of something to see which one performs better. You change one thing, send version A to part of your audience and version B to another part, then measure the results.
Say you have two email subject lines and cannot decide between them. Send one to half of a test group and the other to the second half. Whichever gets more opens or clicks wins, and that is the one you use for the rest of your list.
The key is changing one thing at a time. If you change the headline, the photo and the offer all at once, you will know which version won, but not why.
A famous example
One of the best-known A/B tests came from Barack Obama's 2008 presidential campaign. In a 2010 write-up, Dan Siroker, the campaign's director of analytics, described testing different combinations of buttons and images on the website's sign-up page. The winning version, a family photo with a "Learn More" button, raised the sign-up rate from 8.26% to 11.6%. Siroker estimated that the extra sign-ups translated into roughly $60 million in additional donations over the campaign.
You do not need a national campaign to benefit. The same idea works for a local business with a few hundred customers.
What can you test?
- Headlines: a question versus a bold statement.
- Offers: a percentage discount versus a dollar amount or a free add-on.
- Images: a product photo versus a photo of people using it.
- Calls to action: "Call today" versus "Visit our website" versus "Scan to save."
- Design: colors, layout or the size of the offer.
Yes, you can A/B test print
Many people think A/B testing is only for websites and email, but it works great for printed marketing too. Here is how to test a direct mail piece:
- Pick one thing to test. For example, two different headlines or two different offers.
- Split your mailing list in half at random, so both groups are similar.
- Print two versions that are identical except for the one thing you are testing.
- Give each version a way to track responses. Use a different promo code, a different QR code, a separate landing page web address, or even a different phone extension on each version.
- Count the responses from each version and compare.
Variable data printing makes this especially easy. Instead of running two separate print jobs, the press can change the headline, offer, image or code from one piece to the next in a single run. That makes it simple to split a mailing, and even to personalize pieces with each customer's name.
Tips for a fair test
- Use a big enough group. A handful of responses can be luck. The more pieces in each group, the more you can trust the result.
- Send both versions at the same time so the weather, holidays or the news do not tilt the outcome.
- Decide what success means up front: calls, coupon redemptions, website visits or sales.
- Keep testing. Each winner becomes the new version to beat next time.
Small changes, real results
A/B testing takes the guesswork out of marketing. Instead of debating which design looks best, you get real answers from real customers, and over time those small improvements add up.
How AC Graphix can help
We can help you design two versions of a postcard or flyer, set up tracking codes, and print and mail them so you can see what works. Learn more about our printing and direct mail services, or contact us to plan a test.
Frequently asked questions
What is A/B testing in marketing?
A/B testing compares two versions of a marketing piece that differ in one way, such as the headline or offer, to see which gets a better response.
How do you A/B test direct mail?
Split your mailing list in half at random, print two versions that differ in one element, give each version its own promo code, QR code or landing page, and compare the responses.
