Marketing your brand is not an exact science. Even if you’re the best strategist in the world with a ton of experience under your belt, your best stuff will still take a lot of trial and error to get right.
That’s why good marketers build trial and error into their strategies, in the form of A/B (or ‘split’) testing.
Here, we’ll take you through what A/B testing is, what it’s used for, and how to do it properly:
What is A/B testing?
A/B testing is a bit like running a competition for your content. The aim is to find out which elements are the strongest.
Let’s say, for example, that you’re sending out an email campaign. To get the kind of engagement rates you want, your emails will need to really stand out in the inbox. This means you’ll need a super-interesting subject line.
You’re thinking of grabbing attention with colourful emojis – like this:
“🌞😎Summer sale starts TODAY! Click for super summer discounts!😊🤗”
But are the emojis a bit much? Do they detract from the message? Or even make it look a bit (*gasp 😲*) tacky?
Would a subject line without emojis get more clicks?
Rather than sending the emoji version out into the big wide world and then gnawing your nails in case you’ve got it wrong, run an A/B test.
Pit element A (emoji SL) against element B (emoji-free SL) with a carefully-selected, split group of customers. The one with the most clicks under these test conditions is likely to perform best when your campaign goes live.
If this still sounds a bit confusing, don’t worry. We’ll be taking you through everything you need to consider for a good A/B test in a bit – so hopefully all will become clear!
For now, remember that an A/B test is basically a ‘competition’ which helps you to determine which out of two possible elements will perform the best with your customers.
What is A/B testing used for?
You can run an A/B test in any situation which needs a decision between two factors of a single variable.
For example, you might test to determine:
- The best colour for a CTA (call to action) button
- To use gifs or not in your newsletters
- Whether or not to use questions in email subject lines
- Design elements, like the colour of your website banner or the best placement for your logo
- Tones of voice (for example, formal or informal)
- Personalisation elements (for example, would a popup addressing visitors by their first name be endearing or off putting?)
A/B testing is a popular form of testing that gives clearly defined results. It’s a great tool, and it’s inevitably used at some point in any marketing strategy.
However, it’s not without its limitations.
Next, we’ll take a look at the advantages and disadvantages of A/B testing:
What are the advantages of A/B testing?
- It gives a clear result. It’s very rare for there to be a ‘draw’, or any other kind of ambiguity in an A/B test. Assuming that you tested under fair conditions (more on that later!) an A/B test will clearly show you which element will perform best.
- It’s very specific. If you’ve got the time and energy, you can get granular with A/B testing in a way you just can’t with other tests. We don’t necessarily recommend spending a ton of time A/B testing every teeny-tiny design element of your site…but you can if you want to! No job is too small or too specific for an A/B test.
- It’s popular, widely available, and well-known. This kind of popularity often comes in handy. For example, if you need to convince a team member, boss, investor, client etc that your idea really is the best, run an A/B test and prove it for them. They’ll respect the results of such a well-known and conclusive test format. What’s more, the test’s popularity means that most ad platforms have A/B testing facilities.
What are the limitations of A/B testing?
- It only tests two elements. This is the big limitation. Sure, you can run an A/B/C/D…etc test (that’s called ‘multivariate testing’, btw, if you want to sound clever). But to get that nice, clear result you’re best testing two clearly-defined elements. If you’re after a less defined answer to a query involving multiple variables, an A/B test is no real help.
- It can be a drain on time and resources. Testing only two things at a time can be a lengthy process. Automations will speed things up if you’ve got a lot of tests to run – but even then you’ll be left with a mountain of result data to sift through and analyze.
- It doesn’t reveal existing flaws. A/B testing can only tell you which of two options is the best. If your content or style is unpopular, A/B testing won’t reveal why. It will just show you the least-worst option.
- It doesn’t take context into account. For example, if you are A/B testing subjects for social media posts and judging by engagement, a post with a lot of negative engagement will ‘win’ against a post with a smaller amount of positive engagement.
How can you avoid these limitations? Well, mostly by using your common sense when running A/B tests…but as common sense is never a given amongst digital marketers (sorry, guys! But you know it’s true), here’s a quick primer on how to do it properly:
How can I run a good A/B test?
- Do your research first.
An A/B test is best for getting the details right. Ideally, you want to be testing between two well-informed choices.
Before you get to A/B testing, make sure that you understand what you want to achieve, why you want to achieve it, the audience you’re aiming for, any existing problems you need to fix, and so on.
Basically, make sure you know what you’re doing and why you’re doing it. Then, use A/B testing to nail the details of how you’re doing it.
- Pick the right variable
We won’t labour this point because we’ve been through it above – but A/B tests are best when they involve one clear variable. The more specific and well-defined, the better.
The above example, in which we used emojis as a variable, is good because it’s simple and obvious. If the emoji-ed version wins, it’s a very safe bet that the emojis were the defining factor.
With a less-defined variable you won’t get that kind of clarity. And clarity is the big strength of the A/B test. Without it, your test is more or less useless.
- Pick the right goal
Let’s say you’re testing CTA colours. At the end of your test, you find that the blue CTA button got more clicks than the orange CTA button – but the clicks your orange CTA got lead to more conversions.
Which should you choose? Well, defining your ideal goal in the first place will help make this decision quicker and easier.
Again, understanding your context will be a big help here. For example, if you already get a lot of traffic and just need to boost conversions, orange is the obvious choice, but if the traffic boost is what you need then go for blue.
- Create fair test conditions
Ideally, both variables should be tested on the same audience at the same time.
Obviously that’s not possible when dealing with a ‘live’ audience, but you can get as close to controlled conditions as possible by segmenting your audience, and then testing on two halves of the same segment.
Pick a segment of customers with similar characteristics (ideally as close as you can get to your eventual target audience). Then split that segment, and run variants A and B to each segment at the same time and for the same length of time.
- Consider the context of your results
Yes, we keep going on about context in this blog. It’s getting boring, we know! But trust us, it is important!
You’ve looked at the overall context of your project before running your tests. Now it’s time to do the same with your test results.
Ask yourself, why did this variable perform better than the other? Was it simply more engaging, or was something else going on?
Usually, the answer will genuinely be that the winning variable was just better than the other (or better for your customer base, at any rate). But scrutinising the results and their context is still worth it. If nothing else, it will help you to understand your customers and their preferences better for future reference.
Need a hand? Call Sox!
If all this sounds like Greek to you (or if you just want a hand getting digital stuff done!), give us a shout. We’re here to help with any aspect of web development, and are happy to give pointers when it comes to things like ads, content, design, testing, and all that other fun digital stuff. Get in touch today, and we’ll have a chat about what you need.