Advertising is part art, part science, ironically its very own A/B split. At its core, A/B testing is a simple way to compare two different versions of something. It’s been around for more than a century, but it didn’t originate with advertising or marketing. Many give credit to 1920s biologist and statistician Ronald Fisher for creating the scientific and mathematical principles that make the method reliable. As a consumer, you’re participating in testing of all kinds all the time. You just aren’t aware of it. So, what exactly is an A/B split and how does it apply to advertising? Let’s look at the basics.
The Basics of A and B
Let’s start with something very simple. You’re wondering if you should use an email or a letter to your customers. Say you are a credit union with 100 thousand members. To get a sample size with a margin of error of plus/minus 5%, you will need to email and mail to about 400 people. If your email generates a significantly better response than the letter, then you email the rest of your customers. If there’s a small difference, within the margin of error, you essentially have a tie.
Testing Technicalities
There’s a relatively well-known quote, “The only true test is the marketplace.” That’s true, but a test is designed to approximate the marketplace. Things can go sideways when the test itself is imbalanced. For example, if the email and letter were identical except the email had a premium attached. Suddenly you aren’t testing letter vs. email, you’re testing whether the premium worked. If you want to test the delivery system itself, the offer must be the same on both.
Screen Test
The majority of most testing is done online. We test to measure impressions, clicks, conversions, etc. More complex testing almost always comes into play because…well it’s like the reason for climbing a mountain, “Because it is there.” A/B/C, A/B/C/D testing are there. And while A/B testing is a great way to gain an understanding, it’s not as comprehensive as more complex testing. If it’s part of a promotion, part of a design, part of copy, part of almost anything marketing related, it can be tested.
How much testing is right for you? Call us. Let’s find out.