The term “A/B testing.” If you’re new to working at a web advertising agency or have just started handling web advertising in-house, this might be the first time you’ve heard it, right?
In the online advertising industry, “A/B testing” is a crucial concept for improving the effectiveness of ads and websites.
So, in this article, we’ll explain everything from the benefits to the methods and real-world examples of “A/B testing” in a way that’s easy to understand, even for those hearing the term for the first time.
What Is A/B Testing in Web Advertising?
First of all, A/B testing is basicallyCompare multiple images and titles across two ads or websites"Which one was more effective?"This refers to the method for checking [...].
In addition to images and text,Demographics (age, gender, and region)やDistribution Channels、Page LayoutWe sometimes conduct A/B tests on it.
In addition, when there are a large number of impressions, A/B testing is often conducted with three or more variations to increase the number of tests.
Even when there are three or more options, it is generally referred to as an A/B test, though some people refer to a three-option test as an “ABC test.”
Benefits of A/B Testing
With advertisements such as TV commercials and magazine ads, it’s impossible to know exactly how many purchases resulted from them, and once an ad is placed, it cannot be changed.
However,On the Web,、You can accurately track page views and clicks, and make changes to your ads immediatelyThat's it.
There are two main benefits to this.
You can boost the effectiveness of your ads in a short period of time
The first one is"You can achieve results with your ads in a short period of time"That is the point.
TV commercials and magazine ads have fixed publication periods, and as mentioned above, the ads cannot be changed.
However, when it comes to online advertising,You can easily toggle ads "on" or "off."
That’s why, once we’ve gathered enough data, we can change the ad titles and banners to quickly put the PDCA cycle into action.
Testing requires less effort
When implementing the PDCA cycle for TV commercials and magazine ads, each iteration—from the title and dialogue to casting and design—takes time, resulting in significant advertising costs.
Of course, on the web as well, changing a site’s page layout or implementing a new system takes time, but in most cases, it’s just a matter of A/B testing images and text.
Focus the test solely on specific detailsAs a result, each test requires fewer resources, allowing us to cycle through the PDCA process more quickly.
How to Conduct A/B Testing for Web Advertising
Next, here's how to conduct A/B testing for web ads.
Decide what to A/B test
First, think about which metrics you want to improve for the web ads you’re in charge of.
A key indicator here is"imp × CTR × CVR": The Golden Rule of Web Traffic AcquisitionThat's it.
For example, let's say you're running display ads.
If you want to increase the number of clicks to the landing page relative to the number of ad impressions, you need to improve the CTR of your ad creatives.
In other wordsA/B Testing of Ad Headlines and Bannersinclude the following.
Also, if you want to increase the number of conversions from landing page visits, you’ll need to improve the landing page’s conversion rate.
In other wordsA/B testing of designs such as the LP's first view and the "Proceed to Cart" buttoninclude the following.
As such, in advertising,Depending on which aspect of “imp×CTR×CVR” you want to improve,The location for A/B testing may changeThat's right.
(By the way, right after we start running ads, we begin with A/B testing to improve the CTR of our creatives.)
Use a tool to determine the sample size required for A/B testing
To conduct an A/B test"The results aren't just a fluke."A certain number of samples is required to demonstrate this.That's it.
And the sample size for web advertising is"Page Views"will be.
Technically, you’d have to perform some fairly complex statistical calculations to determine the sample size, but it’s a hassle to do that every time you run an A/B test, isn’t it?
Here’s a website that can help with that: “Calculating Sample Size for A/B Testing"."
Simply enter your current CVR (or CTR) and your target CVR (or CTR), and the tool will calculate the required sample size for you.
For example, if the current CTR for a banner ad is 0.05% and you want to reach 0.10%, you’ll need 36,174 page views.
By the way, you can leave the significance level α and power 1-β at their default values of 5% and 95%.
We achieved good results and will conduct another test.
When conducting A/B testing, it is important to follow the PDCA cycle.
For example, if you’re A/B testing three ad copies, keep only the one that performed well and replace the two that didn’t.
Then we’ll run another A/B test, keep the one that performed better out of the two, replace one of them, and...
The ideal scenario is to repeat this process until you have three highly effective ad copies running simultaneously.
As such,A/B testing isn't a one-time effort; we continue to follow the PDCA cycle.
However, the effectiveness of advertising is not permanent; it tends to gradually decline over time.
For example, it is said that sponsored articles lose their effectiveness within three months, so regular PDCA cycles are necessary.
In other wordsAs long as the ad campaign isn't over, we'll keep running A/B tests indefinitely.That's right.
Specific Examples of A/B Testing
Finally, I’ll wrap up by explaining a specific example of A/B testing for web ads.
Examples of A/B Testing for Creative Assets
When it comes to creative assets, A/B testing primarily involves images and text; to be more specific, even a slight change in the overall color tone of an image or a single character in the text can affect the CTR.
Image
For example, when testing three ad banners for skincare products
Comparing product images, images showing the product in use, or images of women, or
If you liked the product image, you could compare it with a different one, or
Compare the layouts, for example
There are various ways to get creative.
Also, when testing two ad copies,
Comparing two selling points (factors that boost the desire to buy) or
Compare based on whether or not there are offers (such as discounts or special perks), etc.
You can test various things here as well.
LP A/B Testing Case Studies
For landing pages, we primarily conduct A/B tests on elements such as “FV (First View),” “SV (Second View),” “Call-to-Action,” “Layout,” and “Cart Button” to increase the conversion rate.
For example, comparing the first view using images or videos,
Comparing the designs of the "Go to Cart" buttons, for example
It would be a good idea to test various options.
Summary
For digital advertising managers, maximizing the return on investment (ROI) of their ads is a critical factor.
To that end, you need to discover winning creative ideas on your own.
If you're planning to get involved in online advertising, be sure to use A/B testing to improve your ad performance.
This article summarizes the different types of online advertising for those who are new to the field. We hope you find it helpful.
