Home › Guides › Facebook A/B testing
Facebook A/B testing: how to run a test you can trust
Facebook A/B testing splits your audience at random so nobody sees both versions, then names a winner. Build it from A/B Test in the Ads Manager toolbar or in Experiments. Change one variable, run it 7 to 30 days, and choose the key metric with care: Meta crowns the lowest cost per result for that metric.
Most "tests" I find in ad accounts are two ad sets running side by side, with someone switching off the weaker one after three days. That is not a test. Both ad sets compete for the same people, and the early numbers mostly reflect who happened to see which ad first.
This guide covers Facebook A/B testing the way Meta built it: the set-up, the rules that make a result valid, and the key metric that makes the winner bring customers, not cheap form-fills. Platform details come from Meta's help pages, checked on the day of writing.
What Facebook A/B testing actually does
A Meta A/B test shows each version to a separate, random part of your audience so nobody sees both, then judges which did better. Meta's page about A/B testing for Meta Ads describes it as comparing versions that differ in the image, the text, the audience or the placement.
The split is the point. Meta advises against informal testing, turning ad sets on and off by hand, because it causes inefficient delivery, overlapping audiences and unreliable results.
Creating a test costs nothing extra; you pay for the ads that run. Meta's lift studies are harder to reach: Brand Lift needs USD 120,000 of spend in the last 90 days (about AED 440,000), and Conversion Lift needs a high-quality conversion source such as the Conversions API. For most UAE businesses, the A/B test is the tool you will use.
Before you build it: one hypothesis, one variable
Meta's best practices for A/B tests start with two rules: test one variable only, and turn your question into a measurable hypothesis. "Will video do better?" is a question. "A walkthrough video will get a lower cost per qualified lead than the carousel" is a hypothesis, and its result tells you what to do next.
- Creative. Images, videos, text or formats, set at ad level.
- Audience. A custom audience against interest targeting, or different regions, set at ad set level.
- Placements. Automatic placements against a hand-picked set.
- Custom. Several changes across campaign, ad set and ad at once.
Custom has a cost: Meta says that with more than one variable, you won't know which one produced the winner. Keep it for comparing two complete set-ups.
How to set up an A/B test in Ads Manager
Meta generally recommends building an A/B test from the Ads Manager toolbar, which uses an ad you already have as the template. The steps from its page on how to create an A/B test in Ads Manager:
- Click A/B Test in the Ads Manager toolbar.
- Choose how to build it. Make a copy of this ad duplicates your ad set with one variable changed. Select two existing ads compares campaigns or ad sets you already run.
- Pick the variable, or the ad sets to compare (Add Ad Set brings in more).
- Name the test and choose how the winner is decided. You can add extra metrics for context.
- Set the test schedule. It is separate from your campaign dates, so make sure the campaigns run for the whole test.
- Review and publish.
If the labels on your screen differ slightly, the route is the same: toolbar, variable, winner metric, schedule.
The Experiments route, for campaigns already running
The Experiments tool compares up to 5 existing campaigns, or a duplicate of one, and suits campaigns that are already live. Meta's steps to create an A/B test in the Experiments tool: open Experiments, select Create test, click Get started under A/B Testing, then choose Test another campaign or Duplicate an existing campaign. A campaign already in another test, or on the reservation buying type, cannot be used. Results from every finished test, whichever route you used, appear in Experiments.
Budget, audience and schedule
Three rules from Meta's best-practices page decide whether a test result means anything: the schedule, equal budgets and a clean audience.
- Schedule. Tests shorter than 7 days may be inconclusive. If customers usually take more than a week to convert after seeing an ad, run it longer, such as 10 days.
- Budget. Equal for each version. If you duplicate a campaign to build the test, Meta duplicates its budget too, so check total spend before you publish.
- Audience. Large enough, and not targeted by any other campaign during the test. Meta says overlap can cause delivery problems and contaminate the result.
Meta gives no minimum budget, only one that produces enough results to pick a winner with confidence. So do the sum first: daily budget per version, times test days, divided by your usual cost per result. As an illustration only, AED 150 a day per version for 10 days at a cost per lead near AED 75 gives each version about 20 leads. That can show a difference in cost per lead, but it is far too few to judge which version brings more sales.
If each version would get only a handful of results, test a bigger difference or wait until you can fund it.
Pick the key metric before you start
Meta decides the winner by comparing cost per result, where the result is the event you selected when creating the test. Its page on how winning campaigns are determined gives cost per click and cost per purchase as examples.
This is where lead-gen tests go wrong. With Leads as the key metric, the winner is the cheapest form-fill, which is often a short, prefilled form that sends your sales team people who never pick up.
- Choose the deepest event you can measure in the test window: a qualified lead, a booked viewing or consultation, or a purchase.
- Add cost per lead as an extra metric, so you see it without letting it decide.
- Check the CRM before you act. Compare how many leads from each version were reachable and qualified.
If you send lead stages back to Meta through conversion leads, you already have a deeper event to test against. If not, set that up before running lead-gen tests that matter.
What I see in UAE accounts: creative tests judged on cost per lead, a winner declared, budget moved, and a sales team quietly complaining that lead quality fell. In real estate and clinics, where one sale is worth far more than any single lead, I judge tests on qualified leads from the CRM. Cost per lead is the tiebreaker, not the verdict.
Reading Facebook A/B test results
When a test ends, Meta emails you and marks the best version in Experiments with a green trophy and a confidence percentage. Meta simulates the possible outcomes tens of thousands of times; the percentage is the chance of similar results if you reran the test. High confidence is a reason to switch off the loser and move its budget. Low confidence means "no clear difference", not a result.
If no winner was declared, Meta's tips for improving your A/B tests name three causes: audiences too small (each version gets only part, so broaden more than usual), budget too low, or versions too similar. Its example of too similar is women aged 18 to 20 against women aged 20 to 22 who had both shopped on your site.
Then build on it, for example by testing creatives within the winning audience. Meta also warns that a winner on one campaign objective may not win on another.
Mistakes that break a test
Most failed tests I see were broken after launch, not at set-up. The traps, from Meta's pages:
- Editing the variable mid-test. Once a test is running, you cannot change the variable being tested; in a creative test, that means the headline, text and other ad content. Make changes while it is In review or Scheduled.
- Cancelling and assuming the ads stopped. Cancelling a test does not stop its campaigns, including ones created for it. Pause them in Ads Manager or they keep spending.
- Changing the schedule. Moving a running test's dates weakens the results collected so far.
- Calling it early. Meta says a winner can still cost more when the test was too short or had too few results.
Watch the learning phase too. A duplicated ad set is new, so it enters learning, where Meta says ad sets are less stable and usually cost more per result. Against an ad set that has run for months, it starts at a disadvantage (more in my guide to the Meta Ads learning phase).
What to test first in a UAE lead-gen account
Start with tests that could change your cost per qualified lead. These are hypotheses, not results:
- Lead route. An instant form against click-to-WhatsApp ads, judged on leads your team actually reached.
- Form friction. A higher-intent form with qualifying questions against a short one. My guide to Facebook instant forms covers both.
- Offer angle. For off-plan property, a payment-plan message against a location message. For clinics, a treatment angle against a price-led one, within Meta's health ad rules.
- Language. English against Arabic creative for the same audience, both well made.
A new button colour or one changed headline word is rarely big enough for a test to detect. To see which offers and angles other UAE advertisers lead with before you write a hypothesis, check the Facebook Ad Library.
The test order
- Fix measurement, so the test can be judged on qualified leads or sales.
- Write one hypothesis with one variable and the metric that proves it.
- Do the budget sum: enough results per version within 7 to 30 days.
- Build it from the toolbar or Experiments, with equal budgets and no overlapping campaigns.
- Leave it alone until it ends.
- Check confidence and the CRM, then move budget and plan the next test.
One clean test a month teaches you more than a dozen side-by-side ad sets. If you'd rather hand that over, it is part of my Meta Ads management for Dubai businesses, along with budget structure such as CBO vs ABO.
Frequently asked questions
How long should a Facebook A/B test run?+
How many versions can a Meta A/B test compare?+
How does Meta decide the winner of an A/B test?+
Does cancelling an A/B test stop my ads?+
Related
Not sure your tests are telling you the truth?
Message me on WhatsApp or request a free campaign audit. I'll look at how your account tests creative and audiences, check what your tests count as a result, and tell you which test is worth running first.