A/B test
An A/B test shows two versions of a page or element in the same period, assigns visitors to them at random and measures which performs better. Random assignment spreads outside effects such as season or campaigns evenly across both sides. The goal and the sample size needed are fixed before the test begins.
- Area
- Data and CRO
- Also called
- split test
- In Turkish
- A/B testi
- Read more · Service
- Conversion rate optimisation (CRO)
Why it matters
A before-and-after comparison is affected by season, campaigns and news. An A/B test shows both versions at the same time, so it allows more confidence that a difference comes from the change. With changes made without a test, it is impossible to tell whether things got better or worse.
Example
Say a shop tests the button text on a product page. Version A says “Add to basket”, version B says “Buy now”. Half of the visitors see A, half see B. After three weeks A has 150 orders from 5,000 visits and B has 175 from 5,000. The difference is then judged with a significance calculation.
Related terms
- Sample sizeSample size is the number of visitors or conversions each version must see for a test to give a reliable result.
- CROCRO is the systematic work of getting more of a site’s visitors to complete the intended action, such as buying or sending a form.
- Conversion funnelA conversion funnel is the ordered view of the steps a visitor takes on the way to a goal, such as product page, basket, checkout and order.
- Conversion rateConversion rate is the share of visits or visitors that complete the intended action: conversions divided by visits.
- HeatmapA heatmap is a visual report that uses colour to show where visitors click on a page, how far they scroll and where they linger.
- Statistical significanceStatistical significance means the probability that a test’s observed difference arose by chance alone is below a threshold set in advance, commonly 5 per cent.
The definitions are general information. If you think one is incomplete or wrong, write to us and we will reply within one working day. Get in touch
From our journal
From what we have written on Data and CRO; they show how the term turns up in the work.
- How to build GA4 audiences and use them in Google AdsFrom conditions to sequences, from membership duration to predictive audiences: the steps for building a GA4 audience and what must be on for it to reach Google Ads.
- How to improve on-site search in an online shopFrom measuring search in GA4 and reading the terms that return nothing, to synonym and typo rules, the results page and the “no results” page, in order.
- How to write a GA4 measurement plan: events and key eventsStart from business questions, choose GA4 events, write down naming rules and limits, mark the key events, and put it all in a one-page table.
- What is server-side tagging, and who needs it?How a server container differs from tags in the browser, what it does and does not solve, who should bother, and the steps of a first setup.