Learn A/B testing: the beginner guide
From your first test to a routine: hypotheses, duration, typical mistakes. With a free A/B testing course and certificate.
What an A/B test is
An A/B test shows half of your visitors the current version of a page and the other half a changed one. You measure which version brings more revenue or signups, and you decide by data instead of gut feeling.
The point is not the tool but the discipline: from today on, every major website change is backed by a test instead of opinions in a meeting room.
What you need to know before your first test
Four ground rules decide whether your result counts. Significance: a lead of 52 to 48 can be chance. You need a statistically safe margin. Duration: a test runs at least one to two full business cycles, so seven to fourteen days, not until noon. One change per test: if you change layout, copy and price at once, you will not know what worked. Enough traffic: a shop with 200 visitors a day tests big things, not colors.
The workflow in six steps
- Check the data: where do you lose visitors? Analytics and a behavior tool like Microsoft Clarity deliver the candidates.
- Form a hypothesis: "If we change X, then Y will rise, because Z." If you cannot justify the Z, it is tinkering, not testing.
- Build the variant: implement exactly one change cleanly.
- Start the test: 50/50 split, fix the metric before launch.
- Let it run: no peeking in between. Wait the full duration.
- Evaluate and keep: adopt the winner, document the loser. Both results are valuable.
The five most common beginner mistakes
- Stopping early: after two days "the variant is ahead": chance. Whoever decides in between loses in the long run.
- Too many variants: A/B/C/D/E spreads traffic so thin that nothing reaches significance anymore.
- Testing trivia: a button color with 300 visitors a day proves nothing. Test offers, structure, price presentation.
- No documentation: two years later nobody knows why the page looks this way. A test log saves you repetitions.
- Tests without consequence: a result that is not implemented was an expensive toy.
Five test ideas for your shop
- Product page: image before copy or the other way round?
- Cart: show shipping costs earlier or later?
- Checkout: one more step with a summary or one less?
- CTA: a concrete button text instead of "Buy"?
- Trust: reviews next to the price at the top or down in a tab?
The full workflow, from statistics to a testing routine, is in the course A/B Testing Mastery with eight modules and a final exam.