> ## Documentation Index
> Fetch the complete documentation index at: https://crevio.co/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B tests

> Show different versions of a page to different visitors, see which one gets more sales or bookings, and keep the winner.

An A/B test answers a simple question: which version of this page works better? Instead of guessing whether a new headline or a different layout will help, Crevio shows the current page to some visitors and the new version to others, at the same time, and counts who buys, or who takes the step that matters to your business, like asking for a booking.

When one version is clearly ahead, you keep it. Everyone sees the winner from then on.

## Why you'd use this

* **You're not sure a change will help.** A test tells you before you commit to it.
* **You want proof, not opinions.** Real visitors decide, with real orders or requests.
* **You don't want to risk your sales.** Your current page keeps running for part of your visitors the whole time.

## What a test counts

Every test counts one thing, and you choose it when the test is set up:

* **Sales.** A visitor counts when they buy through your site's checkout. This is the default, and the right choice for a shop.
* **Another action.** If your customers reach you through a form, a booking request, or a call button instead of a checkout, a test can count that instead. Tell Crevio what a good visit ends with, like *"count booking requests"* or *"count people who send the contact form"*. If your site doesn't record that action yet, Crevio adds it to the page when it builds the new version, and checks that it works before the test starts.

A visitor counts for the version they saw when they take that step afterwards, anywhere on your site.

## When a test is worth running

A test needs enough visitors and sales, or other actions, to tell two versions apart. Crevio won't name a winner until every version has at least 50 visitors and 5 of them, and a small difference needs many more than that.

If your page gets a handful of orders or requests a month, a test can run for months without an answer. Crevio looks at your traffic before it suggests one, and tells you if your site is too quiet for a test to be useful. In that case, it's usually better to just make the change.

Test one change at a time, like a headline, a price layout, or a shorter page. If you change five things at once and sales go up, you won't know which change did it.

## Ask Crevio to run one

**1. Describe what you want to test**

Tell Crevio which page and what you'd like to try. For example:

*"Test a new headline on my pricing page."*

*"Try a shorter version of my homepage and see if it sells more."*

Say what counts as a good result: a sale, or another action like a booking request or a sent form. For sales, you can also say whether you care most about how many visitors buy, or how much money each visitor brings in. If you don't say, Crevio counts how many visitors buy.

**2. Crevio builds the new version**

Crevio writes the new version into your page, next to your current one. Your visitors keep seeing the current page until the test starts.

**3. Look at each version**

Crevio shows you a picture of every version so you can check them before anyone else sees them. Ask for changes until you're happy.

**4. Approve the start**

Crevio asks you to approve starting the test. Once you do, your visitors are split between the versions within about a minute.

You can test two, three, or four versions of a page, including the current one. Two is best for most small sites: every extra version splits the same visitors into smaller groups, so each one takes longer to reach an answer.

## While the test runs

* **Each visitor keeps seeing the same version.** Someone who comes back next week sees the version they saw the first time, on the same phone or browser.
* **Your page works normally for everyone.** Buttons, checkout, and every other page on your site are unchanged.
* **Ask how it's going any time.** For example: *"How is my pricing page test doing?"* Crevio tells you how many visitors and orders, or other actions, each version has so far.

A running test can't be edited, and that includes what it counts. If you want to change one of the versions, Crevio ends the test and starts a new one, which starts counting from zero.

## Reading the results

Crevio describes each version against your current page with a **chance to win**: how likely it is that this version really does better, not just by luck. A version needs at least a 95% chance before Crevio calls it the winner.

You'll hear one of four answers.

* **Still collecting.** Not enough visitors, sales, or other actions yet to say anything.
* **No clear winner yet.** The versions are doing about the same. Neither is ahead by enough to be sure.
* **Winner.** One version is ahead with at least a 95% chance. This can be your current page, if it beats every new version.
* **The split looks off.** The versions got very different numbers of visitors, more than chance explains. That usually means one version has a problem, so Crevio checks the page before you decide anything.

### Why small sites often see "no clear winner yet"

This is the most common result, and it doesn't mean the test failed. It means that at your current number of visitors, the versions do about the same.

A version that's slightly ahead after 200 visitors can easily be behind after 400. Calling it the winner early is how people end up keeping a change that does nothing. When there's no clear winner, Crevio tells you so and offers two choices:

* **Keep the simpler version.** If they do about the same, keep whichever one you prefer or find easier to maintain.
* **Keep the test running** for a few more weeks to collect more results.

## Shipping the winner

When you're ready, tell Crevio which version to keep. Crevio asks you to approve it, and within about a minute every visitor sees that version.

Crevio then makes the winner a permanent part of your page, puts that live, and ends the test. You don't need to do anything else.

Choosing to keep your current page ends the test the same way, with nothing changed.

## What you approve

Crevio asks you to approve each step that changes what your visitors see:

* starting a test
* choosing the version to keep
* ending a test

Setting up a test, building the versions, and checking the results never change your live site, so they don't need your approval.

## What A/B tests can't do

* **One page per test.** A test covers one page, like your pricing page or your homepage. Testing a change across your whole site, or across a set of pages like every product page, isn't available.
* **One test per page at a time.** Finish the current test on a page before starting another there. Tests on different pages can run at the same time.
* **One thing counted per test.** A test counts sales, or one other action you choose. It can't weigh several actions together, like bookings and sales at once.
* **Revenue only for sales.** Comparing how much money each visitor brings in works only when the test counts sales.
* **Only what happens on that site.** A sale or action counts when the visitor saw the tested page first, on the same browser. Orders from links in emails or social posts that skip your site, and bookings made by phone or in person, aren't counted.
* **Some visitors aren't counted.** Visitors whose browser sends a Global Privacy Control signal always see your current page and aren't counted. Visitors who block cookies may be counted as a new visitor on each visit, which makes the test slower to reach an answer but doesn't favor either version. Tests use the same cookie as [site analytics](https://crevio.co/docs/features/analytics.md#cookies).
* **Your site must be live and public.** Sites that only members or your team can open can't run tests.
* **Chat only.** You start, check, and end tests by talking to Crevio. There's no separate screen for them in your dashboard.

## Troubleshooting

> [accordion] My test has run for weeks and still says no clear winner yet
>
> Your versions are doing about the same at your current traffic. That's an answer too: the change doesn't make a big difference. Keep the version you prefer, or ask Crevio how much longer the test would need at your current pace and decide whether that's worth waiting for.

> [accordion] My test counts bookings but stays on still collecting
>
> Ask Crevio to check that the page records a booking request when someone sends one. If the step isn't being recorded, the test can't count it, however many requests come in. Crevio fixes the page, and the test counts requests from then on.

> [accordion] I only ever see one version of my page
>
> That's expected. You're a visitor too, so you keep seeing the same version every time. Ask Crevio to show you the other versions and it will send you pictures of each one.

> [accordion] My test ended but my page went back to the old version
>
> Ask Crevio to check the page. The winning version should have been made a permanent part of it before the test ended. Tell Crevio which version you chose to keep and it will put it back.

> [accordion] One version got far more visitors than the other
>
> Crevio flags this as "the split looks off". A version that is broken, or much slower to load, loses visitors before they're counted. Don't pick a winner from that test. Ask Crevio to check each version, fix the problem, and start a new test.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.