A/B tests

An A/B test tells you which of two versions of your content AI engines cite more. You pick two pages — or one page you changed on a known date — and the questions the page should win. Rixe then counts, in the answers it already collects every day, how often each version is cited, and declares a winner only when the difference is clear.

Use it when you want a clean answer about one change. Impact measures every change you make, but when several changes overlap it cannot separate them; a test can.

#Where to find it

Actions › Action plan, Impact tab, then A/B tests (https://app.rixe.ai/actions?tab=impact&view=ab).

#Two kinds of test

#Starting a test

1. Under New A/B test, enter a Test name.
2. Under What to compare, choose Two pages (A vs B) or One page, before and after a change.
3. Enter the addresses: Page A (address) and Page B (address) — they must be different pages — or the Page address and the Date of the change. Before that date is A, after it is B; both periods are the same length.
4. Choose the Kind of change in B (page edit, content published, technical fix, source listing, other). A finished test becomes evidence for the forecast of that kind of work.
5. Choose How long it runs: 14, 30, 60 or 90 days.
6. Tick the Questions to measure on. They must be questions you already track in this workspace; if you have none, add prompts first.
7. Press Start test. The first result appears after tonight's measurement.

#How the winner is decided

For each side Rixe computes the share of answers that cite that page, and the difference B − A with a 95% interval.

• If the whole interval is above zero, B wins, the difference is significant.
• If it is entirely below zero, A wins, the difference is significant.
• If it straddles zero, Cannot separate them yet.
• With fewer than 20 answers on a side, there is no verdict at all: Too few answers so far.

For before/after tests, the result says Compared with your other questions when a comparison group was used, or No comparison group: other changes in the same weeks are not removed when the workspace had no other questions to compare with. Treat the second kind with more caution.

#Reading a test

Each test card shows:

• its kind, status (Running, Ending tonight or Finished) and period, or the change date;
• Questions: N and Answers: N;
• the verdict badge;
• Cited: A …, B … (or Cited: before …, after …);
• B − A: N pts (low to high) — the difference and its interval;
• How the winner is decided, a short explanation of the method for that kind of test;
• Measured with the date of the latest result.

End now stops a test early; Delete removes the test and its result.

#Tips

• Pick questions the page can realistically be cited for. A page about pricing will not be cited on "best tools for…" questions, and the test will sit at Too few answers so far.
• Prefer engines that cite sources. Engines that answer without searching return no citations, which means fewer answers that count.
• Change one thing. If B differs from A in five ways, a win tells you B is better, not why.
• Keep the default 30 days unless you have many questions; shorter tests often end without a verdict.

#Limits and plans

A/B tests are in every plan and use measurements Rixe already collects, so they cost nothing extra. A test runs on up to 200 questions. See Plans and limits.

#Related

• Impact
• Forecast
• Content
• Prompts