How to Prioritise A Conversion Test Roadmap

Rank test ideas by expected value, not by ease. Expected value is the traffic the page receives, multiplied by the size of the effect you expect, multiplied by how confident you are that the effect is real. Effort is a tie-breaker, not the primary sort. Most roadmaps get this backwards and spend a year on cheap tests that could never have moved anything.

Why easy-first fails

Easy tests are easy because they are small. Small changes produce small effects, and small effects need enormous samples to detect. So the easy-first roadmap generates a long queue of inconclusive results and a team that concludes testing does not work.

The base rates make this worse. ConversionTeam’s audit of 2,288 tests found 19.1% reached statistical significance per test. If four out of five tests will be inconclusive anyway, you cannot afford to spend them on ideas that could not have mattered.

The three inputs that decide the order

Traffic. A 5% lift on a page seeing 2% of sessions is noise. The same lift on the product template is a quarter’s growth. Always start from the page inventory, ranked by sessions and by revenue influenced.

Expected effect size. Be honest here. Winners in DRIP’s database produced a median 1.88% conversion lift and 2.77% revenue per visitor lift. If your estimate is 30%, you are probably estimating a structural change, and you should say so, because that changes the sample size you need.

Confidence. Where does the hypothesis come from? A session recording where you watched five people fail at the same step deserves more confidence than a competitor screenshot.

A scoring frame that survives contact with a real team

Sum the first three, then use effort only to break ties. The point is not the arithmetic. The point is that four people can argue about a score in ten minutes rather than arguing about taste for an hour.

What belongs near the top of almost every roadmap

Checkout and cart, because the traffic is already qualified. Pricing, shipping and threshold logic, because those change order composition rather than order count. Mobile, because that is where most traffic and most of the gap live. And post-purchase, because almost nobody has tested it and the downside risk is close to zero.

Test categories vary in how often they win, too. In DRIP’s data, shipping and return communication tests had the highest win rate of any category at 41.8%, against 36.3% overall.

What belongs near the bottom

Button colours. Hero image swaps on low-traffic pages. Anything driven by a competitor screenshot with no reasoning attached. Anything the CEO mentioned in passing that nobody can build a hypothesis around.

Those ideas are not banned. They go in the backlog, and they wait behind ideas with evidence.

Re-rank after every result

A roadmap written in month one and followed unchanged to month six is not a roadmap, it is a schedule. Every result, including the losses, changes what you believe about your customers, which changes the expected effect of everything still queued.

The practical discipline is to re-score only the top five ideas after each concluded test rather than rewriting the whole document. Five minutes, once a fortnight, keeps the order honest without turning prioritisation into a project of its own.

What a well-ranked roadmap can produce

The clearest example we have seen was not a clever test. It was a shipping threshold test at a DTC supplements brand that produced two million dollars in profit. It reached the top of the roadmap because it scored high on all three inputs: high traffic exposure, a structural mechanism, and clear analytics evidence that shipping cost was where orders were being lost.

Nothing about it was ingenious. It was ranked correctly, which is most of the job.

Do this with your current backlog

Score every idea in your queue against the four rows above, then look at the top five. If any of them sit on a page carrying under 5% of your sessions, you have found why last quarter felt slow.

The audit, research and roadmap sequence we use is described at Parah Group.