Skip to main content

Time to Completion

The time to completion component appears in the UI for live ABn and Split experience reports. It helps you to calculate how long to run your experience to reach completion and produce a reliable result.

D
Written by Dan Bier

When Time to Completion appears in your report

Time to Completion is only shown when the projection can be calculated meaningfully. It will appear in your report when all of the following are true:

  • The experience type is ABn or Split.

  • The experience is currently live (not paused, archived, or in staging).

  • The experience has been live for at least one hour, so there is enough data to base a projection on.

  • A Key KPI has been selected for the report.

  • Conversion data is available for that KPI.

  • No filters or dimension breakdowns are applied to the report - projections are based on the full audience, so they are hidden while the report is segmented.

If you don't see Time to Completion in a live experience report, check that a Key KPI is selected and that no filters or dimensions are active on the report.


Understanding the inputs

There are two inputs you can adjust: Power and MDE. Both control how cautious the projection is. Tightening either one will increase the estimated time to completion; loosening either will decrease it.

Power (%)

Power is your tolerance for missing a real effect - sometimes called the experience's sensitivity. A power of 80% means that if there genuinely is a difference between your variations of the size you specified, there's an 80% chance your test will detect it.

  • Higher power (e.g. 90%, 95%) → fewer missed wins, but the test needs more traffic and takes longer to complete.

  • Lower power (e.g. 70%) → faster results, but a higher chance of overlooking a real improvement.

The standard industry default is 80%, which we use as the starting value.

MDE (%) — Minimum Detectable Effect

MDE is the smallest difference between variations that you care about being able to detect. If you set MDE to 5%, the experience is being asked to reliably distinguish a 5% lift (or drop) from random noise. Smaller effects are harder to detect and require more traffic.

  • Lower MDE (e.g. 1-2%) → the test can detect very small wins, but needs more traffic and time.

  • Higher MDE (e.g. 10%+) → results arrive sooner, but only meaningful if you genuinely don't care about small changes.

Choose an MDE that reflects the smallest lift that would actually change your decision. If a 2% improvement would lead you to ship the variant, set MDE around 2%. If you only care about clear, large wins, a higher MDE is appropriate.

Note: if the experience's data already shows a large difference in conversion rate between control and variant, changing MDE may not affect the remaining time to completion. Once the observed lift exceeds the lift implied by your MDE, the calculation uses the observed lift instead - so MDE only impacts when it would represent a larger, harder-to-detect effect than what's already in the data.

Applying changes

After adjusting Power or MDE, click Apply to recalculate. The chart updates to show the percentage of required views each variant has reached toward completion at your chosen settings.


Reading the result

  • Time to Completion: X days - the projected number of additional days, at current traffic and conversion rates, until every variant has accumulated enough views to detect a difference of your chosen MDE with your chosen Power.

  • "--" - shown when a projection cannot yet be calculated, typically because the test has not collected enough data.

  • Chart - shows how close each variant is to having enough views to be evaluated reliably. A variant at 100% has reached the required sample size; one at 50% needs roughly twice as much traffic to get there.


Tips

  • Don't shorten Power or MDE just to get a smaller number. The projection is a planning tool, not a target. Loosening the settings doesn't change reality - it only changes when you stop the test, which can lead to acting on results that aren't real.

  • Expect the estimate to move. Early in a test, the projection swings as the conversion rate stabilizes. Treat early estimates as rough, and revisit them after the first week.

  • Traffic matters as much as time. If your test has uneven traffic across variations (for example, after a split adjustment), the slowest variant determines the overall completion time.

Did this answer your question?