An MVT is a meta-analysis of multiple AB/n experiments in one go. With this, comes a few very different things we can or have to analyse - these will be explained in this article.

This article only focuses on differences from normal AB/n reports - it does not detail every number and chart for you.

For AB/n test, you get a simple output for "Best performer" of conversion rate and chance to beat control.

For MVTs, you get a predicted optimal, which is the best performing combination we have observed/inferred:

Imagine running a big MVT like the one above, where you have 8-10 factors where you test different components. Which one/s influence conversion and which don't?

Without you having to study a table of values, this graph illustrates the principle quickly for you. Taller bars contribute more to overall conversion, and shorter bars contribute less.

So, if you're planning follow-ups, focus on things that drive conversion the most, or be more bold with things that didn't make a difference.

This is similar in principle to Binomial reports for AB experiments in that you see the performance by variation run, but have a few key differences:

This looks at the combinations run, and analyses the value and contribution of every variant that belongs to that combination.

You'll be able to determine, for example, that a combination is +10% conversion rate, and exactly which factor/levels contributed how much to that number.

This is a simple breakdown helping you to understand which variants belong to each experiment (combination).