Skip to main content

Multi Armed Bandits

Thompson Sampling & Epsilon Greedy approaches

O
Written by Optimize Team

Multi-Armed Bandit (MAB) automatically sends more traffic to your best-performing variant while your experience runs. You can now choose a second approach, Epsilon Greedy, and optimise for a numeric value such as order value as well as conversions.

What is it?

MAB rebalances your traffic regularly, based on how each variant is performing against the KPI you choose. There are now two approaches:

  • Thompson Sampling: shares traffic in proportion to each variant’s chance of being the best. Traffic shifts gradually as evidence builds.

  • Epsilon Greedy: you set an exploration value, ε. That percentage of traffic is shared equally between all variants, so every variant keeps being tested. The rest goes to the variant performing best right now.

Example: with ε = 30% and three variants (Control, V1, V2), each gets 10% of traffic. The remaining 70% goes to whichever variant is currently performing best. If two variants are tied for best, the 70% is shared equally between them (Control 10%, V1 45%, V2 45%).

With Epsilon Greedy, you can also choose what “best” means:

  • Conversion: the highest conversion rate for your chosen KPI

  • Numeric: the highest average value of a numeric data variable, for example average order value

How to set up Epsilon Greedy

Step 1. On the Experiences screen, expand your experience’s row and open the MAB tab.

Step 2. Turn on Multi-Armed Bandit.

Step 3. Under Approach, choose Thompson Sampling and a KPI, or Epsilon Greedy and enter your ε value %: the share of traffic spread equally across all variants.

Step 4. (Epsilon Greedy only) Choose a KPI Type and your KPI:

  • Conversion: MAB favours the variant with the highest conversion rate for the KPI you choose.

  • Numeric: also choose a numeric data Variable, for example order_value. MAB favours the variant with the highest average value of that variable. The list shows the numeric variables set up for your account or for this experience.

Step 5. Click Apply. MAB starts rebalancing traffic at its next scheduled update.

Good to know

  • Choosing ε: a lower value sends more traffic to the current leader. A higher value keeps testing the other variants more evenly. 10–30% is a common starting point.

  • If Numeric is unavailable, the experience has no numeric data variables yet. Add one in the experience’s Variables section.

  • Every MAB change and rebalance is recorded in the Audit Trail.

  • You can switch between approaches, or turn MAB off, at any time.

Did this answer your question?