When you look at many metrics in one report, the chance that at least one shows a “significant” result by luck goes up. FWER corrections adjust the significance threshold to keep that risk in check, so you can trust a winning result even when you’re tracking lots of conversion points.
What is it?
Each metric and variant you compare is a separate statistical test. At 95% confidence, each test has a 5% chance of a false positive, and across 10 metrics the chance of at least one false positive is much higher.
A Family-Wise Error Rate (FWER) correction controls the chance of any false positive across the whole report. You can now choose one of three standard methods:
Method | How it works | Use it when |
None (default) | No adjustment: each metric uses your confidence level | You focus on one key metric |
Bonferroni | Divides the significance threshold equally across every metric and variant compared | You want the strictest protection |
Holm-Bonferroni | Checks results from most to least significant, relaxing the threshold step by step | You want strong protection with a little more sensitivity |
Hochberg | Checks results from least to most significant; once one passes, all stronger ones pass too | You want the least strict of the three corrections |
With a correction turned on, it’s harder for a result to reach significance, so the significant results you see are more reliable.
Set a default for your account
Step 1. Go to Account Overview and find the FWER correction card in the settings.
Step 2. Click the edit icon, choose a Method and click Save. This becomes the default for every report in your account.
Change the method for a single report
Step 1. Open a report. The significance results use your account default.
Step 2. Click Settings and open the FWER Correction tab.
Step 3. Choose a method and click Apply. Significance is recalculated straight away. This only changes the report you’re viewing.
Good to know
FWER corrections apply to live ABn and Split experience reports. They’re not shown on staging reports.
Corrections aren’t applied while a dimension is selected on the report.
Your confidence level still sets the starting threshold. The correction tightens it from there.