Article under construction
When the scale of transformations are usually considered the bulk of the effort, redirect (or split) tests are often misunderstood as the "easy bit" of the process. Pop in the new URL, click save, and off we go.
But, there's a lot of things that can go wrong. This article explores some of these concepts.
Advanced Consent Mode
Redirect tests on landing pages pre-optin can be a source of great confusion and problems. We have a dedicated article to this problem, found here:
Broken Analytics tracking
TODO
Infinite Loops
Redirect tests can cause users to get stuck in their journey, often seen as an unintended consequence of removing certain steps or pages.
Most commonly, we've seen infinite loops occur in two ways:
Immediate, based on the presence/absence of query strings
URL Rewriting is where websites will manipulate a URL between a user requesting a page and it being delivered to them. e.g. mysite.com/page1 might get rewritten to mysite.com/page-1. Or, mysite.com/?param=value gets rewritten to mysite.com.
Let's take the first scenario - if the test is to "redirect any page to /page1" but it gets rewritte to /page-1, we then believe the user is on a page that needs redirecting, and so they get caught in an infinite loop.
There are two useful things to do in this scenario:
Limit your redirects to N per X minutes. This would limit any potential redirect loops, despite a potential small "glitch" until users hit that limit. Any time you reach this limit, you should fire a conversion metric with as much detail as possible.
Record pages traversed: If the experiment spawns a range of pages, consider logging the URL into custom data. This will help with potential issue debugging down the road.
In the journey, with links back to the redirect
Another unintended consequence we have seen of redirect tests is creating loops in journeys. For example:
User adds a product to their basket page
There would be an addon/upsell page, which we skip if there isn't much to show
The basket page has a link to view addons
The user clicks the link, goes back to the addon page, which the redirect test bounces them away from, thus creating the looping journey.
Good planning will get you past this. It's not enough to skip a page in a journey or move people away from it - it's important to also consider all the places people could arrive at that page.
We see this a lot with homepage redirects too. Redirect the homepage to /v2/, but then all links such as the logo send users back to the old homepage, which requires another redirect to send users to /v2/.
We recommend good knowledge of the website and good planning to get around these issues, and analysing journeys after the fact is typically helpful too.
Also consider more conditional redirects.
Only redirect the page if it's a landing page for the site, i.e. the referrer isn't the current website.
Only redirect the addons page if a user has just added something to their basket
These kind of conditions allow "failsafes" for scenarios you're unlikely to stumble across. And whilst there may be assumptions built into the experiment run, working them into the redirect code itself allows the test to be run a lot more safely. I.e. going from "a lot of people land on the homepage, we would like to redirect it to /v2/" to "the code will only redirect the homepage if you used it as a landing page, sending you to /v2/".
Content flicker
In any situation, content flickering can unwittingly ruin your experiments. This is typically much worse for redirect tests.
Redirect tests are usually adpoted as the changes being made are so big that it's easier to build them into a new page/journey and redirect users to it. If these changes are aesthetic, as they usually are, users seeing the old page and then the new page is likely a very recognisable change.
This is easily solved. The in-built mechanisms of Optimize already handle this for you. But if building the mechanism yourself, be sure to apply masking and make sure users don't see the old page.
Slow page loads to the new page
It's easy to create a bias in page load times between the control and variant, when the typical behaviour for the control variant of a redirect test is "do nothing".
This is often made worse when the new page does not respond or load quickly - while the redirect functionality can request the new page, it doesn't hold any responsibility for how quickly the servers respond. Excessive response times can make the new page look unfairly slow, when perhaps as the page is ready for users more resources are allocated to the functionality, or it is moved onto different servers.
While there isn't much you can do here in terms of the actual response speeds of the new page, you can engineer some fairness and eliminate some biases as part of your process.
Creating a variant control page
One reasonable approach to consider is engineering fairness into the process by degrading the control expreience. Where you may have /v2/ for a new journey, consider allowing the existing journey to be reached through /v1/.
Both the control and variant therefore redirect, and so you are engineering-out any bias in terms of performance being the driving factor for one variant winning over the other.