So you have built your experience, you are happy with how it looks but does it work as intended? Does the experience display correctly and does the original functionality still work? These are the sorts of questions QA will cover off before you set something live.
How do I QA my experience?
There are different ways to QA your experience, we will talk through the options below. Ultimately you are ensuring any changes you have made do not have a negative impact on the page you have made the changes too, its also to ensure the new changes look and feel correct and any new functionality is doing what's expected.
If you are looking for what to 'look out for' when QA'ing your experience, use this guide to find out more.
Methods for QA
Preview links
Preview links are the simplest way to QA an experience however they are also quite limiting in what they tell you. You can access preview links from the 'Experiences' screen (using the eye icon) and you will see the experience open in a new tab:
Limitations
For tests:
Able to preview a specific variation - yes
Able to check Segmentation works - no, the check is skipped
Able to check Location works - no, the check is skipped
Able to check if Metrics track - no
For targets:
Able to preview a specific variation - yes
Able to check if Segmentation works - no, the check is skipped
Able to check Location works - no, the check is skipped
Able to check if Metrics track - no
The preview link creates a preview of the page for you however it won't take into account other tests live on the page nor can you interact with the page to understand if everything is working correctly.
Force experiment tool
The force experiment tool is the recommended way for you to preview and QA your experience. This is a bookmark which you can add to your browser and will allow you to force your way into different tests and targets to QA them.
If you are trying to install the force experiment tool, please refer to this article.
What can I check with the force experiment tool?
For tests:
Able to preview a specific variation - yes
Able to check Segmentation works - yes
Able to check Location works - yes
Able to check if Metrics track - yes
For targets:
Able to preview a specific variation - no
Able to preview if there's only 1 variation - yes, as long as you meet the segmentation criteria set for that rule if any.
Able to check if Segmentation works - yes
Able to check Location works - yes
Able to check if Metrics track - yes
Using the force experiment tool
Once installed, the tool can be used on any page where there is an experience pushed by WTO and you have a range of tools to use to QA things effectively. Once navigated to the page on your site where you want to preview your experience, you will open the force experiment tool and it will look a little like this:
From here there are some basics to understand before we can move forward and we will cover the top toolbar which gives us several options.
Search - you can search for an experience if there are lots running on this page/location.
Status - this refers to, and filters by, the state of the experience, is it live? or in staging?
Test type - you can filter down to specific experience types here such as ABn or a target.
Staging toggle - this changes the mode from live to staging. Discussed more below.
Refresh list - this will refresh this list in case you are making changes in real time.
Clear all cookies - you can clear all cookies and this will remove you from any experiences you have been entered into (we use cookies to identify specific things relating to experiences).
Column definitions and capabilities
There are several columns showing different bits of information relating to each experience. Here we talk through each of those.
Name - this is the name of the experience
Experience ID - this relates to the ID in the WTO system
Status - this is the current status of the experience, staging/live etc.
Throttle - this is the % traffic throttle applied
Info - this will give you detail on audience but also fast links into reporting
State - this confirms things are working correctly
Clear cookies - this allows you to clear cookies for this experience only
Staging mode query string and pinning variations
You can utilise a query string to enable staging mode and this is the most basic why to see staging experiences.
All you need to do is add _wt.mode=staging into your query string to identify as a staging user. For example:
becomes
Typically when using this mode, you will be unable to select a variant to fall into so we need to consider using the pinning option. In the experience screen, and on the experience overview (eye icon) we have the option to 'Pin to 100%' and what this is doing is forcing every user (in staging mode) into that specific variant.
Once saved, you'll only be given the experiment you select. This applies to all users, so can help your colleagues too e.g. when sending the test to the QA team.
Remember to set this back to "At Random" once you're done otherwise your true users will be allocated the same variant.
Limitations
For tests:
Able to preview a specific variation - yes, with pinning
Able to check Segmentation works - yes
Able to check Location works - yes, by inference
Able to check if Metrics track - yes
For targets:
Able to preview a specific variation - no
Able to preview if there's only 1 variation - yes, as long as you meet the segmentation criteria set for that rule if any.
Able to check if Segmentation works - yes
Able to check Location works - yes, by inference
Able to check if Metrics track - yes
Staging vs. Live modes
The option to switch between staging and lives allows you to QA experiences on your live site regardless of what state they are in. By switching modes you are able to fall into experiences depending in their state. By default you will be in live mode and will fall into experience relevant to that state. When switching to staging, you are removed from live experiences and shown those in a staging state.
Want to understand more about states? Refer to this article to find out more.
In the following example you can see that we are in the default 'live' mode and there is a test currently in staging mode with 2 variants called 'New Nav'. You can see both variants are blank currently and we are not in this test. This is because we are in live mode, and the test is in staging mode. To be entered into this test we can do one of two things, we can change the mode of the tool from the toolbar or we can click on one of the numbered variations of the 'New Nav' test. By doing either of these things we can see the view changes:
You are able to force yourself into any experience on any page and any changes you made as part of that experience will be shown on the page itself. You can interact with the page too, the experience will remain as chosen in the tool.
Going live
Once you have completed QA you are ready to go live. This may seem daunting at first but its a simple process and the QA should give you confidence to get your experience out to your users.
From the experiences screen you are able to change the state of an experience, in this example you will be changing 'Staging' to 'Live'.
When selected, you are asked to confirm and at this point the experience will begin to be served to your users.
Congratulations, your experience is live and you will soon start to see data flowing into your reporting.