What is server-side testing?
Server-side testing is the broad term applied to any experimentation that doesn't involve Javascript and CSS manipulating the front-end of a website.
Traditionally, it involves your web server making API calls to Webtrends Optimize to fetch decisions. These are used as feature-flags to route traffic, toggle features on/off, etc.
Some decisions should not be able to be toggled by users, such as pricing strategies, payment methods etc., and server-side testing is the approach for handling this.
Since the inception of JS Frameworks like React, Angular and Vue, server-side testing has taken a slightly different meaning. Instead of being server-side API calls, the frameworks need decisions from Optimize and will toggle components based on the responses received.
{
wto_flags.ta_HomepageUSP === "variant" ? <NewBlock /> : <TheOriginal />
}
Where do you need decisions?
Given the above, the first question to consider is where you need decisions. Is it on the server or is it in the Javascript Single Page App?
Testing on the Server
We have two key and distinctive approaches for how we can handle this.
Inform approach
Quite often, we find that users need a decision for when a process on the server executes, but there is a long user journey before that moment. E.g. if you are testing a new checkout, most users will see a page on the website before then such as Homepage, PDP, Basket, etc.
The inform approach here suggests creating an identifier that you can read on the server to toggle functionality.
The identifiers we see most often are:
A cookie, which you can read from the Cookie Headers in the server-side request handling
A hidden form field, added to a form submission event which you will then pick up and handle.
This approach is recommended if you are running very infrequent server-side tests, where the overhead of learning the Optimize APIs would be a deterrent to running the experiments. Or, if you need a very quick decision made.
Fetch approach
This approach assumes full control is required on the server, and you want to fetch decisions on the fly whenever you decide to - the traditional view of what most users consider to be "Server-side Testing".
Firstly, note that you have access to all of the same options, variables and functionality that you'd expect from client-side tests. State, throttling, segmentation, code updates and more all work as you'd expect.
The key responsibility is to use the Optimize Testing Service (OTS) REST API to get your decisions.
You will need to handle all scenarios if you take this approach - both expected tests users fall into, but all of the reasonable scenarios why you may not get a decision, such as:
The test is not live yet, or has been turned off.
The user hasn't met the % throttle
The user doesn't match the segmentation rules
etc.
We do have flexibility in how behaviour/metrics are collected. Choices include:
Pass cookies to the front-end, and the JS tag will collect all metrics in the same way it does with client-side tests.
You can send metrics from the server itself, using the same REST API (different api route and one additional parameter in the request body are needed).
Pros of this approach
Able to do everything - no feature is off-limits.
Able to make the call at the time decisions are needed and not before - less noise by default.
Granular control over when to count users
Cons of this approach
The OTS REST API is very capable, but because of this there is also a learning curve.
Testing in the SPA
We have two key and distinctive approaches for how we can handle this.
Inform approach
This is where the Webtrends Optimize javascript tag (wt.js) is responsible for the fetching of data. We will then "inform you" of decisions made. This reduces effort for your developers to understand and appropriately use our APIs, instead just getting a simple list of end decisions.
How should we inform you?
1. Local/Session Storage
An easy approach is for Optimize to store decisions in Local Storage or Session Storage. The format would be consistent, and you would be able to lift all flags at any given moment.
Further, browsers have "storage" events that you can hook into, allowing you to know the moment updates are delivered, ensuring you always stay in sync.
Your developers can take these decisions, read them into the state of their SPA, and then toggle components easily.
2. Your exposed function for a Context/Store/State
An alternative approach is to expose an "update state" function from your component into the global window object.
Optimize will then still fetch decisions for you, and will fire results directly into your function.
The key benefit of this is that it's fractionally easier to you to handle, with less work to read and intercept storage events. This is not challenging work, but this approach is still a fraction easier.
Pros of the Inform approach
Ease of implementation
No need to learn the Optimize APIs
Can still segment, throttle and manage state.
Can still track metrics and measure performance
Cons of the Inform approach
All decisions fetched are assumed to be entries to the test. Results are most valid if you track the moment it's displayed as a metric (such as Page_Cart) and apply filters at the time of running analysis. All other users are "noise" to the test, and you're less likely to see "significant" results without filtering
Marginal performance effect, given decisions are fetched along with all of your client-side experiments even if they could be fetched earlier.
Fetch approach
This is an identical approach to the server-side approach detailed above.
The key difference is that you will make the API call to Optimize from within the JS application, and not on the server. All other approaches and reasons remain identical.