The Moovweb XDN makes it easy to conduct split testing without performance penalties by executing splits at the edge through an easy-to-use yet powerful interface. It can be used for A/B and multivariate tests, canary deploys, blue-green tests, iterative migration off of a legacy website, personalization, and more. This guide shows you how to configure a split test.

You can perform two kinds of split tests with the Moovweb XDN:

  • A/B test multiple implementations of the same site
  • Split traffic between multiple sites - This is commonly used to test a new experience against a legacy one.

To A/B test mutliple implementations of the same site, simply deploy each implementation to a separate environment, then configure the rules for splitting traffic between using the XDN Developer Console.

To use CI to deploy A/B tests we recommend that you:

  1. Set up separate branches in source control for the main experience and the new experience, for example master and preview
  2. Create environments called production and preview in the XDN Developer Console.
  3. Configure CI to deploy the master branch to the production environment and the preview branch to the preview environment. (Using xdn deploy --environment={environment name})

To split traffic between multiple sites, first add a backend for each site to xdn.config.js. For example, to split traffic between a new experience hosted on origin.my-site.com and a legacy experience hosted on legacy-origin.my-site.com:

// xdn.config.js
module.exports = {
  backends: {
    legacy: {
      domainOrIp: 'legacy-origin.my-site.com',
    },
    new: {
      domainOrIp: 'origin.my-site.com',
    },
  },
}

Then, add a destination for each site to your router. For example,

// routes.js
const { Router } = require('@xdn/core/router')

module.exports = new Router()
  .destination(
    'legacy_experience', // displayed in the destination dropdown in the traffic splitting section of your environment configuration in the XDN Developer Console
    new Router()
      // additional routing rules for the legacy experience go here
      .fallback(({ proxy }) => proxy('legacy')),
  )
  .destination(
    'new_experience', // displayed in the destination dropdown in the traffic splitting section of your environment configuration in the XDN Developer Console
    new Router()
      // additional routing rules for the new experience go here
      .fallback(({ proxy }) => proxy('new')),
  )

Once you have made these changes, deploy your site using xdn deploy --environment={my production environment name}, then configure the rules for splitting traffic between using the XDN Developer Console.

Navigate to the environment in which you want to configure the split test and click "Edit":

edit

Scroll to the Split Testing section and click "Add Rule":

edit

Select the amount of traffic to send to each destination or environment and click "Apply".

edit

You can add additional rules to, for example, allow testers to get to the new experience all of the time by adding a cookie. In addition to cookie value, you can split traffic based on header value, path, IP address, URL parameters, device type, browser type, and bot boolean. Here's an example:

edit

The order of rules is critical. Rules are matched from top to bottom. When handling a request, the first matching rule will be used. Given the rules we've set up in the examples above, we need to move the force-new cookie rule to the top so that it takes precedence since the other rule contains no criteria. We can reorder the rules by dragging and dropping:

edit

To begin the split test, click the "Activate" button at the top of the environment:

edit

To end the split test, you can either deploy a new version of your app with the router destinations removed, or update the environment to send 100% of traffic to a specific destination.

If the XDN is behind a third-party CDN, it is critical that you update the third party CDN to not cache responses from the XDN nor to affect any cookies that begin with xdn_. You can find more details here.

When a split test is active, all users are assigned to a random number between 1 and 100 via a cookie called xdn_bucket. This cookie assignment is done at edge, before the user's first request hits cache, and so there is no performance penalty for new users.

The experience the user sees is determined by the traffic split percentage you set in the environment configuration in the Moovweb Developer Console and on which side of the split the user's xdn_bucket value falls.

When a split test is active, the XDN will automatically set an xdn_destination cookie to the name of the chosen destination. You can use this value in the browser to report the split test experience assignment to analytics.

Moovweb XDN split testing routes traffic at the edge based on a variety of criteria. It does not identify user cohorts (although it can split on cohorts identified by another tool) or report business metrics about A/B tests since there are many great tools for that. We recommend you utilize an A/B testing tool that supports server-side integration such as Monetate, Optimizely, Adobe Test, Google Experiments, or Visual Web Optimizer. These tools will set a cookie or header that can be used to split traffic using the appropriate criteria described above.