Multivariate Testing

In UX, multivariate testing compares different versions of your design to find which of these versions performs best according to business KPIs.

Updated on March 15th, 2023

Share

This type of testing requires more traffic to reach your testing goals. For this reason, it is more suited for large companies or websites with a lot of traffic in terms of daily users.

One advantage of multivariate testing, however, is that it can save time, effort, and money compared to other forms of testing. You only have to design the core of your product once and make a few variations to get started.

How do you do a multivariate test?

To do a multivariate test, you first have to determine the variables that you want to test. Here's a list of examples that you could use for your test.

  • The size or color of your main call-to-action button.

  • The copy of your headers.

  • Positioning of elements.

Once you've picked the elements you want to test, you have to create your design variations. For a multivariate test, you test every possible combination of variables. So more variables mean more tests! You can see that in the table below.

This table shows how the number of multivariate tests grow as the number of variations grow.
Variations Elements Tests
2 2 4
3 2 8
4 2 16

Testing the variations works just as an A/B test. You take a few days for each variation to be live and compare the results afterward.

Multivariate testing vs. A/B testing

The concept behind multivariate testing is the same as for A/B testing. You design and test a variation of your design to see which one works best according to a business metric you specify.

The only difference is that you are testing multiple versions of the same page rather than just two. Also, unlike A/B testing, multivariate testing allows you to test multiple variables at once.

For example, if you'd A/B test your landing page, you would test the call-to-action color first and then change the header size. Using multivariate testing, you can try these variables at once by creating multiple variations of all variable combinations.

Missing something? Contribute by making edit suggestions for this keyword.

Useful resources

Multivariate Testing for UX Design - Adobe

Multivariate vs. A/B Testing - NN Group

Share

×

Page link

Want to learn more about Multivariate testing?

Learn together with other UX designers in our community. Join by picking one of the options below.

Monthly

  • Same as annual, but billed monthly.

$5/month

Choose plan

Annual

Save 18% compared to monthly

  • 1️⃣ Join our Discord community

  • 2️⃣ Monthly UX Master Classes

  • 3️⃣ Office hours in the community

$49/year

Choose plan

Lifetime

One purchase. Join forever.

  • Support the development and upkeep of UX Dictionary

$99/lifetime

Choose plan

Once you choose your plan, you'll be redirected to our Stripe page. After that, you'll be redirected to our dashboard. You can cancel, pause, or upgrade at any time.