Factorial Test Design - WhichTestWon

Testing Glossary: Factorial Test Design

WhichTestWon's Definition:


A “factor” is any element of a Web page that might be affecting conversions. This would include the layout, images, headline, navigation, etc. If you run an A/B test looking for data on a particular factor, you are limited to changing just one factor on the page at a time. (You can also run an A/B test and test completely different pages, but you would only learn which page won, not how individual factors on the new page made the difference.)

Multivariate tests, on the other hand, allow you to test multiple factors on a page and get specific data on each one’s performance, all at the same time. There are two main type of multivariate tests. Full-factorial tests are, in essence, multi-celled A/B tests, with a test panel for each of the factors and all of its permutations. These can require a lot of traffic to obtain conclusive results, but the data may be worth it. Fractional factorial tests, on the other hand, use complex (often Taguchi-based) mathematics to extrapolate results from a smaller number of test panels. These tests may require less traffic, but some statisticians do not think the results are as conclusive as other types of tests might be.

Click to see Glossary Home with 142 A/B and multivariative testing terms explained.