We use our expertise to answer your question.
The core principle of a conversion rate optimization (CRO) test is comparing the performance of the current state of a website to a change you’re trying to evaluate. Different test methods are just different ways of executing on that. We’ll discuss a few of these test methods below.
A/B test. This is the baseline CRO test method. In an A/B test, traffic is allocated and distributed within the variations of that test.
- Traffic allocation is controlling a percentage of users that go into the test
- Distribution is how the traffic gets split between the variations being evaluated
A/B/n test. This type of test is necessary when you’re looking at evaluating additional variations beyond just A and B. The “n” stands for those additional variations. With this test, you’re distributing traffic evenly between all of those variations.
Multivariate test. This test is used when you’re combining different A/B tests. You’d use this method when you’re testing multiple combinations of changes at the same time.
Multi-armed bandit test. This test involves evaluating multiple variations. However, the test algorithmically adjusts traffic distribution to variations that look like they’ll be better performing.
Painted-door test. With these tests, you’re faking the functionality of a feature to evaluate if it’s worth implementing.
Discovery tests. You can use this test as an investigative method when you don’t necessarily have a set purpose. Still, you want to test multiple different things concurrently to see which channels are better to work with.
Iterative vs disruptive tests. Iterative tests are good for testing degrees of change where you’re improving and iterating on a specific thing repeatedly. Whereas a disruptive test is where you’re changing many things simultaneously.
Superiority or non-inferiority tests. A superiority test is performed when evaluating if the change is explicitly better than the current state. A non-inferiority test is used to assess whether the change you tested is not worse than the original.
