Limited Testing Solution, but Not Without Value

No multivariate tests: Google Content Experiments tests on an A|B|n platform. As such, you must create full iterations of landing pages to test against each other. If you are testing a full layout change, the A|B|n model will suit your needs just fine. However, if you’d like to test multiple elements at one time or a small element, like button color, you would need to create a new landing page for each different color button you will test. This can become quite cumbersome.

Basic testing platform: This platform will also present challenges if you would like to test the influence of one element across multiple pages. You will need a much more sophisticated tool for this type of testing.

Content Experiments defaults to a Multi-Armed Bandit algorithm. This automatically adjusts the amount of traffic allocated to each variation based on sample size and performance metrics, resulting in less traffic to "under-performing" variations. In theory the Multi-Armed Bandit is much more efficient than your classic A|B testing method, concluding tests much quicker while reducing potential revenue loss. Unfortunately, we have found that for testing with small sample sizes, like those commonly conducted for B2Bs or SMBs, the multi-armed bandit has the potential to create an invalid test that will never declare a winner. This is because traffic to the variation is typically reduced so severely and so quickly that there is not a significant enough sample size to give it a competitive opportunity.

Testing on a specific segmentation is challenging: To test on a distinct segment of your visitors requires more advanced knowledge of Analytics to set up a new filter view and cannot be completed within Google Content Experiments.

Not ideal for marketers who want independence from IT: Content Experiments requires a snippet of code being placed on one experiment page for each test. It also requires a full version of the test page with a unique URL.