Limited Testing Solution, but Not Without Value
Updated February 04, 2015

Limited Testing Solution, but Not Without Value

Jenny DeGraff | TrustRadius Reviewer
Score 7 out of 10
Vetted Review
Verified User
Review Source

Overall Satisfaction with Google Content Experiments

As a digital marketing agency, we develop and optimize landing pages and websites for internal and client use. To aid in our conversion rate optimization efforts, we need a robust landing and website page testing solution that incorporates existing and new Google Analytics goals.
  • It is free
  • Tests are easy to set up. Experiment creation is a 4-step process that can be completed with limited knowledge of Google Analytics
  • Experiments is integrated with your Google Analytics account, utilizing existing goals or allows you to create new goals for testing directly within the set up area
  • Allows for advanced segmentation, filtering, and traffic allocation
  • No multivariate tests: Google Content Experiments tests on an A|B|n platform. As such, you must create full iterations of landing pages to test against each other. If you are testing a full layout change, the A|B|n model will suit your needs just fine. However, if you’d like to test multiple elements at one time or a small element, like button color, you would need to create a new landing page for each different color button you will test. This can become quite cumbersome.
  • Basic testing platform: This platform will also present challenges if you would like to test the influence of one element across multiple pages. You will need a much more sophisticated tool for this type of testing.
  • Content Experiments defaults to a Multi-Armed Bandit algorithm. This automatically adjusts the amount of traffic allocated to each variation based on sample size and performance metrics, resulting in less traffic to "under-performing" variations. In theory the Multi-Armed Bandit is much more efficient than your classic A|B testing method, concluding tests much quicker while reducing potential revenue loss. Unfortunately, we have found that for testing with small sample sizes, like those commonly conducted for B2Bs or SMBs, the multi-armed bandit has the potential to create an invalid test that will never declare a winner. This is because traffic to the variation is typically reduced so severely and so quickly that there is not a significant enough sample size to give it a competitive opportunity.
  • Testing on a specific segmentation is challenging: To test on a distinct segment of your visitors requires more advanced knowledge of Analytics to set up a new filter view and cannot be completed within Google Content Experiments.
  • Not ideal for marketers who want independence from IT: Content Experiments requires a snippet of code being placed on one experiment page for each test. It also requires a full version of the test page with a unique URL.
  • Utilization of Google Content Experiments has allowed us to consistently conduct more tests, resulting in a 141% improvement in overall conversion rate.
Google Content Experiments provides significantly more insight, historical data and analysis than Unbounce. However, if you do need a solution that offers a WYSIWYG editor, landing page hosting, and limited reporting and testing, Unbounce is a good all-in-one solution and that can incorporate Analytics tracking.
If you want to conduct very simple (and free) A|B tests, Google Content Experiments is an appropriate solution. Experiments has certain limitations for pages with flow traffic and is not the best solution for people who don't have the ability to code a page from scratch, or need to test without the help of IT. Experiments also (obviously) assumes that you are a current Google Analytics user and does not take into account tracking with a different tool.

Using Google Content Experiments

16 - Client account managers, optimization expert.
Content Experiments just makes it is simple and easy to implement A|B tests. We will be evaluating other tools in search of a more robust system for multivariate and cross-page testing, such as Optimizely or Visual Website Optimizer. However, for basic testing, you can't really beat it.