Google Content Experiments: Quick and easy (and free) A/B/n testing you should already be using.
Updated February 04, 2015

Google Content Experiments: Quick and easy (and free) A/B/n testing you should already be using.

Salvatore Polizzi McDonagh | TrustRadius Reviewer
Score 8 out of 10
Vetted Review
Verified User

Software Version

June 2014

Overall Satisfaction with Google Content Experiments

We use Google Content Experiments to run A/B/n split test and optimize landing pages.

Pros:
* Easy to set up a test and use content experiments to monitor ongoing test metrics, if you already have Google Analytics installed.
* Good entry level tool: Due to the relatively low skill required - so even entry level staff will be able to run a split test.with minimal training and supervision. Interpreting the results however...requires appropriate training. Caveat emptor! As with any other A/B test tool, badly designed tests result in useless data.
* Fully integrated with Google Analytics, so you can use existing goals, all the metrics are the same as in Google Analytics, and are measured in the same way as the rest of the Google Analytics metrics. Makes life much easier when interpreting the results.
* Run test only using a percentage of website traffic (current options are 1%, 5%, 0%, 25%, 50%, 75%, 100%) to reduce risk when testing radical changes.
* Free
* Huge volume of online resources to help with getting started and using it.

Cons:
* Tests cannot be paused - only ended.
* Losing variants cannot be removed before the test is complete - although if you choose the default setup option to not split traffic evenly, Google will optimize throughout the test and reduce traffic to the losing page(s) while increasing it to the winner(s).
* Same proportional split only - i.e. you cannot give one version 90% and the other 10% of the traffic (You might want to do this to confirm that your current champion page is consistently winning over the demoted loser over a long period to rule out seasonal variations, etc. Or you might not want to risk 50% of your traffic on a radically different landing page and subsequent revenue loss if it is a loser.) You can (partially) work around this by having multiple versions of one page and testing against a single version of the other (max 4:1 to get 80%:20% split)
* A/B/n testing only: Google killed it's (in some ways better) Website Optimizer to replace it with Content Experiments - and in so doing dropped the ability to run multivariate tests. This could also be seen as a pro ;-) Multivariate testing requires much better design of experiments, more traffic, and more time than A/B testing. Since Google measures everything, they presumably found that their customers were doing A/B and needed a better integrated tool, and that MVT was not being used, or was not appropriate for this market sector.
* Tests must run for 3 days minimum (works for us at SpamTitan, but in a previous job with much higher volume of traffic, and same day sales conversions, a valid winner could be called much sooner. Your sales cycle, traffic volume and conversion funnel will determine if this is a negative factor)
* Tests cannot be run for more than 3 months (I believe!) so if you have a long, complex sale cycle, this may not work for you - you can only use it for micro-conversions. So you'd best be sure they are actually relevant factors influencing the sale. It probably goes without saying, but I'll say it anyway - it's easy to optimize for registrations at the expense of actual sales. Sometimes a lower conversion rate for a microconversion will result in higher profits.
* Almost zero support from Google - you probably cannot get support direct from the vendor unless you are spending enough on Adwords to have an account manager. Luckily, you probably won't need help as the product is easy to use and limited in scope.
  • Free
  • Easy to use if you already use Google Analytics - literally a few minutes to set up a test
  • Fully integrated with Google Analytics - so you can use your existing conversion goals, and no confusion over metrics.
  • Multivariate testing
  • Ability to drop losing variants during a test
  • Ability to manually choose split of traffic between variants
  • Higher quality leads
  • Reduced sales team effort (due to less low quality leads)
  • Higher conversion from visitor to lead
Do you already have Google Analytics? If so content experiments is a good, free, starting point to dip your toes in A/B testing. Do you need to run Multivariate experiments? If so, Google Content Experiments is not going to fit your needs.

Using Google Content Experiments

2 - 1. Adwords Advertising exec.
2. Marketing Manager.
and ....3rd person in training - Marketing Exec.
With Google Content Experiments it is just so quick and easy to set up an experiment. Using Google Analytics daily, it just feels wrong not to have at least one content experiment running.