It's Easy as A/B/C - Experimentation Made Simple
December 16, 2016

It's Easy as A/B/C - Experimentation Made Simple

Grant Wilkins | TrustRadius Reviewer
Score 9 out of 10
Vetted Review
Verified User

Software Version


Overall Satisfaction with Optimizely

At Northern Tool + Equipment the design and analytics teams primarily use Optimizely to test design, usability and promotional changes to see what impacts our users in a meaningful way. We run experiments to determine whether or not a new feature is worth building, an existing feature or piece of content is useful or clutters the page and should be removed, and also what promotions and marketing messages resonate.
  • Quickly and easily set up new A/B tests. Some tests can be completed in only a few minutes.
  • A robust results engine. It automatically creates segments based off URL parameters and other common criteria (new vs returning) in addition to predefined criteria you set up.
  • The support of the Optimizely implementation team and customer care is fantastic. Optimizely easily ranks as one of our best third party software vendors. They provide not just knowledge of how to use the tool, but how to build a culture around testing and experimentation within your company.
  • Acts as a great lightweight content management system for making temporary changes. We use Optimizely to make changes for 100% of traffic for controlling scheduled promotions and also bug fix patches while waiting for permanent fixes.
  • Geo-targeting for tests does not allow you to use ZIP Codes.
  • URL targeting does not have a mass upload option for lists of unrelated URLs. It does have substring and regular expression matches that cover most other needs.
  • The preview feature rarely works as expected. We run tests on a dedicated testing environment to preview and do QA.
  • We've had at least a dozen different A/B test results that have proven a positive or negative impact on customer metrics and helped drive our product roadmap.
We were a former Adobe Test and Target client and felt that the tool was not keeping up with the competition. Optimizely greatly streamlined the process of setting up and running A/B tests, making results easy to understand, and provided superior customer service. I understand Test and Target has come a long way in the past few years, no doubt because of many other companies like ourselves switching to Optimizely.
Optimizely is a great tool to get you up and running with A/B testing quickly without sacrificing quality. The WYSIWYG editor mode is passable for someone without html and javascript knowledge, but to get the most out of the tool you'll definitely want to have a developer that is fluent in javascript. As of a few months ago Optimizely now offers server side (or full stack) testing in addition to their client side testing. This is an additional feature you have to pay for, but if you have the need and teams to support server side testing it allows you to test a lot of different things you won't get out of client side testing.