It's Easy as A/B/C - Experimentation Made Simple
December 16, 2016
It's Easy as A/B/C - Experimentation Made Simple
Score 9 out of 10
Vetted Review
Verified User
Software Version
Other
Overall Satisfaction with Optimizely
At Northern Tool + Equipment the design and analytics teams primarily use Optimizely to test design, usability and promotional changes to see what impacts our users in a meaningful way. We run experiments to determine whether or not a new feature is worth building, an existing feature or piece of content is useful or clutters the page and should be removed, and also what promotions and marketing messages resonate.
- Quickly and easily set up new A/B tests. Some tests can be completed in only a few minutes.
- A robust results engine. It automatically creates segments based off URL parameters and other common criteria (new vs returning) in addition to predefined criteria you set up.
- The support of the Optimizely implementation team and customer care is fantastic. Optimizely easily ranks as one of our best third party software vendors. They provide not just knowledge of how to use the tool, but how to build a culture around testing and experimentation within your company.
- Acts as a great lightweight content management system for making temporary changes. We use Optimizely to make changes for 100% of traffic for controlling scheduled promotions and also bug fix patches while waiting for permanent fixes.
- Geo-targeting for tests does not allow you to use ZIP Codes.
- URL targeting does not have a mass upload option for lists of unrelated URLs. It does have substring and regular expression matches that cover most other needs.
- The preview feature rarely works as expected. We run tests on a dedicated testing environment to preview and do QA.
- We've had at least a dozen different A/B test results that have proven a positive or negative impact on customer metrics and helped drive our product roadmap.
We were a former Adobe Test and Target client and felt that the tool was not keeping up with the competition. Optimizely greatly streamlined the process of setting up and running A/B tests, making results easy to understand, and provided superior customer service. I understand Test and Target has come a long way in the past few years, no doubt because of many other companies like ourselves switching to Optimizely.