You could be testing in just minutes.
November 16, 2015
You could be testing in just minutes.
Score 10 out of 10
Vetted Review
Verified User
Software Version
Other
Modules Used
- A/B Testing
- DCP
- Personalization
- API
Overall Satisfaction with Optimizely
We use Optimizely across the board, for testing new features before they're rolled out, making layout / content tweaks and testing their impact, personalizing experience for individual visitors, and even as a quick way to fix coding issues and allow developers more time to roll out a more permanent solution.
- A/B testing - this is the core functionality that Optimizely has always been great with and only got better as they add more functionality.
- Test analytics - the reporting pages are great, the impact of the tests on revenue is extremely difficult to calculate correctly. Optimizely has made a lot of updates to their statistics engine, latest round of changes upset a lot of people, but it's still doing a great job presenting very complicated data in a simple way.
- Personalization - still new and barely out of beta, but already looks promising and judging by the quality of the core product, A/B testing, this should be a hit with marketers as well.
- Test results - the under the hood calculations could be more transparent, while the statistics engine does a ton of very complex calculations that we don't really need to see, in the end it provides one number for "statistical significance". As that number changes over time, sometimes its apparent fluctuations make no intuitive sense. This can make it difficult to justify test results to management and the rest of the team.
- Lift in Revenue and Conversion Rates. We've ran hundreds of tests, with a few big wins and a few spectacular losses. But while winning tests are implemented for 100% of visitors and immediately begin to contribute to the bottom line, even the losses can lead to more followup tests, some of which end up as winers by the 2nd or 3rd generation as well.
- Optimizely is by far the fastest and easiest way to change content on pages, while a more permanent change is in the dev queue. This is great for almost all departments - Marketing, Compliance, Dev and QA, Customer Service, etc.