Optimizely Review from an OTA Product Manager
January 12, 2020

Optimizely Review from an OTA Product Manager

Anonymous | TrustRadius Reviewer
Score 9 out of 10
Vetted Review
Verified User

Overall Satisfaction with Optimizely

It is being used to increase conversion in our shopping funnel. The users are mainly the product department -- we build the tests, run them, and relay the results to the rest of the Exec team. Ideas can come from anywhere in the organization.
  • It provides an interface to create tests easily.
  • It sends out webinars and best practice emails.
  • It might be our integration with GA, but it's hard to trust the results at times.
  • Traffic allocation-- this might be our implementation, but the users are never 100% on an A/B test.
  • It's increased conversion and revenue after implementing some tests baseline.
  • Time saved through the fast turnaround to get an idea "tested" versus full development.
We have evaluated Optimize every year against Optimizely. I am a google fan, but their product is not up to the level of Optimizely. They do not offer multi-page tests, and a lot of our changes sometimes span the whole funnel. For that reason alone, it's currently unusable.
Optimizely is well suited for testing changes that you may not believe will influence a guests' decision. It's less appropriate for complete funnel changes and overlapping tests.

Optimizely Web Experimentation Feature Ratings

a/b experiment testing
9
Split URL testing
8
Multivariate testing
Not Rated
Multi-page/funnel testing
7
Cross-browser testing
Not Rated
Mobile app testing
Not Rated
Test significance
7
Visual / WYSIWYG editor
8
Advanced code editor
8
Page surveys
Not Rated
Preview mode
8
Test duration calculator
Not Rated
Experiment scheduler
8
Dynamic experiment activation
7
Client-side tests
9
Mutually exclusive tests
6
Standard visitor segmentation
8
Behavioral visitor segmentation
8
Traffic allocation control
8
Website personalization
Not Rated
Conversion tracking
Not Rated
Goal tracking
Not Rated
Test reporting
Not Rated
Results segmentation
Not Rated
CSV export
Not Rated
Experiments results dashboard
6

Using Optimizely

It's great for the most part -- moving to X introduced some pain points, like every "page" needs separate code, and visual editor changes don't translate so you can edit them.

Optimizely Reliability

I find that when running more than one test on the same page, even not mutually exclusive, the numbers in each test drastically drop -- but again, that may be our implementation of it.