Optimizely - It's Good
Updated December 09, 2014
Optimizely - It's Good
Score 10 out of 10
Vetted Review
Verified User
Overall Satisfaction with Optimizely
- Provides a dead simple way to get up and running with AB and MVT testing on your web properties via its super simple installation and highly intuitive variation WYSIWYG based editor.
- Open and transparent with its inner workings. Curious what that WYSIWYG editor is actually doing? Optimizely provides full access to all variation code. Don't like WYSIWYG editors? Handwrite the standard Javascript based variation code yourself. Uncomfortable with code executing on your site that you can't see? Simply append the "optimizely_log=true" parameter to any URL on your website with an experiment running and check the Javacript console. You'll see a highly verbal log of all that is happening. Want to programmatically execute experiments yourself? Checkout Optimizely's extensive API (https://www.optimizely.com/docs/api)
- Not just for beginners. Being "enterprise" doesn't have to mean being hard to use. See comment above.
- Very competitively priced.
- Offers competitive segmenting capabilities which allow you to target experiments based on location, operating system, device type, referral, javascript variables, cookie values, URL patterns of all kinds, and more.
- Supports mobile web testing with intuitive tools and emulators to create experiments specifically for mobile and tablet devices.
- Allows you to create custom user segments that can be reused across experiments.
- Ability to organize and group experiments under "Projects" which can have their own granular user permissions. Ex: Give a partner or member of your team admin access to one set of experiments and read-only access to others.
- Provides a very simple set of tools to create and track custom experiment goals such as "mouse click" events on a specific button or traffic goals to a specific URL.
- Ability to demo the product immediately without an initial sales call.
- With a recent $28 million round of investment, you can feel confident that Optimizely has the funding to stay in business for a long time and continue to disrupt this market. (http://blog.optimizely.com/2013/04/10/optimizely-raises-28-million-to-go-global/)
- Optimizely does not support the use of alternative algorithms for judging experiment winners such as those of the "bandit" and "adaptive" nature. I can only imagine that doing this would be highly complex, but the ability to select different algorithms could be killer and a true differentiator. I know of no other optimization software vendor that provides algorithmic choices for judging experiment performance.
- Optimizely is not yet able to ingest and segment off of more advanced and / or custom data sources such as your own CRM data, weather and census data, data from re-targeters like BlueKai, and other fancy data sets that some of the other major players have. There are ways to build work arounds via their segment API and custom targeting variables, but it would be nice if more advanced, pre-built segmentation conditions were included in the product.
- The integration with Google Analytics, while extremely easy to set-up, leaves a bit to be desired. Its dependent on Custom Variable slots of which you only get 5 if you are not GA Premium (most aren't), and it does things with visitor scope that make repeat visit conversion tracking challenging to do over the course of an experiment without custom Google Analytics JS code to manually set GA visitor scope. That said, there are also work arounds for this as well.
- There's no experiment version control. You'll often find yourself having to hard refresh while editing variations within an experiment without saving to revert your changes. Not the end of the world because there is this and other work arounds, but it's a nice to have. Again, it's important to point out that few have others have this capability.
- To start, Optimizely is very competitively priced and dead simple to set-up and install, so the bar to start getting a return on your investment starts off low.
- From there, it's up to you. What ideas do you have to improve your digital business? Optimizely is a tool that will help you test and measure them, and quantifiably tell you if they work or not. Your return will be as good as your testing ideas.
Using Optimizely
- A/B Testing
- Multivariate Testing
- A handy work around for CMS and E-Commerce platform limitations.
Optimizely Implementation
- Implemented in-house
Optimizely Training
- Self-taught
The basics of Optimizely are very easy to learn especially if you have web development experience including basic HTML, CSS, and JavaScript skills.
The documentation is solid. The community forums are fairly active, and the Optimizely admins regularly participate.
However, the more advanced tips and tricks that the Optimizely support engineers know like the back of their hands are a little harder to find for the average user. Those tend to be gleamed during 1:1 sessions during support requests.
It would be nice if Optimizely published more advanced "tips and tricks" video tutorials.
The documentation is solid. The community forums are fairly active, and the Optimizely admins regularly participate.
However, the more advanced tips and tricks that the Optimizely support engineers know like the back of their hands are a little harder to find for the average user. Those tend to be gleamed during 1:1 sessions during support requests.
It would be nice if Optimizely published more advanced "tips and tricks" video tutorials.
Optimizely Support
Yes - Premium support gives us consistent access to a highly skilled, named resource who has already helped us many, many times. The support has been very good.