Item: Optimizely Web Experimentation
Rating: 9
Author: Albert Wavering

Overall Satisfaction with Optimizely

Pros and Cons

Pros

Easy to set up test variations, either with their visual tools or with Javascript.
Easy to describe goals - can be clicks on a page element, page views, or custom Javascript.
Can segment and target visitors based on URL, referral information, custom Javascript conditions, or a number of other variables.
Good for previewing and testing variations prior to launching tests - has great live preview functionality and integrates with cross browser testing tools.
Has good flexibility and control over implementation - especially important for avoiding the flash of unstyled content problem, as well as Javascript race conditions.
API lets you integrate with Google Analytics or other third-party analytics system.

Cons

Javascript frontend sometimes loads slowly or requires refresh to load completely.

Return on Investment

Varies from client to client and test to test.

Usability

Extremely low learning curve. Easy to get started with, but offers lots of features to power users.

Alternatives Considered

Google Website Optimizer (now Content Experiments)
Visual Website Optimizer

Likelihood to Recommend

Optimizely improves our website optimization efforts by making it so easy to set up tests that implementation details are no longer a barrier to running tests.

Using Optimizely

Likelihood to Renew

Optimizely is critical to our testing efforts.

Optimizely Implementation

Optimizely offers configuration options for different website setups - be sure to use a snippet that is consistent with your website design. For example, the Optimizely snippet can include full or trimmed copies of the jQuery Javascript library (which it needs to function), but this doesn't need to be enabled if you already have jQuery on your website.