Optimizely: Solid A/B testing solution
October 19, 2013

Optimizely: Solid A/B testing solution

Shawn Daley | TrustRadius Reviewer
Score 7 out of 10
Vetted Review
Verified User

Overall Satisfaction

Pros

  • Our client has been happy with the tools. The set-up and execution of experiments is relatively simple. It has been used extensively to test content, banners, sign-ups, and product locations.

Cons

  • The design needs of the client has required a lot of developer experience to create test content and integration with site functionality that's not 'out of the box'.
  • In my current role I am not measuring the product performance.
The client has been very satisfied with the results and learnings.
Optimizely can be a powerful experimentation tool if there are resources devoted to implementing it correctly and maintaining a robust testing program.

Product Usage

4 - Analytics, Optimization, and Reporting
1 - In my job capacity I work to handle implementation and support of all 3rd party tools, including Adobe Analytics, Google Analytics, eBay Enterprise Attribution, Coremetrics, DART, Mediamind, Ensighten, Tealium, and Adobe Dynamic Tag Manager, etc.
  • A/B & Multivariate testing on a retail website.

Evaluation and Selection

Adobe Test & Target - not chosen due to cost and set-up requirements.

Implementation

The implementation through the tag management system took a bit of trial and error at first, mostly due to the asynchronous nature of the TMS. We had to manipulate the implementation to assure that the Optimizely code was written to the page at the right time to allow the experiment content load in the browser without showing any of the original content first. We also needed to make some adjustments to the TMS code to get the integration with Site Catalyst timed appropriately.
  • Professional services company
We implemented the product for the client using a tag management system. We manage all of the client's analytics and 3rd party tools through the TMS on their behalf.

Training

  • Self-taught
Product training is always preferred over self-training. This helps to have multiple people with a similar product skill-set available to assist with projects and train others to the same level of expertise.

Configuration

There has been a lot of experiment integration with site functionality like email sign up, add-to-cart, product display grids/positions, custom espots with additional functionality, etc.

Support

We have had fairly quick responses to requests for support and any issues have been worked out in a timely manner.
No - Client initially wanted to prove out the ROI of the product.

Usability

Not a lot of documentation or best practice information. Interface is simple, but not very intuitive.

Reliability

We have had some issues with screen flicker when loading experiment content. We had to dial back some of the creative changes to load smaller files.

Integration

  • Adobe Analytics (Site Catalyst)
  • Adobe Dynamic Tag Manager
All of the coding is handled through the Adobe Dynamic Tag Manager. Some difficulty working with the asynchronous default functionality of the tag manager at first.
  • Many of our clients also use eBay Enterprise Attribution (formerly ClearSaleing).
It's unknown if eBay Enterprise Attribution has plans to integrate with Optimizely.

Vendor Relationship

This was handled mostly on the client side.
N/A

Comments

More Reviews of Optimizely Web Experimentation

  1. Home
  2. A/B Testing Tools
  3. Optimizely Web Experimentation Reviews
  4. User Review of Optimizely Web Experimentation