Item: Optimizely Web Experimentation
Rating: 10
Author: Verified User

Overall Satisfaction with Optimizely

Use Cases and Deployment Scope

We recently launched a newly designed website, and at first, we relied on Optimizely to implement "hotfixes" for issues until we could deploy the next code release. Recently, we've been focused on testing the optimization of the checkout flow. We've also been using it to test new features on the site before spending the time and resources on a fully developed solution. Our team is very lean, so Optimizely is primarily used by our agency partner responsible for web analytics, but a few of my colleagues and I use it as well.

Pros and Cons

Pros

A user-friendly (usability) interface for day to day experience setup and monitoring.
A novice can easily set up simple experiments, but an advanced user can set up more complex experiments with code/etc.
Customer service, customer support, and support architects are impressively responsive and knowledgable.

Cons

I just received an auto-generated notification that we've used 80% of our impressions in the first three months of our contract (yikes!!!!). I had mistakenly been going off visitors and thought we were ok. I had to dig for a while to even find where these this logged, and even then I'm not sure which experiments are burning through the impressions. I had to negotiate (internally) to get the budget for the expanded contract to the higher tier with more impressions, so I'm not sure whether I'll be able to talk my way into a mid-year budget review to further expand/extend my contract. Otherwise, I'm guessing I'll have to shut down Optimizely for the next nine months. Bottom line: I felt entirely blindsided by the lack of visibility for how many impressions we were burning through.

Return on Investment

Positive: resource allocation/prioritization for potential new features.
Positive: providing data to determine whether ideas coming down from the executive team would actually increase target metrics.
Negative: because I was incorrectly monitoring visitors rather than impressions, feeling blindsided when I realized we've burned through 80% of our impressions in the first three months of the year, I feel as if I've wasted a lot of hard-earned budget allocation and internal political capital for only a handful of experiments. I'm not sure how I'm going to explain the embarrassingly low ROI of the platform and why I stopped using it after only three months when I fought so hard to get it in the first place.

Alternatives Considered

FullStory, Usabilla, Salesforce Marketing Cloud Email Studio, Salesforce Commerce Cloud, UserZoom and Hotjar

We're not using UserZoom or Hotjar. We use the others for specific functions. Optimizely is the best for website optimization.

Other Software Used

Usabilla, FullStory, InMoment Voice of the Customer, Hitwise

Likelihood to Recommend

Great for hotfixes, and both simple and complex experiments. Apparently (per my early comment about being blindsided about the impressions we were burning through), it's not well suited for any experiments that run across multiple pages, such as global navigation.

Optimizely Web Experimentation Feature Ratings

Using Optimizely

Usability

Sometimes it's quite difficult to share a test with stakeholders to give feedback before going live. I'm also not able to see how many impressions we're burning through (which caused significant negative consequences). Otherwise, it's easy to use.

Optimizely Reliability

Scalability

Because our optimization "team" is generally two people, we don't use the program management functionality that came with our subscription. I could see how this would be a great function if I needed to scale to more teams.

Comments

Please log in to join the conversation

Great for novice and advanced users, and for small and large teams