Item: Optimizely Feature Experimentation
Rating: 10
Author: Andrei Beno

Overall Satisfaction with Optimizely Feature Experimentation

Use Cases and Deployment Scope

At our organization, we've integrated Optimizely Feature Experimentation into our development and release processes to address several key business challenges, including employing it for feature flags for controlled rollouts of new functionalities, and for A/B testing which has become integral to our decision-making process. We conduct gradual rollouts and experiments across our application and website to optimize user experiences and drive key performance indicators and to reduce rollout risks before doing a full general availability rollout.

Pros and Cons

Pros

Its robust feature flagging system, which allows our development teams to gradually roll out new features to specific user segments, enable or disable features without deploying new code, and test features in production environments with minimal risk
For example, we would release a new checkout process to 5% of users, monitor its performance, and gradually increase the rollout percentage based on real-time data
Server side performance optimization is really important at our scale, as it ensures improved page load times by avoiding client-side rendering of experiments and the elimination of visual flickering often associated with client-side testing
Precise targeting and segmentation, including the use of custom attributes for granular audience segmentation, ability to target users based on behavior, demographics, or technographics dimensions and creating nested audience definitions for sophisticated experiments that imply potentially overlapping audiences

Cons

The code editor for setting up complex audiences can be challenging to use, especially for team members without strong technical skills. A more intuitive interface for creating sophisticated audience segments would be beneficial
A one-click rollback option or a more streamlined process for quickly reverting to previous versions would enhance our risk mitigation agility and reduce risk in production environments
Although Optimizely has an expansive list of integrations options, we could benefit from an even wider range of native integrations with popular development and analytics tools to be able to analyze our experiment data faster

Return on Investment

Reduced risks for new releases
Faster time-to-market
Improved decision-making

Usability

Overall, i find the platform quite user-friendly, especially for our technical team members. The feature flagging system is intuitive and allows us to easily control rollouts. I particularly appreciate the server-side capabilities, which have been crucial for maintaining performance on our high-traffic sites.

However, there's a bit of a learning curve, especially for our non-technical team members. The audience targeting features, while powerful, can be complex to set up. I've found that creating sophisticated audience segments often requires input from our developers.

Alternatives Considered

Mixpanel, Google Analytics, Hotjar and A/B Smartly

Key Insights

Do you think Optimizely Feature Experimentation delivers good value for the price?

Yes

Are you happy with Optimizely Feature Experimentation's feature set?

Yes

Did Optimizely Feature Experimentation live up to sales and marketing promises?

Yes

Did implementation of Optimizely Feature Experimentation go as expected?

Yes

Would you buy Optimizely Feature Experimentation again?

Yes

Other Software Used

Hotjar, Google Analytics, Mixpanel, Tableau Cloud, Heap, Miro, Slack, Jira Product Discovery, Notion, Atlassian Confluence

Likelihood to Recommend

Based on my experience with Optimizely Feature Experimentation, I can highlight several scenarios where it excels and a few where it may be less suitable.

Well-suited scenarios:
- Multi-Channel product launches
- Complex A/B testing and feature flag management
- Gradual rollout and risk mitigation

Less suited scenarios:
- Simple A/B tests (their Web Experimentation product is probably better for that)
- Non-technical team usage
-

Comments

Please log in to join the conversation

Optimizely Feature Experimentation is great for complex A/B tests and feature flag management!