a program for beginner experimentation, all the way to a mature testing & personalization program
Updated November 20, 2024

a program for beginner experimentation, all the way to a mature testing & personalization program

Anonymous | TrustRadius Reviewer
Score 9 out of 10
Vetted Review
Verified User

Overall Satisfaction with Optimizely Web Experimentation

We use Optimizely Web Experimentation for content testing (images and/or copy), recommendation algorithm tests for product carousels, and feature testing (net new features that we develop and want to validate & calculate revenue incrementality before launching). With the first two: we want to test along the customer journey to understand what pieces of content or product assortments provide the best benefit to her on site. We also use audience segmenting to be able to target the right users (coming from X marketing channel, mobile-only users, or logged-in users) to be able to refine our testing.

Pros

  • Extensions - allows the company to tailor the tool to unlock certain capabilities
  • Developer-friendly, with GA and other integrations
  • Intuitive results page that can be shared out

Cons

  • Hard to use extensions without some knowledge of HTML/CSS
  • Inconsistent user counts between GA & Optimizely when it comes to calculating MAUs, hard to be able to forecast budget and overages when our source of truth (GA) differs from our testing platform (Optimizely)
  • Impressions model doesn't support scaling personalization experiences - hard to run a serious data driven testing program when you have to cut tests short before 14 days (ideal length to get to any statistical read) in order to save on impressions
  • Validated business case for going into new testing approach
  • Increased conversions from certain audiences by running and launching successful experiences tailored to those users
It has been easy to deploy Optimizely on our site, however, we continuously monitor the load of the Optimizely PJS on our site, and we find that the load of having extensions, custom metrics, custom audiences, targeted tests, tests that have been paused but not archived, all have a negative impact on the site in that it is extra Javascript that might weigh down the load of the page and hurt our page performance. We haven't yet been easily able to manage this with the current Optimizely set up we use – it requires a lot of manual maintenance and attention to ensure it is kept as clean as possible.
For launch and analysis: it is so helpful for us to only look at one report to see key test results. For targeted experiences: if we're testing on Paid Search audiences, and if we want to launch that experience at 100%, it is much easier for us to scale that experience out.
We use AEM so Adobe Target would be a natural choice, it integrates naturally with the Experience Fragments and all the content we already hold there. However - with extensions - we've been able to unlock a similar workflow to be able to seamlessly test. Optimizely has the unique capability to be able to feature test, which is how we have been able to evolve our onsite experience and not only be able to quantify the revenue impact of our content or copy changes, but of entire site changes.

Do you think Optimizely Web Experimentation delivers good value for the price?

Yes

Are you happy with Optimizely Web Experimentation's feature set?

Yes

Did Optimizely Web Experimentation live up to sales and marketing promises?

Yes

Did implementation of Optimizely Web Experimentation go as expected?

Yes

Would you buy Optimizely Web Experimentation again?

Yes

Optimizely Web Experimentation is well suited for a team that tests across multiple facets of testing – full tech developed features, pieces of content or recommendations, landing page optimization. However - if a team only does limited testing, and only in the content testing area, other tools may integrate better IF they are not using Optimizely's CMS (and if they have a CMS that easily supports testing - like Adobe AEM x Adobe Target). However - as testing scales - Optimizely Web Experimentation will start to make more and more sense.

Optimizely Web Experimentation Feature Ratings

7.9
a/b experiment testing
10
Split URL testing
Not Rated
Multivariate testing
8
Multi-page/funnel testing
8
Cross-browser testing
Not Rated
Mobile app testing
Not Rated
Test significance
7
Visual / WYSIWYG editor
5
Advanced code editor
5
Preview mode
10
Test duration calculator
Not Rated
Experiment scheduler
10
Experiment workflow and approval
Not Rated
Dynamic experiment activation
Not Rated
Client-side tests
Not Rated
Server-side tests
Not Rated
Mutually exclusive tests
Not Rated
10.0
Standard visitor segmentation
10
Behavioral visitor segmentation
10
Traffic allocation control
10
Website personalization
10
10.0
Form fill analysis
Not Rated
Goal tracking
Not Rated
Test reporting
10
CSV export
Not Rated
Platform Integration Features ()
Not Rated

Using Optimizely Web Experimentation

10 - product management, site experience, CRM
2 - we have a product manager and a web experimentation lead for optimizely web experimentation. the product manager helps develop additional areas for the web experimentation lead to run testing on. web experimentation lead is not technical and deploys content, recommendation strategy, and copy or linking tests as part of the testing program.
  • testing feature changes before we roll out code
  • testing copy or content changes to inform promotions or site design
  • test and then deliver targeted recommendation strategies to certain audiences
  • tested hiding links in our navigation, resulted in a +1% improvement in conversion rate
  • deploy custom landing pages for certain incoming channels (using optimizely, add/swap/remove content) for a personalized experience
we’ve seen huge improvements in our program with optimizely and it’s the only tool that makes it easy for product to validate site changes and also our site team to test and learn on content placement.

Using Optimizely Web Experimentation

Comments

More Reviews of Optimizely Web Experimentation

  1. Home
  2. A/B Testing Tools
  3. Optimizely Web Experimentation Reviews
  4. User Review of Optimizely Web Experimentation