Overview
What is Optimizely Web Experimentation?
Optimizely Web Experimentation empowers teams to conduct experiments (without having to rely on developer resources) in order to test various user interactions, make website changes backed by data, and personalize customer experiences.
Great web experimentation framework for marketing and engineering
A solid tool suited for a solid optimization programme (emerging up to connected)
Not value for money
Optimizely Review
Recommended for medium/advance users
Good platform
Data has no opinions
Optimizely Web Experimentation
Optimizely Web Experimentation for Digital Marketing
Take the Guess Work Out of Your CRO with Optimizely!
Great tool to get your team into the testing mentality.
Great experimentation tool
Optimizely: Streamlining Experimentation for Smarter Product Strategy Decisions.
Optimizely Web Experimentation Review
Awards
Products that are considered exceptional by their customers based on a variety of criteria win TrustRadius awards. Learn more about the types of TrustRadius awards to make the best purchase decision. More about TrustRadius Awards
Popular Features
- a/b experiment testing (101)9.393%
- Standard visitor segmentation (93)8.989%
- Test significance (92)8.585%
- Preview mode (90)8.080%
Reviewer Pros & Cons
Video Reviews
1 video
Pricing
What is Optimizely Web Experimentation?
Optimizely Web Experimentation empowers teams to conduct experiments (without having to rely on developer resources) in order to test various user interactions, make website changes backed by data, and personalize customer experiences.
Entry-level set up fee?
- Setup fee optional
Offerings
- Free Trial
- Free/Freemium Version
- Premium Consulting/Integration Services
Would you like us to let the vendor know that you want pricing?
46 people also want pricing
Alternatives Pricing
What is Oracle Marketing?
Oracle CX Marketing (formerly Oracle Marketing Cloud) is a solution designed to enable marketers to plan and execute automated marketing campaigns via email, display search, video advertising, and mobile while delivering a personalized customer experience for their prospects.
What is Instapage?
Instapage, from AirSlate since the October 2023 acquisition, offers post-click automation to help digital advertisers create more relevant and impactful advertising, unifying the post-click stage of the advertising funnel. Instapage helps marketers to maximize conversions by creating,…
Features
Testing and Experimentation
These features enable companies to plan, set up, and execute different types of tests (e.g. A/B, A/B/n, multivariate, split URL tests).
- 9.3a/b experiment testing(101) Ratings
Create and test variations of a website, changing site elements such as headlines, CTAs, images, page design and layout, technical SEO changes, and new feature additions and collect statistical results of each variation’s conversion rates or other metrics.
- 8.5Split URL testing(82) Ratings
Test out larger design changes by splitting your site traffic across two different landing pages to identify which site performs the best. It can be used to test the impact and feasibility of things such as new designs, personalization efforts, and new site architecture.
- 8.8Multivariate testing(84) Ratings
Ability to test multiple site design changes at once across one or multiple variations and identify which variation impacts conversion rates, or other predefined goals, the most.
- 8.2Multi-page/funnel testing(79) Ratings
Create an experiment that makes changes across multiple pages, like a funnel or a site-wide experience.
- 7.4Cross-browser testing(54) Ratings
Preview your experiments across multiple browsers at once.
- 7.6Mobile app testing(43) Ratings
Ability to run tests to optimize mobile applications.
- 8.5Test significance(92) Ratings
Ability to set the statistical significance level, or confidence interval, of a given test, for example at the 90% or 95% level.
- 8Visual / WYSIWYG editor(81) Ratings
Set up A/B testing campaigns using a WYSIWYG editor to create site versions and preview design changes before testing them. These editors often don’t require coding knowledge in order to operate them.
- 7.5Advanced code editor(74) Ratings
Allows users to create and edit experiments with HTML, CSS, JS.
- 6.9Page surveys(17) Ratings
Create on-page surveys and select which segment of users are asked survey questions using defined audience segments (e.g. new vs. returning users, mobile users, desktop users, etc).
- 8.4Visitor recordings(18) Ratings
Watch recordings of user sessions to gain insights on site visitor behavior and identify areas to improve site visitor experience.
- 8Preview mode(90) Ratings
Preview your experiment before running it live on your site or app.
- 6.9Test duration calculator(62) Ratings
Automatic calculation of the estimated test duration needed to gain statistically significant results.
- 8.1Experiment scheduler(61) Ratings
Ability to schedule experiments to run, or not run, during specific times (e.g. Not to run during a holiday weekend or while a site-wide promotion is going on).
- 6.5Experiment workflow and approval(42) Ratings
Ability to assign different phases of the experiment process to your team and approve next steps for an experiment or campaign.
- 7.4Dynamic experiment activation(38) Ratings
Ability to activate an experiment after the page’s initial load based on a set of conditions (e.g. if the visitor takes certain actions).
- 9.2Client-side tests(56) Ratings
Ability to run client-side tests (e.g. A/B, A/B/n, multivariate, and funnel tests) to test out UI changes.
- 9.1Server-side tests(24) Ratings
Ability to run server-side tests (e.g. A/B, A/B/n, multivariate, and split URL tests) to test out more complex design changes, roll out features to specific audience segments, or split site traffic between different site versions.
- 8.5Mutually exclusive tests(42) Ratings
Ability to make tests mutually exclusive so that a given visitor is only part of one test at a time, this helps prevent tests from interfering with one another.
Audience Segmentation & Targeting
A set of tools used for website optimization experiments (e.g. A/B, A/B/n, funnel, split URL, multivariate tests) that can help users segment their audience in to different groups for the purpose of exposing specific audiences to tests or personalization efforts.
- 8.9Standard visitor segmentation(93) Ratings
Ability to segment, or target audiences based on criteria you set (e.g. URL, cookies, IP address, custom javascript, traffic source, device, browser, language, ad campaign, geo-targeting, time of day) and enable tests to run for specific visitor segments.
- 8.3Behavioral visitor segmentation(73) Ratings
Ability to segment, or target audiences based on whether or not they have performed certain actions, such as clicking on a CTA, and enable tests to run for specific visitor segments.
- 9.4Traffic allocation control(89) Ratings
Ability to set what percentage of website traffic receives specific test variants in order to roll out code only to a subset of site visitors.
- 7.9Website personalization(62) Ratings
Ability to optimize user experience for individual site visitors based on certain characteristics and past actions (e.g. past purchases, geolocation, demographics, device type, referral source, etc..). An example of this is product and/or content recommendations based on visitor characteristics.
Results and Analysis
Tools that allow users to evaluate the results of website optimization tests (e.g. A/B, A/B/n, multivariate, and split URL tests), or view visitor interaction with webpages and specific site elements.
- 9.1Heatmap tool(13) Ratings
A tool that shows which elements of the page generate the most visitor engagement.
- 8.7Click analytics(33) Ratings
Click analytics reports display how many clicks certain page elements receive and provides visitor engagement insights.
- 8.1Scroll maps(17) Ratings
Scroll maps display how far down the page users scroll.
- 8.3Form fill analysis(32) Ratings
Enables users to view visitor interaction with forms and identify which parts of the form visitors fill out first and which fields lead to increased visitor drop-off.
- 8.7Conversion tracking(44) Ratings
Enables users to set up and customize conversion funnels to track site visitors' journeys and determine areas that see the most visitor drop-off.
- 8.4Goal tracking(79) Ratings
Enables users to set up key website/mobile performance metrics on their landing pages and track them.
- 8.1Test reporting(82) Ratings
Provides users with reports for each test that record the performance of each variation tested against selected metrics such as conversion rate. These reports indicate when a given test variation has performed statistically better than the original (control) site version.
- 8Results segmentation(49) Ratings
The ability to segment test results by specific criteria (e.g. browser type, device type, source, time of day, campaign).
- 7.6CSV export(54) Ratings
Ability to export test results as a CSV file.
- 8Experiments results dashboard(49) Ratings
Provides users with a dashboard displaying test results for all live tests. Some products may also include test result histories accessible from the dashboard.
Product Details
- About
- Integrations
- Competitors
- Tech Details
- Downloadables
- FAQs
What is Optimizely Web Experimentation?
Optimizely Web Experimentation Features
Testing and Experimentation Features
- Supported: a/b experiment testing
- Supported: Split URL testing
- Supported: Multivariate testing
- Supported: Multi-page/funnel testing
- Supported: Cross-browser testing
- Supported: Mobile app testing
- Supported: Test significance
- Supported: Visual / WYSIWYG editor
- Supported: Advanced code editor
- Supported: Preview mode
- Supported: Test duration calculator
- Supported: Experiment scheduler
- Supported: Experiment workflow and approval
- Supported: Dynamic experiment activation
- Supported: Client-side tests
- Supported: Server-side tests
- Supported: Mutually exclusive tests
Audience Segmentation & Targeting Features
- Supported: Standard visitor segmentation
- Supported: Behavioral visitor segmentation
- Supported: Traffic allocation control
- Supported: Website personalization
Results and Analysis Features
- Supported: Form fill analysis
- Supported: Goal tracking
- Supported: Test reporting
- Supported: Results segmentation
- Supported: CSV export
Platform Integration Features Features
- Supported: API
- Supported: Web analytics integration
- Supported: Content Management System Integration
- Supported: Integration with CRM or DMP software
Optimizely Web Experimentation Screenshots
Optimizely Web Experimentation Video
Optimizely Web Experimentation Integrations
Optimizely Web Experimentation Competitors
Optimizely Web Experimentation Technical Details
Deployment Types | Software as a Service (SaaS), Cloud, or Web-Based |
---|---|
Operating Systems | Unspecified |
Mobile Application | No |
Optimizely Web Experimentation Downloadables
Frequently Asked Questions
Comparisons
Compare with
Reviews and Ratings
(502)Attribute Ratings
Reviews
(126-150 of 186)- Very easy to configure experiment targeting, traffic allocation, and other settings you would want to customize for your testing program
- Allows easy selection of experiment types like Multivariate, Multipage, A/B
- Permits simple UI changes using its code editor and other options which allows for decreased reliance on development resources. Product and Design teams can launch new experiments easily on their own
- Support is first rate: knowledgeable, great communicators who take the time to explain things so that you can learn and apply to future experiments. Optimizely customer support is among the best I've encountered from a web software service
- Improvements can be made with experiment results reporting specifically customization of data within the data export or whatever is viewed on the summary page
- Different organizations could have different thresholds for statistical significance. Optimizely is set to 95%. It would be nice to have this value configurable.
- It is difficult to ascertain whether a losing variation is a statistically significant loss (only shows low chance to beat baseline). Having this would assist with pausing variations that are not performing very well.
- Customizable, distributable reports would be a great feature to add. Currently you must click through each individual experiment to see experiment details. Seeing a summary through email or selecting multiple experiments to view on one page would be a huge improvement for scanning health of tests.
Optimizely, good for some things, not for all!
- A/B testing
- The impact of CSS and HTML edits
- Integrating with our database to figure out revenue impact.
- Optimizely could be more intuitive. You have to dig through documentation in order to set up a simple A/B test.
Optimizely does its job pretty well
Optimizely helped us test content ideas, solve navigation issues, improve conversion rates, and better understand user behavior.
- Optimizely's WYSIWYG interface allows marketers who do not know how to code to make design changes easily.
- Optimizely informs a marketer when a test is complete and doesn't assume you have a deep understanding of statistics.
- Optimizely's documentation is thorough and helpful.
- Optimizely can allow novices to make many small changes that, in sum, add a lot of overhead to code execution.
- Optimizely does not have a pleasant way of viewing past tests or exporting results.
- It can be buggy.
- Quick and easy test setup if you've got the html/css changes ready to go.
- Detailed analytics, providing all sorts of quality information such as totals vs uniques.
- Support always picks up the phone right away! Great with answering your questions and often also give suggestions on better ways to test
- Please add more APIs for exporting data and integrate into a data warehouse so that we could perform our own analytics!
- The Optimizely script sometimes slows page loading time.
- It would be nice to be able to auto-pause experiments at a particular time.
You could be testing in just minutes.
- A/B testing - this is the core functionality that Optimizely has always been great with and only got better as they add more functionality.
- Test analytics - the reporting pages are great, the impact of the tests on revenue is extremely difficult to calculate correctly. Optimizely has made a lot of updates to their statistics engine, latest round of changes upset a lot of people, but it's still doing a great job presenting very complicated data in a simple way.
- Personalization - still new and barely out of beta, but already looks promising and judging by the quality of the core product, A/B testing, this should be a hit with marketers as well.
- Test results - the under the hood calculations could be more transparent, while the statistics engine does a ton of very complex calculations that we don't really need to see, in the end it provides one number for "statistical significance". As that number changes over time, sometimes its apparent fluctuations make no intuitive sense. This can make it difficult to justify test results to management and the rest of the team.
Optimizely is a Solid Platform for Testing
- Great for fast development especially with basic CSS components.
- Transparent results allow results to be shared across all departments.
- The user interface is very straightforward and Optimizely's free online documentation is an excellent resource.
- Advance code has issues especially with conditional activation on heavy JavaScript frameworks.
- Direct customer service via phone is not as transparent as it could be.
- Experiments can take a while to load, and previewing is sometimes not accurate due to slow caching issues.
Optimizely and Me: A Love Story
- Optimizely has a really powerful WYSIWYG editor. Not every marketing department is blessed with a talented developer. So this puts the power in the business user's hand, without getting bogged down in coding.
- If you are blessed with a talented developer (as we are) you also have the option of doing very advanced coding through the Optimizely program. This gives us flexibility we never had before and a window to testing we never thought possible.
- Optimizely segmentation is a power tool. The ability to customize the testing experience based on the segment is a strong advantage to any testing tool. We haven't even scratched the surface on the segments we can test through.
- It would be helpful to add some advanced reporting. We have some scenarios where to only want to proof our test is to follow a user down a full path which can at times be harder to accomplish. We have solved this problem, mainly, by building customer trigger events.
- A roll-up of past tests and their outcomes might be a nice feature. We have an offline report that our team lead develops at the end of each test.
For a beginner, Optimizely offers great support and an engaging interface to start in the optimization field
- The interface is easy to use and is always improving. I can select tests easily and navigate to goals and audiences with ease.
- The analytics are very detailed, especially with their latest update that shows us totals vs uniques.
- They eagerly engage with their clients, offering suggestions in how and what to test.
- Grouping of related goals together to easily add them to similar projects. There are times when I'm running 10 tests with 12 similar goals. It would be nice to add a group of goals and then edit what else I want or don't want.
- Some of their suggestions do not relate well to the exact context of how your industry works and how customers use your website.
- The ability to hide/display and move goals within the results. It can be done in a spreadsheet but I would really like to see it in the web interface.
If you are wanting to improve your site, use Optimizely!
- Their User Interface is so easy to use. You don't have to be a coder to set up a test. It's simply amazing.
- Their support is by far the best I've ever experienced. They always pick up the phone right away and are so helpful to answer your questions and to also give suggestions on better ways to do your tests.
- Sometimes it's a little tricky using third party jQuery plugins that we have on our site. It's not necessarily bad, but it could be better.
- The reporting is good, but it could be more robust. Simply a data table with a time chart. Would be nice to have more of a graphical interaction for their reports.
Doing it fast with Optimizely
- Optimizely has a very intuitive interface that really makes learning to use the tool quite easy. Even a complete novice can develop and deploy simple A/B tests with minimal effort.
- Optiverse, Optimizely's support forum, is an excellent place to learn how to take your A/B testing to the next level. I've gone from running simple copy change tests to advanced multi-page funnel testing.
- The ability to simply modify existing pages on your site rather than recreating them and redirecting traffic has been immensely valuable and allowed us to deploy tests much faster than we did before.
- I'm often frustrated with how hard it is to monitor results from my phone. I wish there was a mobile optimized version of the results section. I'm often travelling on the weekends and need to check on our tests via my phone. It can be done, but it's a very sub-optimal experience.
- I wish I could see the results of all my active tests on a single dashboard. Currently you have to drill into each experiment individually. Since I sometimes have as many as 10-15 experiments running, a good part of my day is spent monitoring a ton of open tabs for each test.
- Just nitpicking, but it would be nice to be able to reorder your experiment's goals without having to exit editing mode and return to the main projects page.
Optimizely > All Else
- Growing in terms of capabilities.
- Unmatched service.
- Further ecomm reporting.
- The product itself is very user friendly. There are varying degrees of technical expertise on the teams that employ this program. The product's ease of use was a real selling point.
- Their Customer Service has been nothing short of spectacular. We have a dedicated Account Manager who is really attentive to our questions and connects us the the correct person in their organization to help solve any issues we have.
- Their developer API connections offer so many custom solutions that are not included out of the box. As each business is different, this flexibility has been a tremendous asset to continue to help us grow.
- Showcase their users' implementations.
What level of support do they offer?
What ROI will this tool help us achieve?
What business needs will this help us solve now and in the future?
How will this tool impact my current website architecture?
How easy is this tool to use and what level of technical knowhow will our user need to have to be able to use?
How easy is it see the results of my test and be able to interpret these results into actionable items?
How does this tool integrate with other tools in my Marketing stack?
Optimizely Rocks
- Stellar support. Every person we've worked with on the support side has been knowledgeable and super helpful. Top notch for sure, and a great attitude to go along with the support.
- The product is so easy to use. Flexible while being user-friendly. We are able to do things quickly and easily that we could not do - even with consultant support - with competitive software.
- Community. The online community is great - it has great content, frequently updated with lots of people contributing. Most questions can be answered by checking the online community/online support. The community manager, Amanda, rocks!
- Reporting/exporting is a challenge if you want to report on data across multiple campaigns. This may not be an issue for some, but for us we typically have multiple test campaigns set up in support of a particular project. We compile/aggregate results manually. We have to look up/export results campaign by campaign, which is time-consuming.
- While the product, development/engineering, support and community folks at Optimizely are rock stars, their sales folks are not. Most of the people have impressed us in so many ways - so very customer-oriented, as Optimizely tends to be as a company/brand. However, from our perspective it's felt a little like the sales folks are not even part of the same organization. Very 'traditional' salesy. :(
- We'd like to be able to set up multiple 'projects' in Optimizely for better/easier organization. However, each project requires its own code snippet for your site(s). This precludes us from creating multiple projects, and thus it's a little unwieldy in our interface. Also, since we have so many sites and campaigns, our code snippet is much larger than we would like. This again is a symptom of our own organization, where we have many, many sites (unlike many other companies).
Why we're happy with Optimizely
- The interface is extremely easy to use. If you are a novice, the WYSIWYG allows you to easily make simple changes to your site. If you are comfortable doing so there is also the code editor which is powerful enough for you to make dramatic changes to your site.
- Optimizely's support is top notch. This was the primary reason we switched from our previous testing provider. They have extensive knowledge in scripting and are always quick to respond with useful information.
- Optimizely has created a community of A/B testers and optimizers that allows us to share ideas with other groups.
- Optimizely doesn't seem as buttoned up on the sales side as they are on the product and support side. Granted those other groups set a very high bar.
- Optimizely doesn't have a useful engagement metric out of the box. You have to integrate your analytics platform with it which for us was a challenge.
- We were grandfathered in from a Platinum plan which Optimizely no longer uses; they now have Enterprise and Starter I believe. We assumed since we had the Platinum plan that we were converted to the Enterprise plan but when we tried to access certain features we realized that wasn't the case.
- When we started implementing Optimizely on mobile (iOS and Android), the team at Optimizely was particularly helpful in working with the engineers and product team. This really accelerated the process for us to dive in and get started using the product.
- The Optimizely team seems very passionate about helping their customers get the most out of their platform. They are always available to answer an e-mail, support request, forum post, jump on a phone call and even come by for a visit. On the flip side, they are always asking for feedback on the product, which provides a sense that we are working with a living, breathing company.
- Optimizely has spent a lot of time understanding the intricacies of multivariate testing and statistical analysis and made it dead simple to see the results in their product. Even if your focus isn't around data, you can plainly see how your experiments are performing and focus on what you are best at. If you want to learn more, they've provided a lot of resources to help you understand concepts like statistical significance.
- I'd like to see more APIs to be able to export data and integrate it ina data warehouse so that we could perform our own analytics.
- Sometimes it's not obvious how long an experiment will need to run based on a product's existing traffic. Having an estimate integrated directly into the experiment would be very helpful.
- Sometimes I am concerned that they've made things too simple to get in and start running experiments, without having proper constructs on what experiments you should be running and what you are actually testing. Just because you've seen results on testing a button in an early stage of a funnel does not mean your end conversion is going to change.
Optimizely-Improve your website
- A/B Testing
- Goals Results
- Integration with GA
- Storing unnecessary data
Why we chose Optimizely
- Optimizely UX is very easy to use and find what you're looking for / set-up small changes on the site.
- Customer service has been amazing.
- There is a great user community that helps answer questions. Chances are, someone has run into the same issue you're experiencing.
- In some instances page load speed can be affected by the Optimizely tag. Fortunately, in all cases that this has impacted a test, customer service has helped speed up the test.
- Some changes require dev effort. It would be nice if the tool had more functionality to limit dev and design resources.
Your first step in A/B testing.
- Plug and Play
- Transparent
- Helpful
- Make sure that other elements can be tested throughout a back-end experience as well.
Best tool to learn about your client base.
- Ease of use, no need to involve the developers when we want to test on the front-end.
- Stable platform. So far we never had any noticeable outings or downtime because of it.
- Powerful tools for advanced users. Writing your own code and skipping the WYSIWYG editor is a breeze.
- The company is industry leading, hence the pricing is adjusted accordingly. Makes sense for bigger companies, not so much for small businesses - unless of course they can leverage use on a daily basis with significant results.
Optimizely Should Be Your Testing Software of Choice!
- Easy to use
- Analytics and data is easy to comprehend
- Very fast learning curve to be using the product to its fullest!
- Ability to auto-pause experiments at a set time.
Great product, amazing team perfect for A/B testing
- Product is very easy to implement and to ramp up with. Even non developers can use it.
- Extremely easy to manage your experiments, to turn them on/off and dashboard very user friendly.
- Great capability for user targeting, traffic allocation, etc.
- Reliable analytics tool (they've made a lot of progress in 2015 with their new results page).
- Amazing team, very nice to work with, always on top of things and very efficient technical support.
- Using Optimizely has enabled us to double our testing velocity.
- We also use optimizely for promotion campaign or quick bug fixes.
- Sometimes the software can be slow, the dashboard takes some time to load, etc.
- We still use our internal tool for more complex tests with back-end part.
- I am still paranoid about the flicker and don't use Optimizely for big tests if it is above the fold.
Do you have another tool for back-end tests?
Before Optimizely we had no tools to do A/B testing (beyond inacurate Google Analytics week by week changes) so this was a huge upgrade and allowed us to quickly iterate, experiment and learn from our users.
- You can set up an experiment without any assistance from your IT staff (after implementation). We don't have to request and wait. We just create a test, deploy and learn.
- The software is fairly easy to use, flexible and powerful–even for a non-coder.
- The reporting is really easy to digest and gets you a detailed picture of your test and your testing goals. If you choose to implement the test 100% that is simply a click away.
- The reporting could be better, maybe more standard. We use Google Analytics on top of the Optimizely reports to verify and dig deeper.
- If you aren't careful you could easily break your site. This isn't a flaw exactly, but is a potential pitfall.
- I've heard complaints from IT staff that Optimizely adds too much weight to the page loads.
- A/B testing configuration ease & deployment
- A/B testing results
- Content managment system through Javascript
- Personalizating different audiences
- Mobile App for account management
- Mobile Web responsive
- Results Exporting
- Scenarios recommended when internal decisions are based on "beliefs" with many sides and point of views
- Recommended when implementation timing gets too long for minor changes
- Recommended for segmenting audiences and personalizing experiences
- Recommended from short businesses to big ones
- Support is one of the best things Optimizely has
Optimizely is Best-In-Class!
- It's very easy to set up tests
- Their tech support is amazing
- it gives very quick and strong results.
- I would have liked to see improvements with the sitecatalyst integration.
Great tool overall, challenging for complex media sites
- Testing CTAs (calls to action)
- Simple HTML changes
- Measurement
- Analytics Integration (Google, SiteSatalyst, etc.)
- Easy to use visual editor
- customer support
- Hard to run significant tests without a major development investment on our side--it seemed like it would be a lot easier to implement tests without the help of our dev team
- Seems slightly less well-suited for media sites or sites where the KPIs are more complex than e-commerce sites (simpler KPIs such as click here or buy now)