Optimizely Web Experimentation

Create and test variations of a website, changing site elements such as headlines, CTAs, images, page design and layout, technical SEO changes, and new feature additions and collect statistical results of each variation’s conversion rates or other metrics.

Test out larger design changes by splitting your site traffic across two different landing pages to identify which site performs the best. It can be used to test the impact and feasibility of things such as new designs, personalization efforts, and new site architecture.

Ability to test multiple site design changes at once across one or multiple variations and identify which variation impacts conversion rates, or other predefined goals, the most.

Create an experiment that makes changes across multiple pages, like a funnel or a site-wide experience.

Preview your experiments across multiple browsers at once.

Ability to run tests to optimize mobile applications.

Ability to set the statistical significance level, or confidence interval, of a given test, for example at the 90% or 95% level.

Set up A/B testing campaigns using a WYSIWYG editor to create site versions and preview design changes before testing them. These editors often don’t require coding knowledge in order to operate them.

Allows users to create and edit experiments with HTML, CSS, JS.

Create on-page surveys and select which segment of users are asked survey questions using defined audience segments (e.g. new vs. returning users, mobile users, desktop users, etc).

Watch recordings of user sessions to gain insights on site visitor behavior and identify areas to improve site visitor experience.

Preview your experiment before running it live on your site or app.

Automatic calculation of the estimated test duration needed to gain statistically significant results.

Ability to schedule experiments to run, or not run, during specific times (e.g. Not to run during a holiday weekend or while a site-wide promotion is going on).

Ability to assign different phases of the experiment process to your team and approve next steps for an experiment or campaign.

Ability to activate an experiment after the page’s initial load based on a set of conditions (e.g. if the visitor takes certain actions).

Ability to run client-side tests (e.g. A/B, A/B/n, multivariate, and funnel tests) to test out UI changes.

Ability to run server-side tests (e.g. A/B, A/B/n, multivariate, and split URL tests) to test out more complex design changes, roll out features to specific audience segments, or split site traffic between different site versions.

Ability to make tests mutually exclusive so that a given visitor is only part of one test at a time, this helps prevent tests from interfering with one another.

Ability to segment, or target audiences based on criteria you set (e.g. URL, cookies, IP address, custom javascript, traffic source, device, browser, language, ad campaign, geo-targeting, time of day) and enable tests to run for specific visitor segments.

Ability to segment, or target audiences based on whether or not they have performed certain actions, such as clicking on a CTA, and enable tests to run for specific visitor segments.

Ability to set what percentage of website traffic receives specific test variants in order to roll out code only to a subset of site visitors.

Ability to optimize user experience for individual site visitors based on certain characteristics and past actions (e.g. past purchases, geolocation, demographics, device type, referral source, etc..). An example of this is product and/or content recommendations based on visitor characteristics.

A tool that shows which elements of the page generate the most visitor engagement.

Click analytics reports display how many clicks certain page elements receive and provides visitor engagement insights.

Scroll maps display how far down the page users scroll.

Enables users to view visitor interaction with forms and identify which parts of the form visitors fill out first and which fields lead to increased visitor drop-off.

Enables users to set up and customize conversion funnels to track site visitors' journeys and determine areas that see the most visitor drop-off.

Enables users to set up key website/mobile performance metrics on their landing pages and track them.

Provides users with reports for each test that record the performance of each variation tested against selected metrics such as conversion rate. These reports indicate when a given test variation has performed statistically better than the original (control) site version.

The ability to segment test results by specific criteria (e.g. browser type, device type, source, time of day, campaign).

Ability to export test results as a CSV file.

Provides users with a dashboard displaying test results for all live tests. Some products may also include test result histories accessible from the dashboard.