Skip to main content
TrustRadius
Optimizely Web Experimentation

Optimizely Web Experimentation

Overview

What is Optimizely Web Experimentation?

Optimizely Web Experimentation empowers teams to conduct experiments (without having to rely on developer resources) in order to test various user interactions, make website changes backed by data, and personalize customer experiences.

Read more
Recent Reviews

TrustRadius Insights

Ease of Use: Users consistently praise the platform for its elegant and user-friendly design, which enhances productivity across various …
Continue reading

Optimizely Web Experimentation Review

8 out of 10
June 22, 2024
We use the Optimizely Web Experimentation software to facilitate our A/B testing on the front end of our website. This allows us to freely …
Continue reading

Not value for money

5 out of 10
April 24, 2024
Incentivized
We use Optimizely Web Experimentation to launch different experiments in our site. We have different type of audiences, coming from …
Continue reading

Optimizely Review

8 out of 10
April 23, 2024
Incentivized
We use Optimizely for A/B testing and soft rollouts of changes. This allows us to test and make changes without needing to use engineering …
Continue reading

Good platform

8 out of 10
April 23, 2024
Incentivized
I have used Optimizely in order to conduct simple AB tests and more complex MVT test to improve an ecommerce website. We had issue …
Continue reading

Data has no opinions

8 out of 10
April 16, 2024
Incentivized
It's simple for us, we want to make data driven decisions and Optimizely Web Experimentation gives us the tools and data we need to …
Continue reading
Read all reviews

Awards

Products that are considered exceptional by their customers based on a variety of criteria win TrustRadius awards. Learn more about the types of TrustRadius awards to make the best purchase decision. More about TrustRadius Awards

Popular Features

View all 33 features
  • a/b experiment testing (105)
    9.3
    93%
  • Standard visitor segmentation (97)
    8.9
    89%
  • Test significance (96)
    8.5
    85%
  • Preview mode (94)
    8.0
    80%

Reviewer Pros & Cons

View all pros & cons

Video Reviews

1 video

Optimizely Web Experimentation Review from a Director of Partnerships
08:08
Return to navigation

Pricing

View all pricing
N/A
Unavailable

What is Optimizely Web Experimentation?

Optimizely Web Experimentation empowers teams to conduct experiments (without having to rely on developer resources) in order to test various user interactions, make website changes backed by data, and personalize customer experiences.

Entry-level set up fee?

  • Setup fee optional
For the latest information on pricing, visithttps://www.optimizely.com/plans/

Offerings

  • Free Trial
  • Free/Freemium Version
  • Premium Consulting/Integration Services

Would you like us to let the vendor know that you want pricing?

46 people also want pricing

Alternatives Pricing

What is Oracle Marketing?

Oracle CX Marketing (formerly Oracle Marketing Cloud) is a solution designed to enable marketers to plan and execute automated marketing campaigns via email, display search, video advertising, and mobile while delivering a personalized customer experience for their prospects.

What is Instapage?

Instapage, from AirSlate since the October 2023 acquisition, offers post-click automation to help digital advertisers create more relevant and impactful advertising, unifying the post-click stage of the advertising funnel. Instapage helps marketers to maximize conversions by creating,…

Return to navigation

Features

Testing and Experimentation

These features enable companies to plan, set up, and execute different types of tests (e.g. A/B, A/B/n, multivariate, split URL tests).

8
Avg 8.3

Audience Segmentation & Targeting

A set of tools used for website optimization experiments (e.g. A/B, A/B/n, funnel, split URL, multivariate tests) that can help users segment their audience in to different groups for the purpose of exposing specific audiences to tests or personalization efforts.

8.6
Avg 8.7

Results and Analysis

Tools that allow users to evaluate the results of website optimization tests (e.g. A/B, A/B/n, multivariate, and split URL tests), or view visitor interaction with webpages and specific site elements.

8.3
Avg 8.5
Return to navigation

Product Details

What is Optimizely Web Experimentation?

Optimizely Web Experimentation empowers marketers to conduct fast experiments and personalization campaigns without relying heavily on technical resources. Marketers can test various user interactions, make website changes backed by data, deliver personalized customer experiences, and do it all from edge networks.

Optimizely provides a single workspace for experimentation collaborators to have control and visibility across their experiments — from setting up experiment components to interpreting test results — ultimately to improve velocity and collaboration, and elevate performance.

Optimizely Web Experimentation Features

Testing and Experimentation Features

  • Supported: a/b experiment testing
  • Supported: Split URL testing
  • Supported: Multivariate testing
  • Supported: Multi-page/funnel testing
  • Supported: Cross-browser testing
  • Supported: Mobile app testing
  • Supported: Test significance
  • Supported: Visual / WYSIWYG editor
  • Supported: Advanced code editor
  • Supported: Preview mode
  • Supported: Test duration calculator
  • Supported: Experiment scheduler
  • Supported: Experiment workflow and approval
  • Supported: Dynamic experiment activation
  • Supported: Client-side tests
  • Supported: Server-side tests
  • Supported: Mutually exclusive tests

Audience Segmentation & Targeting Features

  • Supported: Standard visitor segmentation
  • Supported: Behavioral visitor segmentation
  • Supported: Traffic allocation control
  • Supported: Website personalization

Results and Analysis Features

  • Supported: Form fill analysis
  • Supported: Goal tracking
  • Supported: Test reporting
  • Supported: Results segmentation
  • Supported: CSV export

Platform Integration Features Features

  • Supported: API
  • Supported: Web analytics integration
  • Supported: Content Management System Integration
  • Supported: Integration with CRM or DMP software

Optimizely Web Experimentation Screenshots

Screenshot of the visual editor, used to make changes to the appearance and layout of a website.Screenshot of the results dashboard that displays experiment results.Screenshot of the integrations page, where integrations can be enabled.Screenshot of Advanced Audience Targeting: Custom Real-Time Segments provide customers with the ability to create rapidly-updating segments which can be used as audiences for testing or personalization. These segments can be based on profile data, omni-channel behaviors, and AI-based observations.Screenshot of AI Content Suggestions: AI-powered suggestions in the Visual Editor accelerate ideation.Screenshot of Extensions for Edge: reusable elements minimize developer dependency when launching multiple tests with the same element.

Optimizely Web Experimentation Video

Optimizely Web Experimentation Competitors

Optimizely Web Experimentation Technical Details

Deployment TypesSoftware as a Service (SaaS), Cloud, or Web-Based
Operating SystemsUnspecified
Mobile ApplicationNo

Optimizely Web Experimentation Downloadables

Frequently Asked Questions

Optimizely Web Experimentation empowers teams to conduct experiments (without having to rely on developer resources) in order to test various user interactions, make website changes backed by data, and personalize customer experiences.

AB Tasty, VWO, and Adobe Target are common alternatives for Optimizely Web Experimentation.

Reviewers rate Traffic allocation control highest, with a score of 9.4.

The most common users of Optimizely Web Experimentation are from Mid-sized Companies (51-1,000 employees).
Return to navigation

Comparisons

View all alternatives
Return to navigation

Reviews and Ratings

(520)

Community Insights

TrustRadius Insights are summaries of user sentiment data from TrustRadius reviews and, when necessary, 3rd-party data sources. Have feedback on this content? Let us know!

Ease of Use: Users consistently praise the platform for its elegant and user-friendly design, which enhances productivity across various tasks such as A/B testing.

Reporting: The robust reporting capabilities, multi-account support, and advanced settings have been highlighted to be key strengths provided by the tool by reviewers who value detailed results engine provided by the platform.

Streamlining: Additionally, users commend the intuitive WYSIWYG editor for simplifying editing processes and aiding in thorough live and post-test analysis to streamline experimentation workflows. The ease of experiment setup with metrics tracking features further adds to the platform's appeal and has provided a quick streamlining experience.

Slow User Interface: Users have reported that the user interface can be sluggish, impacting their overall experience negatively. Some find the platform's navigation confusing, leading to inefficiencies. The WYSIWYG editor is described as clunky, hindering quick and effective changes, affecting user productivity.

Steep Learning Curve: Non-technical users may face usability barriers on the platform, needing improvements for easier use without technical expertise. Having a longer learning curve period hinders by spending more time getting to know the product rather than getting work done.

Limitations: Additionally, users desire more a la carte feature availability rather than being restricted by tier-locked features.

Attribute Ratings

Reviews

(1-5 of 5)
Companies can't remove reviews or game the system. Here's why
Score 10 out of 10
Vetted Review
Verified User
Incentivized
Optimizely is the key tool we utilise for our onsite optimisation and personalisation programs. It is used across multiple divisions of our company. We utilise it to allow us to quickly test new concepts or hypothesis online. It has enabled us with a much faster speed to market and minimises the risk by making improvements onsite.
  • Integrates well with our digital marketing stack
  • Easy to use
  • Provides detailed reporting out of the box
  • It is expensive in comparison to some of its competitors
  • Full stack component can be hard to get up and running
Optimizely is a very powerful tool for onsite A/B and multivariate testing. It also provides very good audience identification and selection to allow running of multiple tests concurrently. In addition it has good personalisation capabilities which has allowed us to start proving the concept and value of of onsite personalisation quickly and easily.
  • Experimentation
  • Personalisation
  • Reporting
Testing and Experimentation (19)
71.05263157894737%
7.1
a/b experiment testing
100%
10.0
Split URL testing
100%
10.0
Multivariate testing
100%
10.0
Multi-page/funnel testing
100%
10.0
Cross-browser testing
100%
10.0
Mobile app testing
N/A
N/A
Test significance
80%
8.0
Visual / WYSIWYG editor
80%
8.0
Advanced code editor
100%
10.0
Page surveys
N/A
N/A
Visitor recordings
N/A
N/A
Preview mode
90%
9.0
Test duration calculator
80%
8.0
Experiment scheduler
90%
9.0
Experiment workflow and approval
80%
8.0
Dynamic experiment activation
N/A
N/A
Client-side tests
90%
9.0
Server-side tests
70%
7.0
Mutually exclusive tests
90%
9.0
Audience Segmentation & Targeting (4)
90%
9.0
Standard visitor segmentation
90%
9.0
Behavioral visitor segmentation
90%
9.0
Traffic allocation control
90%
9.0
Website personalization
90%
9.0
Results and Analysis (10)
49%
4.9
Heatmap tool
N/A
N/A
Click analytics
N/A
N/A
Scroll maps
N/A
N/A
Form fill analysis
N/A
N/A
Conversion tracking
100%
10.0
Goal tracking
100%
10.0
Test reporting
100%
10.0
Results segmentation
90%
9.0
CSV export
N/A
N/A
Experiments results dashboard
100%
10.0
Platform Integration Features
N/A
N/A
  • Faster speed to market for tests
  • Allowed us to minimise development effort
  • Improved onsite conversion
Whilst Optimizely is more expensive (for us at least) we found it was far technically superior and easier to use. We have not run into the technical constraints that we have with other tools. In addition when we went to market to evaluate different suppliers optimizely won out in terms of features, capability and usability. This more than offset the additional cost.
Digital Marketing
Digital Development
4
Technical competency
Understanding of digital marketing technologies
  • Onsite experimentation
  • Onsite personalisation
  • A/B Testing
  • Utilise it to quickly deploy messages on site during outages
  • Mobile app experimentation
We are about to enter contract renewal and at this stage we ware vbery unlikely to even look at other solutions.
Yes
Visual Website Optimizer - as it wasn't able to run the tests we wanted to.
  • Price
  • Product Features
  • Product Reputation
The single most important factor was it's ability to integrate with our site architecture and technologies.
We wouldn't. The ability to work well with our react apps was critical to us.
Very good, they are always abvailable and respond in a timely fashion.
No, we didn't feel we needed it and this has proven to be correct.
No
During rollout - we had a number of challenges integrating experiments into our application architecture but optimizely were there to help us resolve the issues.
We found that our entire optimisation team was able to get up to speed quickly and easily. Whilst we received training as part of the onboarding it was still very easy to utilise and we have had no issues finding our way around it.
  • Reporting on results
  • Managing tests
  • None really
Yes, but I don't use it
We have not come close to limits with the tool. What is holding us back is the amount of traffic we receive which limits the number of tests we can run at any one time. We have found Optimizely to be highly performant and reliable with next to no issues observed.
Never ever had any issues. Our systems are less reliable than optimizely.
It performs very well. No issues with laggy page loads or other performance related issues.
Brynn Arego | TrustRadius Reviewer
Score 8 out of 10
Vetted Review
Verified User
Incentivized
We utilize Optimizely to test our clients UX on the Magento platform. It is easy to install and there is a handy module for Magento that makes it easy for clients to add the snippet to their site. Optimizely is very easy to use and even better if you have jQuery knowledge. It loads fast and allows you to test many things on the site without a lot of overhead and minimal effort. If you are a savvy user it is easy to work with. If someone isn't savvy and tries to use the editor there is a risk of breaking your site. For example, if working with a product detail page template, be wary of using their WYSIWYG editor on any dynamic information like pricing. It can turn it into a static html element on the page. Overall it's a great tool that allows you to be nimble with your testing and has great reporting on the results of your test.
  • Test setup is easy and doesn't take a lot of company resources.
  • The impact on page speed is incredibly small.
  • The results dashboard gives great insights to how your test is performing and you can track multiple goals.
  • I'd like to see more tools comparable to Visual Website Optimizer (VWO). Like user recordings and in page analytics.
  • If you have a dynamic area of information and edit it you pose a risk to making the page templates dynamic data static across the site. Pricing is a great example of this.
If you want better insight into how users are specifically interacting with a page, a different tool would be able to do this whereas Optimizely is lacking.
  • Increased click through rates from homepages for clients.
  • Improved conversion rates from product detail pages.
  • Tested category pages and increased conversions.
Optimizely is lacking a lot of the tools that VWO offers. My favorite being user recordings because of the valuable insight you get on the UX.
1
UX / CRO
1
Front-end development, jQuery, HTML, CSS
  • Testing CRO for clients
  • Testing internal lead generation
  • Testing our lead generation funnel using mutliple landing pages / funnels.
  • Testing checkout processes for our clients.
I'm starting to lean more towards VWO because of the added value and pricing.
No
  • Price
  • Product Features
  • Product Usability
  • Product Reputation
  • Vendor Reputation
None, but they change their pricing structure too often and we are considering other vendors.
  • Implemented in-house
No
Change management was a small part of the implementation and was well-handled
  • None.
If you use Magento they have a module to help with implementation.
  • Self-taught
It was easy and I became Optimizely platform certified.
The configuration is right for the product.
Implement the snippet high up in the <head> so you avoid any flashing while loading the test.
No - we have not done any customization to the interface
No - we have not done any custom code
No
I've only had to use it a couple of times but response time isn't the fastest.
No
They provided a great detailed explanation on testing pricing using a workaround method.
  • Changing static text or css elements.
  • Viewing the results. They are easy to interpret.
  • Editing anything dynamic.
No
It is easy to use and quickly learn. Setup is simple with the snippet and tracking is easy to interpret.
Score 7 out of 10
Vetted Review
Verified User
Incentivized
Used for simple A/B tests and to force visitor traffic to pages that have specific messaging or UI elements
  • Very easy to configure experiment targeting, traffic allocation, and other settings you would want to customize for your testing program
  • Allows easy selection of experiment types like Multivariate, Multipage, A/B
  • Permits simple UI changes using its code editor and other options which allows for decreased reliance on development resources. Product and Design teams can launch new experiments easily on their own
  • Support is first rate: knowledgeable, great communicators who take the time to explain things so that you can learn and apply to future experiments. Optimizely customer support is among the best I've encountered from a web software service
  • Improvements can be made with experiment results reporting specifically customization of data within the data export or whatever is viewed on the summary page
  • Different organizations could have different thresholds for statistical significance. Optimizely is set to 95%. It would be nice to have this value configurable.
  • It is difficult to ascertain whether a losing variation is a statistically significant loss (only shows low chance to beat baseline). Having this would assist with pausing variations that are not performing very well.
  • Customizable, distributable reports would be a great feature to add. Currently you must click through each individual experiment to see experiment details. Seeing a summary through email or selecting multiple experiments to view on one page would be a huge improvement for scanning health of tests.
Review how front end or back end-heavy your tests plans will be. Optimizely allows for easy implementation of front end changes which won't bog down development resources. They should be able to focus on bigger fish than changing button colors, texts, or adding images to pages.
  • Optimizely has helped us optimize on micro conversion goals throughout our site
  • Our macro conversion goals have also been optimized using Optimizely
  • Homegrown A/B platform and Adobe Test & Target
Not as strong.
5
Growth, Digital Product, Engineering
1
A/B testing knowledge, Front End JS/CSS, QA
  • Test new ideas/variations against current control version
  • Force visitors to see certain site changes
  • Reports to understand impact of site change
  • Segment traffic from the various inbound channels
  • Analytics
  • Performance indicators
Overall a great experience from ease of use and flexibility for the tool. Reporting improvements could be made, but what exists currently is enough to give businesses a sense of the directional movement of a change to the site.
No
  • Price
  • Product Features
  • Product Usability
Simple and effective tool to use for quick site changes and A/B tests.
Talk with power users for each product more, and understand what feature they wish they could have but currently don't.
No
Not sure if the benefits are justified for this particular product and organization.
Enterpriise support is great when they respond to a request, but if only on standard plan good luck.
No
When critical tests were impacting our paid campaigns due to issues with the way Optimizely handled our page, then my request was escalated and team worked to address right away.
  • Button color and copy
  • Segmenting to different audiences
  • Working with updating images in a slider carousel
  • Reporting
No
Decently usable but if doing advanced stuff it's not easy.
Sometimes when too many A/B experiments are running, weird things happen on the site. It'e best to only run a few amount of experiments concurrently.
For the most part Optmizely's availability has been solid.
Sometimes there are glitches for variations in production, but depends on the change.
Score 10 out of 10
Vetted Review
Verified User
We use Optimizely on our company website to primarily help increase our customers / visitors conversion. We are an eCommerce site so our primary focus is on CTA and making sure our customers are able to easily find what they are looking for. We primarily have used it to test layout changes, but have done some redirect testing as well.
  • Their User Interface is so easy to use. You don't have to be a coder to set up a test. It's simply amazing.
  • Their support is by far the best I've ever experienced. They always pick up the phone right away and are so helpful to answer your questions and to also give suggestions on better ways to do your tests.
  • Sometimes it's a little tricky using third party jQuery plugins that we have on our site. It's not necessarily bad, but it could be better.
  • The reporting is good, but it could be more robust. Simply a data table with a time chart. Would be nice to have more of a graphical interaction for their reports.
If you are trying to improve your website, Optimizely is the model standard. Their product is unbelievably easy to use, with the ability of advanced functionality. Their support team is always there to help you out. Really no key questions that I can think of, just be up front and be honest with what you want.
  • We have found new ways of displaying different aspects of pages, and have thusly increased our conversion and revenue. Some areas have higher areas of improvement. But you know quite soon if something is helping your business or not.
Mainly it was the cost. We have a seasonal business, and they were willing to work with us to meet our needs.

We've had others reach out to us to switch them, but Optimizely just stands out above them all in our eyes.
They always are quick to respond, and are so friendly and helpful. They always answer the phone right away. And [they are] always willing to not only help you with your problem, but if you need ideas they have suggestions as well.
Yes
Absolutely. I submit a ticket quite a bit, even for the simplest of questions. They always respond in a timely fashion and make sure my questions are answered.
Score 9 out of 10
Vetted Review
Verified User
Optimizely is used heavily across all our product lines of business' websites. It is primarily managed by our eCommerce team, executing tests asked for by business owners and/or through our own internal website optimization test ideas. The core of adopting Optimizely was to enable us to better manage website testing without the need for extensive development, due to our having a small eCommerce team. Its flexibility (after initial installs and modifications) allow non-technical colleagues to create and manage their own tests, which is vital given I am not able to support all items due to lack of resources beyond just myself. This is critical given the number of tests needed on an ongoing basis for us to continue to refine and improve our overall customer experiences.
  • Installing it is quite simple for more standard A / B testing. Only taking a few minutes to actually place code on site is a dream and you can almost immediately get tests designed, let alone launched. (If you had the right folks in place at the right time, I cannot imagine you could not get a live test going within a single business day.)
  • Their support team is highly active and responsive to requests for information or troubleshooting. (NOTE: We have the Platinum account giving us more dedicated support staff). They are highly knowledgable and able to answer most questions quickly; else, they are highly effective on researching issues with other members of their organization (which speaks to the company's fluidity)
  • The UI of the tool is very simple to use and elegant in design. That is an essential piece of why the tool is so effective / usable for non-technical staff.
  • Due to the nature of the tool, it can be difficult to make the tool work correctly with the standard install for tracking tests for more complex web pages, example being checkout pages. Due to checkout pages having a great amount of dynamic content (due to user input of forms, etc), it can conflict slightly with other online marketing tracking. Custom scripting is needed for this kind of testing, so its ease-of-install-use is a harder path for these more complicated pages.
  • There are certain times the online tool will not work correctly and have to relogin via a new browser session. In a crunch time for test creation, it can be a bit troublesome. (But a minor issue.)
  • The tool tends to be a bit "too smart" for tracking the changes made in creating a variation. Example is if you are working to change text on a page, say larger and different color. Through the process of using the tool to make the modifications, you possibly would try different sizes/colors. Each time you make a selection, the change is "logged" for the Optimizely test variation... this creates a great amount of "extra change code" that is unnecessary and have to clear it out via the code editor.
The tool is fantastic for simpler, easy A/B testing. Just getting that strip of code on your site can enable a great number of test options. Multi-variate testing is also not too difficult, only for more complex coded web pages.

When considering the tool, I would highly suggest (almost require) that a demo with the Optimizely team has participants from your entire organization who'd be involved -- business owners for budget/ongoing Goals, marketing team who likely will use the tool ongoing to support business owners, technical/web ops individual to understand nature of the technology, analysts who'd monitor and report findings.

The key questions to ask when considering:

- Do I have enough technical support to install / test / finalize the overall Optimizely code install? Can they help if I need more technical integrations?
- Do I have the right design resources in place to assist with building out specific assets needed for tests?
- Are the right people in place to monitor and track the performance?
- Do I have 6 months worth of tests in mind so I can start right away?
- What is the overall budget capacity to continue paying for the service to ensure a return on the investment?
  • Highly effective in providing internal efficiency in getting A/B tests live
  • Has shown us already opportunities to improve overall customer pathing throughout our websites, resulting in more conversions
  • Enables us to quickly remedy serious errors in content and/or add elements to site that may not be able to be added due to other development projects "locking" website code bases. (Recently, we had a large development features update being done, and we needed to launch a small edit to our footer. We used Optimizely instead and took about 5 minutes!)
  • Adobe Test and Target,Google Analytics,Monetate,Magento
I've had limited experience with some of these, extensive with others. I think overall they are solid tools, and mainly depends on the organization you're working for. There are a few main reasons we selected Optimizely, considering we are a very small eCommerce department servicing an aggressive direct-response organization:

1) Budget vs. Ease of Use. The pricing model of Optimizely fit within our budget, especially considering the ease of use of the tool for most tests. The others we considered were just too expensive (and better for enterprise-level / high budget organizations.)
2) Integration. For most, really just adding the script to your websites. Tools like T&T, though valuable, are a MUCH steeper learning curve to get onto your website, let alone it working effectively for reporting back on results. (NOTE: I at one point was a reseller of the T&T product).
3) Comfort with the team. Overall, the Optimizely support teams are highly knowledgeable, approachable and always ready to help. Customer service is vital.
There's no question we'll continue with the tool. The ROI is too good. We've already embedded it recently into a new customized eCommerce platform we launched. The design of it had Optimizely in mind and is one of the technologies we won't continue without.

I'm also confident future refinements of the tool / extended functionality will address some of the minor headaches for the complex testing we wish to do. I am certain of this given the number of iterations I've seen since starting with the tool.

It's definitely one of the top three technologies we're using on a daily basis and that says a lot, as we never could test before.
9
The entire eCommerce team uses the tool, but each individual is a different part of the process. Designers use to create test variations. Technical folks use for installing the tests/working more complex testing. Analyst maintains the tests for ongoing reporting. This combination of expertise on a small team needs a tool like Optimizely in order to handle testing smoothly. It also speaks to its ease of use given the various types of individuals who can all use it effectively.
3
To use it effectively, the ideal support team know basic web code to some degree. They also need a bit of creativity to understand/utilize the various test variation creation functions offered within the tool. Though its easy to use for even the most un-technical person, the management of the tool itself on the site / interactions with their support DOES need folks who are more technical in nature / background.
  • I need to test whether content A or content B will engage folks more on my website. (A/B)
  • I need to make an emergency change to the website and our webmaster is out of town. (Legal issues / quick fixes)
  • Would adding advertising to my site (banners) as a revenue stream impact my ability to convert people on my own site?
  • Adding snippets to my site due to not having ability to embed in actual site code.
  • Quickly hide pages within my site that were not supposed to be made live / but doing so remotely b/c I was not at the machine allowed to access the web server.
  • Utilize it for CSS changes globally across the site using the multi-page test variation.
  • Developing a full mobile-rich web experience with responsive design techniques / using Optimizely to understand what's working (since mobile optimization is somewhat new for us this year as a focus.)
  • Further split-test variations in relation to online marketing channel advertising / personalization of content.
  • Implemented in-house
Yes
Ph1 - Adding standard script to the website

Ph2 - Refine the script to allow for better full-path tracking along side of other OLM tracking (like GA), determined after some initial live tests.

Ph3 - Branching the technology across all other owned websites (each having a small flavor of the OLM and script enhancements.)
Change management was a minor issue with the implementation
It wasn't too difficult to utilize the technology once folks got comfortable. The change happened with the approach of "we can do tests, how do we decide what to do?" We needed to work with business owners to create an ongoing testing roadmap, which hadn't existed before. Before Optimizely, we were lucky to get even a few tests off the ground in a year's time. With Optimizely, the backlog of 50+ tests became the focus, and we had to really figure out how we manage this moving forward, as multiple tests would dilute results and/or clutter our entire website. A project management-esque process had to be introduced.
  • GA tracking. It would break and we couldn't see conversions via our GA reports for transactions involving tests. Took a bit of time with Optimizely to figure out a solution.
  • Being ready with tests. We were so focused on getting it installed, we didn't have anything on deck to actually test the implementation and had to resort to small minor changes to validate. We could have save a good amount of time had we been able to use the first planned test as the basis for ensuring the tech was working properly.
  • Getting the tech to span across entire website. Due to how our custom platform is developed, it was a bit more difficult than anticipated to install the Optimizely code across every single page of our website.
In retrospect:

- I think I should have stressed more demo's / workshopping with the Optimizely team at the start. I felt too confident during demo stages, and when came time to actually start, I was a bit lost. (The answer is likely I should have had them on-hand for our first install.. they offered but I thought I was OK.)

- Really getting an understanding / asking them prior to install of how to make it really work for checkout pages / one that uses dynamic content or user interaction to determine what the UI does. Could have saved some time by addressing this at the beginning, as some things we needed to create on our site for Optimizely to "use" as a trigger for the variation test.

- Having a number of planned/hoped-for tests already in-hand before working with Optimizely team. Sharing those thoughts with them would likely have started conversations on additional things we needed to do to make them work (rather than figuring that out during the actual builds). Since I had development time available, I could have added more things to the baseline installation since my developers were already "looking under the hood" of the site.
Yes
We knew testing would be a HUGE focus for us for the coming years and was critical we had a clear path to helping troubleshoot and resolve issues. Having the premium support team available to us allows that freedom. Additionally, we know as a business we are launching new product lines / sites frequently, so having the additional support available to us ensures we can keep on track (and not have to worry too much about the increased level of investment... )
They are just fantastic. We have our main account rep but then a few others have helped us out in a pinch (or if he's not available.) Always respond within same day of a request and the advantage for ongoing account reps is they know our business, is growing with our technical enhancements and then lead-time to resolve issues is much shorter b/c they're are effectively part of our own team.
No
We had a big problem -- there was a huge desire to test messaging / layout on our shopping cart page. We were about 2 days from our promised deadline and had a huge amount of traffic coming with planned-for online marketing campaigns. We realized during our build-out that there was an issue with "activating" the variation on the page. Our site code was designed in a way that Optimizely couldn't "see" the elements we were testing unless something was in the cart. Raising the issue with the Optimizely team, they needed about three hours to determine the root-cause of the issue and a proposed solution. They provided this to us same-day and I took it to my development team. We then were able to get this proposed fix in place on our side (effectively creating a small "beacon" for Optimizely to see) and we managed to get the test running just in time. (And the result was great and we ended up installing the variation as our new control!)
  • Starting a variation. Quickly asking for the URL, quickly loads and you're off and running to make your variation changes.
  • Targeting the traffic. Their simple sliders and/or input % to allocate the traffic (and how you target them) is very easy to understand and use.
  • Adding the script. It generates in seconds after creating a project and elegant in delivering the link to you to use for integration.
  • Preview functionality. It sometimes takes a while for the live preview to push to their servers for a full check.
  • Google Analytics integration. It allows for the insertion of your account information, but the tracking/tool sometimes would break more complex pages like for a checkout, and then would render our GA data to be incorrect. Took some time to refine the tool / adding additional scripts to get it right. (NOTE: Our GA reporting needs are slightly custom as far as the metrics we're tracking..)
  • Editor Settings VS. Live pages to target. It's a bit confusing to understand the difference between the two and sometimes would get confused that I wasn't developing off correct editor setting VS what page(s) I was actually activating the test on.
Yes, but I don't use it
It's overall great and easy to use. I think I don't rate it a 10 because I'm actually TOO technical... because its meant for a broad audience, I think I'm looking at the UI differently than someone with less technical expertise. It's to0 layman in some cases, but yet can also be confusing when dealing with their proprietary Optimizely programming language.
Return to navigation