Likelihood to Recommend Optimizely Web Experimentation is appropriate if you have a big budget and your company is mature enough in experimentation to take all the advantage of a such an expensive price model. If you are just starting into AB testing, or not enough budget, I do not recommend Optimizely Web Experimentation, as it becomes quite frustrating to manage usage limits and technical set up to avoid extra charges.
Read full review It's the best for A/B tests or tests where you need a broad audience. If you don't need a real niche audience (that you should recruit then rather yourself), then you can rely on UserTesting. It's good because it's so quick, almost instant, it takes about 10 minutes on average before I can start reviewing and documenting sessions.
Read full review Pros Considers every kind of experiment from simple code change, code additions, code removal, javascript functions, CSS changes, and split landing redirects. Very friendly UC interface for easy navigation of implementation of pages, click events, experiment set up, and metric set up. The ability to let me know when something has reach stat sig without having to do complicated math on my own ability to carry UTM parameters through landing page redirects with a click of a button. Read full review Taking notes with video clips. I can take notes while I am watching a session and I can create clips of UX issues or bugs occurring. This really helps when I share them with other teams. Exporting all your notes. I analyze all usability tests by exporting my notes from UserTesting. I wish I could bulk export multiple tests though. Setting up demographics. This helps me reach out to users who are similar to our audience. Setting up screeners. Some studies require specific audiences (using a specific tool etc.), by setting screeners I can make sure I reached the correct audience. Read full review Cons Filtering capabilities in the reporting tab. To be able to filter based on a custom metric Tracking scroll depth within the reporting tab A more interactive graph within the reporting tab. Ability to manipulate there vs having to update the whole page Read full review The organization of the work spaces/dashboards could use some work. I don't feel as supported as I used to from a customer success perspective. I used to have dedicated, go-to professionals to reach out to about my account who knew my project context, account history, etc. Recently we've been pushed into general support pools, and it doesn't feel like my account gets any personalized attention anymore. I would like to see a more refined experience for collaborators - the test plans I share with them are plain text/hard to read. Would be nice to be able to conduct focus groups using this tool. Read full review Likelihood to Renew Because it's an incredible and essential tool for my line of work as a conversion optimization specialist. Really couldn't do my job nearly as effectively without it. It's paid for itself many times over and I feel like I'm only beginning to unlock the tools potential.
Read full review I'm very happy with my experience of the product and the level of service and learning resources they provide. If the service becomes more expensive than it currently is then we might not be able to justify additional cost - but this is theoretical. I would recommend UserTesting and would ideally renew our contract.
Read full review Usability Usability is mostly great. I like the WYSIWYG functionality and adding in real code is simple as well. It's easy to target specific pages or audiences. I've knocked a couple of points off because of how difficult it is to set up URL redirect experiments, confusion around creating pages, and lack of data that can be further analyzed.
Read full review User Testing is very easy to use. You first select the profile of your participants. This is as simple as checking on checkboxes and creating some questions to target the right market and user. You may also just send the tests to your own customers with a simple link. After selecting your audience, you just need to drag and drop the tasks and set them up. You're always guided throughout the process and if you're missing something, the user testing platform will let you know. Really simple stuff.
Read full review Reliability and Availability I would rate Optimizely Web Experimentation's availability as a 10 out of 10. The software is reliable and does not experience any application errors or unplanned outages. Additionally, the customer service and technical support teams are always available to help with any issues or questions.
Read full review Never encountered any problems
Read full review Performance I would rate Optimizely Web Experimentation's performance as a 9 out of 10. Pages load quickly, reports are complete in a reasonable time frame, and the software does not slow down any other software or systems that it integrates with. Additionally, the customer service and technical support teams are always available to help with any issues or questions.
Read full review Perfectly fine. Never had any problems.
Read full review Support Rating They always are quick to respond, and are so friendly and helpful. They always answer the phone right away. And [they are] always willing to not only help you with your problem, but if you need ideas they have suggestions as well.
Read full review Taking a long time When we had issues or questions, it often took a long time to receive a response from the support team. This delayed our testing and caused frustration for our team.
Not helpful in our specific questions
When we did receive a response, it often did not fully address our specific questions or issues. This made it difficult to resolve problems or move forward with our testing.
Read full review Online Training The tool itself is not very difficult to use so training was not very useful in my opinion. It did not also account for success events more complex than a click (which my company being ecommerce is looking to examine more than a mere click).
Read full review Implementation Rating In retrospect: - I think I should have stressed more demo's / workshopping with the Optimizely team at the start. I felt too confident during demo stages, and when came time to actually start, I was a bit lost. (The answer is likely I should have had them on-hand for our first install.. they offered but I thought I was OK.) - Really getting an understanding / asking them prior to install of how to make it really work for checkout pages / one that uses dynamic content or user interaction to determine what the UI does. Could have saved some time by addressing this at the beginning, as some things we needed to create on our site for Optimizely to "use" as a trigger for the variation test. - Having a number of planned/hoped-for tests already in-hand before working with Optimizely team. Sharing those thoughts with them would likely have started conversations on additional things we needed to do to make them work (rather than figuring that out during the actual builds). Since I had development time available, I could have added more things to the baseline installation since my developers were already "looking under the hood" of the site.
Read full review From a technical perspective, the implementation was extremely smooth. Most of the change management / implementation hurdles were clearing use of the tool through our various security, legal, and information privacy teams. Once these concerns were addressed (UserTesting.com was very helpful in providing all the needed documentation), the implementation process was very simple and we were able to get going right away.
Read full review Alternatives Considered Overall, the tools we compared against were great, but we went with Optimizely because it has all the features we needed and has the market leadership that gave us trust we would be successful in our experimentation efforts.
Read full review UserTesting allows for a quicker recruiting process for our studies. Additionally, UserTesting has more unmoderated research features and capabilities. I think that their payment model is also easier than UserInterviews. We typically user UserTesting for reaching our hard to reach segments and recruiting prospective users not already within our product user population.
Read full review Scalability While granted, I am not the one making the changes to deploy the software; it seems basically as simple as placing the optimized snippet on your site. If you place it in the header, it should reach across your entire site easily. I do think the test setup can be a bit temperamental at times, and having too many cooks in the kitchen can lead to errors. So, I would caution how many people have the access needed to make actual changes.
Read full review The package we have limits the number of people who can set up tests. This prevents us from scaling the use of the platform.
Read full review Return on Investment Customer retention: We've reduced subscription service client churn by 20%+ using optimized unsubscribe flows. Risk mitigation: Testing into full site redesigns has saved clients millions of dollars. Feature prioritization: Identifying what painted door changes add value has allowed developers to focus on changes that add hundreds of thousands or even millions to the bottom line. Read full review Cost savings of over $100K as we dont involve our research vendors as much as we used to do in the past Better tested experience get out in the market quickly even when we don't have enough time available with our researchers; our designers who are well-trained on usertesting.com can test out quickly Read full review ScreenShots Optimizely Web Experimentation Screenshots