Optimizely Web Experimentation vs UserZoom

• Optimizely makes it extremely easy to setup almost any type of split or multi-variate testing experiment. It has a variety of test triggers types & many ways to segment incoming visitors.
• I believe Optimizely's results dashboard is one of the best among split testing software. It allowed my team to get insights into the running of tests, like time to significance and a view of metric changes over time.
• Optimizely is very easy to use for non-engineers. It has a powerful visual interface that allows marketers and advertisers to setup complex variations without 1 line of code.
• In terms of documentation and implementation, I believe Optimizely is ahead of its competitors in terms of its focus on reliability and speed.

Read full review

• Quantitative testing can be done on the cheap compared to other quant tools or vendors... once you buy the license, you only pay for additional recruiting and you are managing the project, so there isn't the high cost of having other vendors doing the project management or running a survey etc.
• Farily robust means of tracking and recording data. I also relied heavily on the excel spreadsheet of all the raw data, which you can have UZ customize for you to get what you need.
• Great support system from the UserZoom team - they have researchers who can help with coding and also provide research support for a very good price.
• Also they have tree testing, card sorting, survey capabilities (in addition to usability testing)... and mobile testing. and they seem to be continously be developing the tool to support additional methods.

Read full review

• Actual creation of an A/B test is confusing and difficult (like the differences between Pages vs URLs etc.).
• Setting up a URL Redirect Experiment is so convoluted it seems like it's not even a feature Optimizely wants you to use. Instead, it's like I'm tricking Optimizely into alternating between 2 URLs. That really needs to be cleaned up as it would be very helpful.

Read full review

• Compared with some other tools, because their video conferencing platform is web-based (vs. using Zoom, which has a desktop app), I find the connection buffers a lot and it's kind of annoying
• Their screening tool on the UserZoom side (not Go) is clunky. Just let me select May, Must, or Should not for criteria rather than the weird interface design they have now.
• Logic panel is a little confusing, sometimes I do a double-take, sometimes it's wonky and resets itself. It's just not as easy to use as it could be.
• They haven't integrated UserZoom and UserZoom Go (formerly Validately)
• It's expensive!
• We signed up for a high sample size license with unlimited tests, but we're finding their panel doesn't fill as fast as their biggest competitor, UserTesting. We've struggled to fill n=100 sample tests with fairly general criteria.
• A lot of their licenses only allow limited concurrent studies, whereas UserTesting had unlimited tests that you could launch at a time. It's slowed us down.

Read full review

• Optimizely has had a tremendous positive impact on the ROI of our clients. Whether you're just starting your testing program and just want to get your feet wet with a few simple WYSIWYG tests, or you're ready to dive in to complex MVT test designs and Personalization Campaigns, Optimizely has everything you need to create, and maintain, a highly successful testing center.

Read full review

• Overall the ROI has been fairly useful. The price for the tool itself on occasional use cuts into our profit margin, and as I mentioned previously it's difficult to predict how much UserZoom will charge for a given project.
• On a positive note, UserZoom allows us to run projects that we wouldn't be able to on any other tool. We have landed projects with new clients that wouldn't be possible without UserZoom, so that helps drive future business in other areas.
• On the negative side, the fact that UserZoom also provides consulting research services means we can sometimes be in direct competition with them over clients, which allows them to beat our pricing since they own the tool. All's fair in love and war, but it is a tricky situation.

Read full review