Optimizely Feature Experimentation vs. QuestionPro

Based on my experience with Optimizely Feature Experimentation, I can highlight several scenarios where it excels and a few where it may be less suitable. Well-suited scenarios: - Multi-Channel product launches - Complex A/B testing and feature flag management - Gradual rollout and risk mitigation Less suited scenarios: - Simple A/B tests (their Web Experimentation product is probably better for that) - Non-technical team usage -

Incentivized

QuestionPro is great for forced-choice and Likert-scale data, and it easily turns raw data into readable statistics. The simple interface lacks attractiveness but is easy to learn and use, even for members outside the digital age. QuestionPro is less effective for qualitative data and short answer responses in large quantities. Reporting and analytic features leave this information in the bulky design, clogging the report and preventing cohesive presentation of the data.

Incentivized

• It is easy to use any of our product owners, marketers, developers can set up experiments and roll them out with some developer support. So the key thing there is this front end UI easy to use and maybe this will come later, but the new features such as Opal and the analytics or database centric engine is something we're interested in as well.

Incentivized

• Before choosing QuestionPro, we evaluated it and Qualtrics, SurveyGizmo, and SurveyMonkey. A committee tested each of 141 key features on all four products. QuestionPro did well in the comparison and we have ended up using even more features that we tested. The summary of that work can be found here: https://oit.utk.edu/research/documentation/qualtrics-to-questionpro/.
• The way it handles anonymous surveys is particularly good. With most products, you have to remember to check the "anonymous" box to prevent it from saving email addresses or other identifiable data. But if you forget to do so, you can't tell by taking the survey. Question pro uses "Respondent Anonymity Assurance" that must be turned on by the company (a quick request from the chat window will suffice) and then it can never be turned off. This activates a link that appears on the bottom left-hand of each survey page. Clicking it takes the survey participant to a company page which assures them that the anonymous feature is indeed turned on.
• It has a very nice combination of great power and ease of use.
• The support we have received from the company has been excellent. Our team has worked with around 30 research software vendors for over 35 years and this company stands out as being extremely responsive to our needs. We told them we needed a migration tool to help us move from Qualtrics to QuestionPro and they created one in just a few weeks. They've added or improved a number of other features for us, at no charge.

• Would be nice to able to switch variants between say an MVT to a 50:50 if one of the variants is not performing very well quickly and effectively so can still use the standardised report
• Interface can feel very bare bones/not very many graphs or visuals, which other providers have to make it a bit more engaging
• Doesn't show easily what each variant that is live looks like, so can be hard to remember what is actually being shown in each test

Incentivized

• The current user interface for designing and editing questions is truly awful. A big step backwards from earlier user interfaces which were great.
• Customer support is not very helpful.
• Results of survey are not exported in a way that is easy to translate into R data frame.

Competitive landscape

Incentivized

It's a fine product, but it's also a very competitive field, so it wouldn't take much to knock QuestionPro from top tier status. I would like to see more functionality in all programs, so whichever program does that first is likely to get my money in the next round of budgeting.

Easy to navigate the UI. Once you know how to use it, it is very easy to run experiments. And when the experiment is setup, the SDK code variables are generated and available for developers to use immediately so they can quickly build the experiment code

Incentivized

QuestionPro is very easy to use. There are lots of question types and drag and drop functionality. There are lots of ways that make the platform easy to use

Support was there but it was pretty slow at most times. Only after escalation was support really given to our teams

Incentivized

They offer email, chat, and phone support. I have used the chat support several times. Response time was fast, but the rep did not always have the answer. Inquiries are answered quickly and thoroughly.

It’s straightforward. Docs are well written and I believe there must be a support. But we haven’t used it

Incentivized

Try every feature and test take your surveys multiple times, running through different scenarios.

When Google Optimize goes off we searched for a tool where you can be sure to get a good GA4 implementation and easy to use for IT team and product team. Optimizely Feature Experimentation seems to have a good balance between pricing and capabilities. If you are searching for an experimentation tool and personalization all in one... then maybe these comparison change and Optimizely turns to expensive. In the same way... if you want a server side solution. For us, it will be a challenge in the following years

Incentivized

First of all, QuestionPro has powerful futures when we compare it with others. I am not sure but somehow interface of the product is always important for some more than its functionality. QuestionPro has many features, functionality and has a great interface. It is easy to use and a software that is easy to understand.

Incentivized

had troubles with performance for SSR and the React SDK

Incentivized

• We have a huge, noteworthy ROI case study of how we did a SaaS onboarding revamp early this year. Our A/B test on a guided setup flow improved activation rates by 20 percent, which translated to over $1.2m in retained ARR.

Incentivized

• QuestionPro is being used to automate surveys that were previously done by hand. We would not have been able to do this without the anonymous tracking feature. It has cut turnaround of reports by more than half.
• This is of course more efficient, but this could be expected of any software that automates a survey for you.