Maze vs. Optimizely Feature Experimentation

Maze User Testing is great if you're interested in doing user research from the comfort of your own desk. You can easily setup usability tests, surveys, card sorting and tree tests among other things to get a better understanding of how customers use your product. The only limitation at the moment with Maze that I can identify is only being able to do unmoderated tests, so if you'd like to be able to ask follow up questions in the moment, Maze is not the tool for you.

Incentivized

Based on my experience with Optimizely Feature Experimentation, I can highlight several scenarios where it excels and a few where it may be less suitable. Well-suited scenarios: - Multi-Channel product launches - Complex A/B testing and feature flag management - Gradual rollout and risk mitigation Less suited scenarios: - Simple A/B tests (their Web Experimentation product is probably better for that) - Non-technical team usage -

Incentivized

• Reporting is top-tier with filtration, heatmaps, user data, and public URLs for stakeholders
• Figma integration with user testing software is about as fast as it gets
• The experience for testers is practically seamless going from our site to a Maze. Loads of completed Mazes.

• Splitting traffic between variants and enabling you to scale up or down the amount of traffic in each one
• Giving a standardised report that you can share with a huge number of users
• Showing a large variety of results/metrics you can then dive into

Incentivized

• Change/Audit log to understand who is doing what and when
• Some simpler templates for simpler situations
• Additional means to take data out into 3rd party products for advanced analytics

Incentivized

• Difficult integration if your data is not front end
• Costly MAU model needs to be based on experiments not on site visits
• It's not easy to understand how to build an Experiment
• Onboarding team is more focused on punching through their slides and not focused on your needs or understanding.

Incentivized

Competitive landscape

Incentivized

Easy to navigate the UI. Once you know how to use it, it is very easy to run experiments. And when the experiment is setup, the SDK code variables are generated and available for developers to use immediately so they can quickly build the experiment code

Incentivized

Any issues that presented themselves were dealt with in a quick and efficient manner and fully rectified by the knowledgeable team over at Maze.

Incentivized

Support was there but it was pretty slow at most times. Only after escalation was support really given to our teams

Incentivized

It’s straightforward. Docs are well written and I believe there must be a support. But we haven’t used it

Incentivized

A Lookback is an alternative option if you think Maze User Testing is quite expensive for you, but look back has a different approach to Maze User Testing. Lookback focuses on qualitative usability testing instead of quantitative UserTesting. And also, Maze User Testing has a free option but Lookback doesn't have it, but Lookback has a cheaper option at $19/month than Maze.

Incentivized

When Google Optimize goes off we searched for a tool where you can be sure to get a good GA4 implementation and easy to use for IT team and product team. Optimizely Feature Experimentation seems to have a good balance between pricing and capabilities. If you are searching for an experimentation tool and personalization all in one... then maybe these comparison change and Optimizely turns to expensive. In the same way... if you want a server side solution. For us, it will be a challenge in the following years

Incentivized

had troubles with performance for SSR and the React SDK

Incentivized

• Easy to run quant test
• Easy to test with large number of people on production
• Easy to run unmoderated competitor studies

Incentivized

• We have improved various metrics throughout the course of our experimentation program with Optimizely and therefore sharing numbers is tricky. Essentially we only implement versions of the product that perform the best in terms of CVR, revenue/visitor, ATV, average order value, average basket size and so forth dependent on the north star we are trying to move with each release.

Incentivized