If you plan to become a heavy user of A/B tests to optimize your business, I strongly recommend ABTasty over Kameleoon and Optimizely. Greatest ROI for sure.
Sylvain Marbach profile photo
June 15, 2017

If you plan to become a heavy user of A/B tests to optimize your business, I strongly recommend ABTasty over Kameleoon and Optimizely. Greatest ROI for sure.

Score 10 out of 10
Vetted Review
Verified User
Review Source

Software Version

Enterprise

Overall Satisfaction with AB Tasty

As a direct marketing/publishing group, we are using it to A/B test everything we do. AB Tasty brings this opportunity to the next level; our marketing department is fully autonomous. With strong reporting capabilities, combined with session recording and heat maps, we can really understand and optimise the user experience.

AB Tasty is now [being used] inside all marketing teams across all companies of the group. We started with the acquisition department to optimise both the volume and cost of leads. We then integrated this approach to increase the conversion rate on the order forms. We're now working on up-sells/cross-sells and renewals efforts.

After having reviewed Optimizely, Kameleoon, and AB Tasty... we benchmarked the last 2 solutions for 8 weeks. I was already using Kameleoon for a non-profit organization for 18 months but I opted for AB Tasty for an entire commercial group of 9 entities. I was convinced it was the right choice. I'm not the only one today. Provided you solve the minor issues while onboarding staff on the solution for their first tests... they will soon be autonomous. Some are now addicted to A/B testing. The ROI is excellent.
  • Increase conversion rate on leadgen pages by more than 22%
  • Increase content sharing on social networks for a petition by... 1870% (from 0,5% to almost 10%)
  • Increase ratio of recurring/one-shot donations for a non-profit organization by 285% (from 7% to 27%)
  • Session recording and heat maps are new features, and they are still below Hotjar capabilities... but Hotjar makes no difference today for versions of an A/B test... so I hope AB Tasty will further enhance those features to catch up.
  • Reporting and "insights" are great, but we [wish] AB Tasty could automatically analyze all the segment combinations to bring valuable info (like 8% overall increase but covering +16% on mobile and -2% on PC so we should exclude this segment from this optimized version).
  • Overall control on tests to send warnings like "you should not stop this test until it runs for one complete week", or "you should not consider overall performance as it is not stabilized" (when conversion curves are crossing each other so the winning version is not always the same...).
  • Excellent ROI on the acquisition of leads, great ROI for conversion and monetization of customers.
  • Can be tricky sometimes as audience and news are changing over time... so we need to retest after a couple of months to avoid keeping an under-performing version live. Sometimes a version resonates with news and is better at that time, but not in the long term.
We benchmarked AB Tasty vs. Kameleoon and Optimizely - the clear 3 leaders for me at the time I made the survey. Optimizely was and is still too expensive compared to the other two.

We then tested AB Tasty against Kameleoon with the acquisition team for 8 weeks. I was using Kameleoon for another non-profit organization for 18 months but I decided to opt for AB Tasty. Usability is just a bit under when you start, but AB Tasty is clearly superior in many dimensions when you become a heavy user. AB Tasty is also better if you manage different entities (we have a group of 9 companies using the solution).
Excellent to check optimization hypothesis. Great to know what happens for a user cohort on any web page (website, landing, order form...) thanks to the action tracking. Powerful to change live content without asking IT, optimize pages and customer paths without any technical knowledge.

Difficult to handle a lot of content modifications or reorganizations in one single test. Use a new page instead and create a traffic split. Never used the classification of the pages visited during sessions (not well marketed or too complex?).