A competent tool to get you AB testing
December 14, 2015
A competent tool to get you AB testing
Score 6 out of 10
Vetted Review
Verified User
Software Version
Small Business
Overall Satisfaction with Visual Website Optimizer
We currently use Visual Website Optimizer (VWO) as part of our optimization suite of tools to run AB/MVT tests. With a good pricing model and when paired with a tag manager, it allows us to run hundreds of tests over 45 different websites with relative ease. With a significant required for competitive tools, this proved a compelling way to get management excited about the potential of CRO while also fitting within our budget constraints.
- VWO makes it easy to get up and running with AB/MVT testing - without loads of code knowledge or a large budget.
- VWO makes it easy to visualise complex statistics using graphs and visualisations.
- VWO has an evolving toolset that is becoming more and more mature every week.
- VWO has a good set of segmentation tools to identify how tests perform with different customers.
- VWO reliability can leave me scratching my head. Code verification sometimes fails during a running test and previews will sometimes inform me that "test is not running on this page", even when it's set to run sitewide.
- Code generated by the visual editor can be slightly buggy. CTA buttons were duplicated or missing, but this wasn't visible in the preview - only on site when the test was live. This is okay if you have good QA testing and the javascript skills to tweak the generated jQuery code.
- Conversion tracking can sometimes be flaky. Tracking using css selectors isn't as robust as I would expect, especially when tracking clicks on modified objects (like buttons). This can be frustrating when it requires me to reset tests after a couple of days due to lost tracking. It would be nice to have a build-in verification tool so I can quickly see if these won't work correctly.
- It would be nice to see full access to the segmentation options on all plans if pricing model is primarily charged by number of visitors.
- Positive impact on ROI from PPC campaigns through refined landing pages.
- Improved contribution to websites from across the business by being able to quickly test ideas from everyone, at any level.
- Helped move the company towards a 'test everything' culture, where data-driven digital products are the norm and every bit of money spent on product development can be justified.
- Increase in sales leads generated through form optimisation.
VWO has a similar toolkit of features to Optimizely, although in almost every respect Optimizely is a more mature offering. VWO is catching up, which is visible through the addition of SmartStats (to compete with Optimizely's Stats Engine) and acquisitions like Navilytics. Optimizely also has a more mature personalisation offering (although VWO looks to be heading in this direction). VWO has compelling pricing, especially for smaller and medium sites, that make it stand up well to Optimizely's more refined product.
Visual Website Optimizer Implementation
- Implemented in-house
Change management was minimal - The biggest challenge implementing a testing tool such as VWO is letting everyone know that everyone may not be seeing the exact same version of a site, and that that isn't a big problem. The biggest challenge to communicate was if people were seeing different things to the person next to them that it wasn't necessarily a bug, or that customer service might be talking someone through a screen that is different to what they're seeing.
- Tag manager and simple code base made this straightforward.
- Verification of tests inside the UI can be the most time consuming thing if code verification isn't working.