A competent tool to get you AB testing
December 14, 2015
A competent tool to get you AB testing
Score 6 out of 10
Overall Satisfaction with Visual Website Optimizer
We currently use Visual Website Optimizer (VWO) as part of our optimization suite of tools to run AB/MVT tests. With a good pricing model and when paired with a tag manager, it allows us to run hundreds of tests over 45 different websites with relative ease. With a significant required for competitive tools, this proved a compelling way to get management excited about the potential of CRO while also fitting within our budget constraints.
- VWO makes it easy to get up and running with AB/MVT testing - without loads of code knowledge or a large budget.
- VWO makes it easy to visualise complex statistics using graphs and visualisations.
- VWO has an evolving toolset that is becoming more and more mature every week.
- VWO has a good set of segmentation tools to identify how tests perform with different customers.
- VWO reliability can leave me scratching my head. Code verification sometimes fails during a running test and previews will sometimes inform me that "test is not running on this page", even when it's set to run sitewide.
- Conversion tracking can sometimes be flaky. Tracking using css selectors isn't as robust as I would expect, especially when tracking clicks on modified objects (like buttons). This can be frustrating when it requires me to reset tests after a couple of days due to lost tracking. It would be nice to have a build-in verification tool so I can quickly see if these won't work correctly.
- It would be nice to see full access to the segmentation options on all plans if pricing model is primarily charged by number of visitors.
- Positive impact on ROI from PPC campaigns through refined landing pages.
- Improved contribution to websites from across the business by being able to quickly test ideas from everyone, at any level.
- Helped move the company towards a 'test everything' culture, where data-driven digital products are the norm and every bit of money spent on product development can be justified.
- Increase in sales leads generated through form optimisation.
VWO has a similar toolkit of features to Optimizely, although in almost every respect Optimizely is a more mature offering. VWO is catching up, which is visible through the addition of SmartStats (to compete with Optimizely's Stats Engine) and acquisitions like Navilytics. Optimizely also has a more mature personalisation offering (although VWO looks to be heading in this direction). VWO has compelling pricing, especially for smaller and medium sites, that make it stand up well to Optimizely's more refined product.
If you're looking to get into user testing on your website, and have a little patience to get started then Visual Website Optimizer could be the tool for you. With a pricing model that can cater very well with even a moderate number of visitors, this is an excellent choice for starters. It can consume some time in setup and is probably best used by people with some development skill, but is otherwise a well equipped tool that gives users an excellent way to increase conversions and justify further spend on CRO tools.
Visual Website Optimizer Implementation
Overall, the implementation of VWO is straightforward. If you've got a straightforward way of deploying code to all of your test pages, either a good CMS or a TMS, then implementation should be a breeze. There is no tweaking to be done to the code itself, and once deployed it has the flexibility to cope with different VWO modules (tracking, conversion analysis, session analysis) without modification.
Change management was minimal - The biggest challenge implementing a testing tool such as VWO is letting everyone know that everyone may not be seeing the exact same version of a site, and that that isn't a big problem. The biggest challenge to communicate was if people were seeing different things to the person next to them that it wasn't necessarily a bug, or that customer service might be talking someone through a screen that is different to what they're seeing.