- The set up, configuration and previewing of tests is quick and easy.
- Checking results throughout the test cycle is clear and easy to report and understand how much more time is needed for the test to run.
- Calling a test and implementing the changes is done with confidence.
- There are some minor adjustments where changing the title of the control or variation or even the test can present an alert that a change was made which can't be cleared. I understand alerting when something may affect the data results but label changes in the VWO portal should not affect the metrics and therefore shouldn't add an alert message to the test.
- We're just getting started with their analysis set of tools to better understand customer interactions through click and heat map data but there could be improvement. For example, I've had challenges ensuring the tool showed the total available data for a particular page which required some regular expressions to ensure the total was displayed. It's a bit awkward to get working properly.
- Easy to implement
- Easy to monitor tests
- Much simpler to learn than other testing tools
- Once it's set up, one of the best features of VWO is that the WYSIWYG editor allows us to make comprehensive revisions to the layout, images and content without needing to get designers or IT involved. Of course, we use these resources on the more important tests, but this flexibility makes the testing process much smoother.
- I think that the heatmaps/clickmaps are a very cool feature. VWO gives us the option of collecting clickmap data with every test that that we run. This data is fantastic to have as it shows us what elements on the page are getting the most attention from visitors. We can then adjust these elements in future tests to draw more or less attention, as needed.
- Not only does VWO support A/B split tests, multivariate tests and split URL tests, but it also lets us segment the traffic that sees a test by different criteria such as geo location and traffic source. Also, when running tests concurrently, we can prevent visitors that see one test from seeing another. This is a great feature because it allows us to speed up our testing process without skewing results with "cross-over" visits.
- The revenue tracking feature is not one that we currently use since our sites are built for lead generation. But I've used this feature with e-commerce sites before and it is very helpful to see the impact on revenue in addition to the impact on conversion rates, especially when your products or services have different price points.
- VWO support is outstanding. They have always responded to my questions and requests quickly, and have assisting in helping us solve complicated technical issues.
- I really don't have anything bad to say about VWO. If I nit pick, I would say that I'd like to see graphs displayed in the dashboard so that we can get a snapshot view of our tests without having to click into the test details.
- I also wish that it would save a snapshot of the test variations instead of pulling them live. After a test has run its course, we sometimes update the page and, in doing so, we lose an example of what the original variation looked like. I capture screenshots before updating the pages, but it would be great if VWO did this for me.
- Great UI and features available for beginners or advanced developers.
- Excellent support with great response time.
- Created by a company that is very forward thinking.
- Complete integration with Universal Analytics.
- Easy to use and set up goals
- Only one tracking code needs to be installed
- Will alert you when a metric has reached statistical significance
- The editor seems finicky at times, some elements need to be coded in the variation to work properly
- I rarely get a correct preview of my variation, I have to view it in the editor to see it
- After a test is completed and stopped, some of the data may be lost, I have noticed days missing from several reports, so always document results outside of VWO
VWO Review: "Flexible, Easy, and Powerful Optimization Tool - for Testing Addicts without a Tech Team"
- Rules based, allowing advanced segmentation
- Very easy to create new tests, or modify testing content
- Very flexible in ways to conduct and measure your test
- Push notifications to mobile devices
- Tie-in business results with back office (e.g. uplift on revenue, conversion attribution for long sales channels, etc.)
- It's a very simple system to use with some very advanced/flexible features.
- The support team is very knowledgeable and on-hand to provide assistance when required.
- The pricing plans are very generous as well as flexible if required.
- Some of the advanced functions that really help to make this a great product are sometimes hidden and you may not know of their existence until after speaking with support.
- It's great for getting tests set up quickly. As soon as you've added the one piece of JS across your site, you can start testing simple ideas in minutes.
- It displays actionable results attractively and in a way that's accessible to non-statisticians.
- It gives you good control over segmentation. If you need to, you can use custom JS, cookies and other technical trickery to get very fine-grained control over segmentation. At the moment we're running several tests on a landing page, each targeted at a different segment.
- You can track multiple goals, including page views, clicks on page elements, form submissions and even custom JS triggers.
- It's not always easy to set up a test. We had significant teething trouble due to certain idiosyncrasies of our website. One split test affected pages on the site that it wasn't supposed to, another caused huge lags in display times; more recently we had a test fail to capture visitors correctly. Over time, we've worked out how to address each problem, but it's certainly not as simple as it appears to be on first glance. I used to use VWO across a wide range of clients and almost all suffered from teething trouble of one kind or another.
- Goal tracking is not always reliable. Some types of goals seem to be captured more consistently than others. This isn't a deal-breaker: you just have to make sure that your main testing goals are being tracked correctly.
- We have several different views with the same URL, depending on viewer state. I'd like to be able to turn on tracking across all the views without all the views serving a test variation. We've managed to work around this, but I wish it had been simpler.
- We've also had to develop our own workaround for capturing the versions of pages our visitors have seen in our own database. It's crucial for us to be able to track our visitors across different variations of offline media as well as the online media and store the details for later cohort analysis. I'd wish for VWO to provide an API of some sort for this purpose.
You're ready for VWO if:
- you understand the fundamental principles of hypothesis-driven testing and you're looking to understand, empathise with and learn more about your visitors and customers
- you have a clear idea of the business goals you're going to address through split testing and the metrics you're going to optimise for
- you've developed a testing plan across a series of tests and you're prepared for the reality that some tests won't get to significance – or might surprise you in other disappointing ways...
- you're new to split testing in practice – or you've tried one of the free testing platforms and found it too confusing, buggy or difficult
- you're keen to split test and you want to be able to set up relatively simple tests quickly and with a minimum of technical involvement
- but you're prepared for some headaches and tinkering to get it working just right
- your budget isn't high enough to afford a more fully-featured or 'done-for-you' split testing solution.
- Very easy to setup new tests
- Pretty easy interface for making WYSIWYG changes to the page
- Very good "bottom line" reporting on the outcomes
- It can be difficult to get exactly the combination of elements you want in the multivariate test
- Getting detailed data (not just bottom line answers) is difficult to impossible, with the best case coming through its Google Analytics integration
- The preview windows do not work consistently
- Revenue tracking
- Basic Heatmaps
- Mobile/Tablet rendering
- There are some glitches with Goal and Revenue tracking, e.g. if the goal is on the same page and user refreshes, it sometimes can count extra for one or another, but in essence it should be impossible to have 500 sales and 536 revenue conversions.
- Also could do something better with de-duplication, perhaps a setting in the test setup area that only 1 conversion per IP/session in 24h can be counted.
- Ah, and the awful plan to upgrade to new interface, it looks horrible. The plain and simple that is now works the best.
- GEO filtering, e.g. if I have 10000 visitors tested, I want to be able to segment them, say look at just USA or UK portion in that test, instead of say setting up individual pages.
- Also being able to set conversion goals and test that would need to account say if mobile user went from mobile site to desktop version, it still should count under mobile in certain circumstances.
- revenue tracking
- good heatmaps
- Sets up tests in a couple of minutes (literally).
- Powerful reporting.
- Extremely easy to use.
- Move tracking code from Amazon's CloudFront CDN to Akamai. AWS has slow DNS and a much smaller CDN than Akamai. Main competitor uses Akamai.
- Allow creating groups of goals. Now I have to clone experiments and add or delete necessary goals manually.
- Allow goals reordering as it makes reports more useful.
VWO Review: "Cost Effective, Feature Rich, Website a/b/n/MVT Testing Leads to Increased Performance and User Experiences"
- Easy to use
- Great support, fast and accurate
- Easy to implement
- New features - though they tend to release updates frequently
- User workflow - to provide approval process with limited permissions for various users
- Visual Website Optimizer has a very user friendly dashboard. I have experience with other testing tools and I can confidently say it is very competitive in this particular area.
- I experienced terrific customer support. Their support team was always helpful when I had questions about their methodology or encountered tracking issues.
- The WYSIWYG editor could use some advancements. The layout can break when stacking multiple page changes. It is hard to trouble shoot what is causing the issue.
- Adding a jQuery editor that displays the code generated when you make changes via the WYSIWYG editor. From a power user perspective, being able to see the generated code would allow for more precious changes and streamlined code.
- VWO makes it easy to get up and running with AB/MVT testing - without loads of code knowledge or a large budget.
- VWO makes it easy to visualise complex statistics using graphs and visualisations.
- VWO has an evolving toolset that is becoming more and more mature every week.
- VWO has a good set of segmentation tools to identify how tests perform with different customers.
- VWO reliability can leave me scratching my head. Code verification sometimes fails during a running test and previews will sometimes inform me that "test is not running on this page", even when it's set to run sitewide.
- Conversion tracking can sometimes be flaky. Tracking using css selectors isn't as robust as I would expect, especially when tracking clicks on modified objects (like buttons). This can be frustrating when it requires me to reset tests after a couple of days due to lost tracking. It would be nice to have a build-in verification tool so I can quickly see if these won't work correctly.
- It would be nice to see full access to the segmentation options on all plans if pricing model is primarily charged by number of visitors.
- A/B Split Testing
- Speed of Implementation
- A lot of options to use
- The test setup interface is VERY user friendly which makes setting up tests and teaching others how to set up tests quick and easy.
- The test results are easy to understand and the cumulative conversion chart helps with determining when data has normalized.
- The test duration calculator has been very helpful in test planning and expectation setting. Sometimes getting a significant result takes longer than we would hope and the test duration calculator helps validate that.
- POST-SEGMENTATION would be so helpful. If there was a way to segment the data after the fact in order to look at segments with significant change rather than flat results, our organization would get many more learnings out of out tests.
- As a frequent A/B tester I know that running a test for 1 day usually isn't going to have confidence and that every day of the week will probably show a different result. So having a test display 99% confidence after 1 day is a bit annoying, since it's too early to tell.
Member pages (preview doesn't work if it's a "logged in" page)
Rearranging elements of the page with sliders
Elements with dynamically populated product.
Elements with hover states
- Very easy to implement
- Very simple and intuitive to use
- Very inexpensive compared to other tools.
- VWO fails in allowing complex functionality, such as customer segmentation. It does allow you to target specific segments based on cookies, but these filters must be added to individual tests and can not be stored as universal variables. You must create cookies that contain individual customer attributes, or use the generic, predefined attributes such as geolocation, browser type, time of day, day of week, etc....
- Reporting is not as thorough as it seems to be in some other tools.
- Offers split url tests
- Targeting possibilities are huge as you can almost add everything, it just has to be on your page
- Shows you if a test is significant, so you do not have to make the extra math calculations yourself
- Offers extra tools like Test Duration Calculator, Ideafox , Heatmaps or the tool 'What To Test' where you can get feedback on your website (you need to pay extra for that)
- Has a very good and very fast support, questions are always answered within a day!
- The tool is all in all easy to use
- Sometimes the tool for rearranging elements on the page to test does not work with our Website as it is supposed to
- Add something where we do not have to add every goal by hand but have them within a group we only choose
- Being able to measure things like add to cart rate without having to integrate an external tracking tool
- Visual Website Optimizer allows you to create several types of tests easily.
- The menus are very easy to navigate and it walks you though the process of creating tests.
- It is very easy to see your currently running tests and see how each one of your tests are performing.
- During test setup some of the goals don't always work properly.
- The difference between an A/B test and a Split URL test are not well defined.
- The online documentation is hit or miss. Sometimes it is very helpful and other times it is extremely difficult to find out what you are trying to do.
- Its very easy to use.
- It was quick to get up and running.
- It gave a winner very quickly!
- There's a new version enroute, but I'm yet to receive access, even though I've been requesting it online and spoken to a VWO representative.
- It looks likes the new version will address this, but currently it feels like there's alot of detail/advanced features that could be hidden.
- The language on the old version of the site is sometimes not that clear.
VWO Scorecard Summary
VWO is an A/B testing and conversion optimization platform that enables growing businesses to conduct qualitative and quantitative visitor research, build an experimentation roadmap and run continuous experiments on their digital properties.
With its 5 capabilities Plan, Track, Test, Analyze, and Target, it brings the entire CRO (conversion rate optimization) process at one place. VWO helps online businesses follow the process- and data-driven conversion optimization to boost their conversion rates.
With features like heatmaps, surveys, visitor recordings, and others, VWO enables businesses to develop an in-depth understanding of on-site visitor behavior. VWO facilitates conducting research-driven A/B testing to optimize web experience across desktop, mobile, and other devices, to deliver a unique experience based on visitor preferences.
According to the vendor, VWO has helped more than 4,500 brands across the globe to run over 600,000 experiments to date. VWO’s global customer base includes brands like Vodafone, Samsung, Toyota, and eBay.
|TESTING||Get a Demo||The classic VWO A/B testing solution|
|CONVERSION OPTIMIZATION||Get a Demo||The all-in-one platform for all your optimization needs|
|ENTERPRISE||Get a Demo||Customized solution with advanced AB testing and conversion optimization capabilities|
For any additional information about pricing and plans, visit https://vwo.com/pricing/
VWO Support Options
|Free Version||Paid Version|
|Video Tutorials / Webinar|
VWO Technical Details