- Multivariant testing async. We have a limited development staff, so having our marketers be able to control experiments was a big plus.
- Multiple dimensions for testing, allowing us to isolate tests based on device was a benefit.
- Screen recordings, being able to view customer recordings was valuable to see what people are doing on our pages.
- Heatmaps loaded very slow. We have very long form sales copy and VWO could not load the pages with a heatmap view a majority of the time.
- Google and Facebook slap. We were shutdown from running ads to pages that had VWO embedded in them as it was perceived as cloaking. This was the biggest issue for us as we rely on optimizing for display spend.
1) Monitor success of split and A/B tests, and
2) Draft new hypothesis to fuel next rounds of tests.
A/B & Split testing tools are being used by the Development Team to set up and execute tests on our Shopify stores.
- Test set up is a breeze and settings are presented in a thought out sequence.
- Recordings, Heatmaps, and other observation tools are easily segmented by audiences we care about, making observation simple for our UX team.
- Our customer success manager is always available for meetings and proactive in his advocation for the platform.
- Observations currently (9/14/18) don't support outside domains.
- Observation collection sample rate is a little low (for the price) - takes a bit longer to collect meaningful amount of data.
VWO Review: "A cost effective tool with strong potential for increasing incremental revenue and learning about your customers."
- The dashboard and ease of setting up tests with VWO is particularly easy and intuitive.
- VWO are constantly adding new features so we can analyse different parts of the customer journey and gain more insightful data on our customers.
- The VWO conversion all-in-one platform is excellent for gathering insights on how customers use the website and backing up theories. It has helped us prioritise tests and identify issues we hadn't thought of.
- We're on an Enterprise plan and there hasn't been much communication with our account manager. Suggestions for improvement and test ideas from the VWO team would be appreciated and would help to merit its cost and benefits over other competitor tools.
- Adding more personalisation features to the tool would be very beneficial.
- Observations and Hypothesis layouts and structure could be improved so this area of the platform is easier to manage. Checklists, outcomes and graphs would be advantageous.
VWO Review: "Our all-in-one insight and testing platform of choice - could not live, or work, without it."
With VWO's change to being CRO focused, we find the integration between observations and testing to put it ahead of the many competitors.
- Integrated features and benefits between insight based tools and variate testing
- Great UI for what is a pretty complex system
- Really good support team, quick to answer and always find a solution
- Auto report on activity, unless I'm not aware of this feature my reports are created manually
- Grouping tests and analysis into folders, I have a lot of tests and analysis running so scanning and finding the right item can be tricky
- Usage reports, a weekly or monthly report emailed from the platform on our usage stats
• Pages that need to increase their CR
• Newly launched pages and products where we can monitor activity and engagement
• Proving or disproving hypothesis with variate testing
• General problem pages/content on which observations and hypothesis can be based
• Validating design decisions
• Solving arguments or disputes on a particular pattern or feature.
• I'd be concerned if we ever had a situation where we could not use VWO to learn something and better meet the needs of our users and business.
- Goal Measurement. I enjoy the different objects you can target, and how easy they are to set up. Specifically, targeting elements and measuring their usage.
- Survey Data. The survey data is very interesting. I do wish you could make the parameters for showing a survey more complex, but it works very well as is.
- A/B Testing. I much preferred the UI layout of these screens over Google Optimize. It's easy to set markers utilizing goals and visualize the performance of variations.
- A/B Testing Goals. Sometimes when I conduct an A/B test, I want to measure a goal I'm already tracking. However; I can't see these goals from the Test set up screen, so I end up having to recreate them. You might consider a method for adding them similar to the way funnels are built.
- The documentation and integration with GA is under-stressed. GA is a staple for all web companies, so I might suggest working to integrate this better and bring it to the forefront. There are some measurements that are difficult to accomplish with VWO, such as conversion tracking or bounce rate.
- I ran into constant problems distinguishing between https:// and http:// . I actually ended up just using "contains" with the url, which was less than ideal. I might suggest a better way to clarify this -OR- a setting that will accept either.
- Images: Which pictures interact best with the end user over others
- Copy: A/B testing copy as an overlay on top of photos or below. Placement of copy is crucial to mobile first strategies and above the fold landscapes.
- Videos: A//B testing videos and video length. Trying to cap the engagement rate against time being watched. Also A/B testing where videos are best placed. Are they best used in the home page or product page or both.
- Visually, for people who are not familiar with the platform- can be overwhelming. Copy outruns the visual capacity and can cause fear of use.
- Training: a university type excercose is also recommended. Though VWO can be used for many functions- an in-depth course should be offered and/or required. Especially for how A/B tests run.
- All in one platform: The fact that Visual Website Optimizer offers all the features available in one platform facilitates the research work of marketers a lot.
- A/B Test feature: Within the A/B test feature, users have the possibility to analyze the results not just on a summary level, but on a detailed one, too. The best part always comes from the detailed report, which offers a clearer view of the performance of the test.
- Support team: Visual Website Optimizer has a great support team for their users. Fast, knowledgeable, friendly people that you can lay your trust in them that by the end of the day (if not quicker,) they will be there with a solution for you. The support team can assist you in any difficulty you might face at that specific moment with clear directions, both by email or a call.
- Surveys are a great way to get insights from your website visitors. The current version offers you the possibility to do so, but in a very basic way: question -> answer -> the end. My recommendation would be to upgrade this feature by adding branching logic to each answer.
- Customers of Visual Website Optimizer can allocate visitors to their sub-accounts by designating 'testing visitors' or 'research visitors'. Both of them are equally distributed among the different features. But my recommendation would be to have the option to allocate visitors from the 'research visitors', for example, by each individual feature. To know from the start that 100% of research visitors will be distributed depending your needs. e.g. 25% recordings + 25 heatmaps + 30% surveys + 5% forms + 15% Track features.
- It would be great if the Heatmap feature would have the option to save the created reports in a PDF form. This will allow any additional user of the VWO account to overview them on a later point.
- Automatically creates heatmaps of test variations, so no extra "syncing" work is needed to be done with Hotjar or other tools.
- Easy to use interface that let's you quickly create, develop, implement, and start A/B tests.
- Extremely cost-effective compared to Optimizely
- A/B test analysis is not great. Instead of giving you a specific number on lift, it gives you a range and calculates "likeliness to beat control". This is due to the fact they use Bayesian statistics and not Frequentist. Though I don't think they need to move away from B Statistics, having a specific numbered lift, like X% better/worse, would be extremely helpful in communicating the effect a test has to clients.
- No easy way to see how long a test has been running. Must look at the date range on a different view in analysis and then count the days. Very annoying since it could easily be solved by just listing the duration in days somewhere.
- Cannot compare conversion rates over time. Optimizely and most other AB testing platforms create a nice chart that shows how the conversion rates of different variations have changed in relation to each other overtime. VWO does not. Instead it gives you a line chart of the amount of conversions per variation over time. So naturally the line chart just keeps growing because conversions keep coming in. But comparing amount of conversions is very misleading since its conversion rate we care about.
- Setting it up is easy. You can do it either through Google Tag Manager or directly inside the code.
- Support is very good; we always get a fast and in depth response.
- The VWO editor is intuitive and is often easy to use
- Some detailed parts of the editor are difficult to understand (regarding select parent). But after getting the explanations from the support it is solved and now we know how to set up that type of test.
- A/B Testing. The tool is very advanced and the editor allows you to make live changes to the existing website
- Heatmaps. We love to see where people click the most and which CTAs trigger the most conversion
- Funnels and Goals. You get clear details of what steps people take to reach the end goal
- The recordings feature. It's practically useless. You have so many videos to go through to actually identify what is happening that it's a waste of time.
- Very powerful heatmaps and scrollmaps.
- Great screen recording with segmentation capabilities.
- Point and click interfaces for heatmaps. No need to use manual selectors or code.
- Usability. There's so much to take in and what feels like a steep learning curve.
- Performance. The app feels really clunky and slow.
- An inexpensive product allows small teams with limited budgets support the business with A/B testing, which is considered by some as a nice to have.
- Very easy to deploy and use.
- Visitor analysis tools give us more options for qualitative analysis.
- I think some of the automated reporting could be more intuitive.
VWO Review: "Lots of potential, but the confusing UI, data anomalies, and confusing support interactions made us switch to another tool"
- Fairly inexpensive for the suite of conversion optimization tools
- UI is confusing and there is not enough help tips (or they are poorly written) to denote what each setting or metric actually does/is.
- Support is provided by obviously non-native English speakers and we had a hard time getting each support rep to understand what we were asking, which was typically a question about what something within the platform (setting/metric) actually means.
- Statistics engine underlying result data is not explained well and often doesn't line up with other conversion tracking applications (by a large margin).
- Form analysis feature, which is something that we really wanted, doesn't work natively with Marketo forms without significant custom development.
- VWO uses sampling on the overall goal tracking, despite us explicitly negotiating a visitor/session quota price point so that we could track everything on our website. This was one of the most frustrating parts of using VWO--we couldn't trust the accuracy of the data for our overall goal tracking due to this sampling issue because it was wildly different from our actual conversions as tracked in our application.
- Heatmaps automatically on all pages of the website
- A/B testing quick/small changes on a page and see the results in term of form filled or button clicks
- Split URLs for bigger A/B tests where we test some brand new designs
- Targeting specifics users (new visitors, mobile users) for some A/B tests
- The hypothesis part where we need to enter the hypotheses before doing A/B test seems to be useless in our case, as we do that more outside of VWO and don't really see the added value of that to be inside the software
- The sampling of recording/heatmap seems to sometimes be pretty low for automatic recording, and the fact that we need to explicitly ask for one page to be tracked with more sample is sometimes a bit irritating
- AB testing
- Removing Outliers from the data and provide actionable insights
- Provide lots of options as a filter to target a specific audience set
- Heatmap and Visual recording of all the test
- Provide a Mobile app or an improved version of Mobile web.
- Send alerts of any actionable results happening on my AB test
- Algorithm to sample and remove outlier needs a lot of improvement
VWO isn't perfect, but it is a great tool if your organization is looking to get its feet wet with Conversion Rate Optimization. They are developing great tools and features, as well as providing a library of testing ideas and case studies in case you're having trouble thinking of what to test.
Their results reporting isn't iron-clad, but for many efforts it's good enough. I personally am not a big fan of how "confident" they are in their declaring winners - I've seen tests where one day it says a variation is a winner at 100% confidence, then the next day that's down to 81% (think about that for a second).
However, for most new testers, it's a great tool and definitely a lot easier to use than a tool like Google Optimize.
The price point is very reasonable as well, making it accessible to companies of all sizes.
- Their UI for creating tests is brilliant. Does not require coding knowledge to set up basic to intermediate tests.
- VWO's support team is one of the primary reasons to stick with them. They are very fast and always provide an in-depth solution.
- The targeting options are good too, they give you the liberty to test a different set of audiences as well.
- There is a problem with creating responsive tests. The process involves creating two sets of test for mobile and web audiences. Which is time-consuming.
- Their reporting could be better. At times there is just too much data to see on the reporting page and the viewer gets confused.
- The tests are slow to load on slow networks. Which gives away the purpose at the start itself.
We use VWO as the primary A/B testing platform in our optimization efforts. This lets our product, optimization, and engineering teams test out anything from simple changes to large impact concepts with minimal resources.
- Easy to use for most tests. The learning curve is generally low so we have been able to ramp up use quickly.
- Multi-goal reporting. This is highly valuable and easy to setup.
- Limited real time targeting. VWO lacks in behavioral segmentation such setting up segmentation by 'people who add something to their cart'.
- Reporting at scale is a very manual process. The API allows access to data, but that requires more dev work to setup useful reporting at scale.
Enterprise level and highly mature companies that are at a point where they need real-time behavioral segmentation (user has an item in their cart) are out of luck with VWO.
- Easy Set Up
- Easy Modifications
- Fantastic Support
- With cons - keep in mind it may be we are just not aware of functionality. I would say targeting segments could be better
- Better flexibility with 3P design integrations (to test their effectiveness)
- Being able to see test elements in various screen sizes
Easy to use, great support staff. The platform has a slight learning curve but is a welcome and necessary tool for anyone who wants to expand their marketing operations to include A/B testing and site analytic functionality.
- Calculates test duration - VWO instantly calculates the duration your test will need to produce a significant result based on the level of certainty you require.
- Editor - The page editor is very useful for making simple edits to use for A/B tests. It makes the job of editing HTML very easy with no technical expertise needed.
- Reporting - real-time reports for test are kept very simple and are very easy to understand.
- Mandatory hypotheses - VWO recently made it mandatory to add a hypothesis to each test. I dislike the structure I am given to write each hypothesis so I simple fill it empty text.
- Amazing and fast support team
- Easy to use and very flexible solution for split testing
- Well structured reports
- Heatmaps looks really appealing even though we didn't use them
- An option to load and send split testing data via APIs on front end would be appreciated
Primarily, VWO is used by our product and marketing teams to test headlines, funnel changes, and changes in page design.
From a business perspective, we use it to test for conversion optimization. Essentially, we want to track how users convert, across multiple goals within our platform.
If you want to make bigger changes you have to rely on split URL testing, which also works good.
VWO Scorecard Summary
VWO is an A/B testing and conversion optimization platform that enables growing businesses to conduct qualitative and quantitative visitor research, build an experimentation roadmap and run continuous experiments on their digital properties.
With its 5 capabilities Plan, Track, Test, Analyze, and Target, it brings the entire CRO (conversion rate optimization) process at one place. VWO helps online businesses follow the process- and data-driven conversion optimization to boost their conversion rates.
With features like heatmaps, surveys, visitor recordings, and others, VWO enables businesses to develop an in-depth understanding of on-site visitor behavior. VWO facilitates conducting research-driven A/B testing to optimize web experience across desktop, mobile, and other devices, to deliver a unique experience based on visitor preferences.
According to the vendor, VWO has helped more than 4,500 brands across the globe to run over 600,000 experiments to date. VWO’s global customer base includes brands like Vodafone, Samsung, Toyota, and eBay.
|TESTING||Get a Demo||The classic VWO A/B testing solution|
|CONVERSION OPTIMIZATION||Get a Demo||The all-in-one platform for all your optimization needs|
|ENTERPRISE||Get a Demo||Customized solution with advanced AB testing and conversion optimization capabilities|
For any additional information about pricing and plans, visit https://vwo.com/pricing/
VWO Support Options
|Free Version||Paid Version|
|Video Tutorials / Webinar|
VWO Technical Details