A good enterprise web analysis package
February 20, 2019
A good enterprise web analysis package
Score 7 out of 10
Overall Satisfaction with VWO
It's being used to analyze our marketing and e-commerce site. We use A/B testing, recordings, heatmaps and surveys to gather information about how users are able to use our sites and what they are understanding. A/B testing has been valuable in getting buy-in from decision makers as we are able to directly prove the effectiveness of the changes we are proposing.
- Visual editor for A/B testing is really solid, especially if your site is fairly static. It gets a bit tougher when you have very dynamic content but it's still much easier than having our dev team make two versions of a page and try and split traffic.
- Recordings and Heatmaps are invaluable assets for figuring out how well certain pages are working. While they don't replace standard user testing they can supplement it very well and reinforce your position to make a design change.
- Support has been fairly quick to resolve rather complex issues, even with a bit of a language barrier a lot of the time. Discussing technical challenges over chat is not easy but I can usually get these difficult challenges ironed out in a day or so.
- I don't really use the "plan" features much. They seemed nice at first but they are too slow and cumbersome to use in practicality. I don't like being forced to use it to do testing.
- Goal tracking seems limited. I'd love to be able to use advanced e-commerce goals (like how many people bought a specific product) for my test but have to use overall revenue and 3rd party tracking like GA to fill in the gaps.
- Dynamic content is really hard to edit. Anything built on a modern JS framework is hard because manipulating multiple states on the same page starts to get really complex. There's a lot of workarounds, and I haven't found a better solution, but it's not as easy as point and click if you have something more complex than a static site.
- I cannot think of anything negative other than the price of enterprise options. While we certainly get value out of it, huge returns from A/B testing that is, it's not cheap by any stretch of the imagination.
Unless it's your sole job, rapidly testing hundreds of changes is a bit silly. Changes, even just tests, take time to get approval. We'd like to rapidly test but organizational processes make that somewhat of a pipe dream. It has given us good statistical validity of results which we have used to convince those higher up of some fairly radical ideas.
We haven't really used segmentation too much. Some pages we test barely get enough traffic to them to have a test conclude quickly so filtering it to smaller portions doesn't seem to work. We'd love to have the pool of users to do more specific segmentation and gain those insights, but we just don't have the traffic to do so right now.
It's okay. Editing a headline here and there is fine. Editing dynamic content breaks down. Oftentimes, I need to resort to editing the code itself because editing it with the visual editor breaks that element or something else. If you have a static site it's probably easy to use. Add in dynamic content and you have a recipe for disaster if you aren't careful to review your variations before you start the test.
VWO is a good compliment to GA and Hotjar but it's expensive. Hotjar has a lot of really good analysis features for a very, very reasonable price. GA is free but comes with a high operational cost and learning curve. GA and its suite of tools is improving but I don't have enough experience with their other tools to make a judgment call on that.
I wouldn't recommend VWO to non-enterprise users. Hotjar does a lot of the tracking equally as well at a fraction of the cost and there are split testing options out there that are much, much cheaper. It's convenient if you have the budget to have it all in one place, but if you don't, then doing it on two different platforms is still workable. If I'm price sensitive I'd probably explore some other options in combination with Hotjar.