1. Home
  2. Usability Testing Tools
  3. UserTesting Reviews
  4. User Review of UserTesting
Real Users, Real Results, with UserTesting
July 25, 2014

Real Users, Real Results, with UserTesting

Jim Hostetler | TrustRadius Reviewer
Score 9 out of 10
Vetted Review
Verified User

Overall Satisfaction with UserTesting

Our E-commerce group uses UserTesting to put live and prototype experiences in front of our users. These experiences come from both our site and from others in similar industries. We use their panel of users and put them on tasks ranging from simple 5-seconds tests, all the way through completing MOST of a transaction with our website. UserTesting helps us to validate concepts, test for usability, and learn more about general users who haven't necessarily been exposed to our business previously and have a fresh perspective.
  • UserTesting's panel of users comes from a wide variety of backgrounds and levels of internet experience. This helps to get a diverse set of users in front of your product before you put it out to the general population.
  • UserTesting has a simple set up process. This allows people from all different disciplines to set up and run user tests. Given this, we're able to look at business challenges from a variety of angles and allow many different types of business units answer their own unique questions.
  • UserTesting has awesome service and makes sure that we get what we ask for. Sometimes there are technical challenges with using an online product (both on our end and theirs), especially one that requires users to record their screen and voice. When these problems arise, UserTesting is great about working to resolve them and providing credit to give it another shot.
  • While the users are diverse, they are, after all, basically professional testers. This gives them a seed of information and a wide range of experience in dealing with different experiences, probably more than the average web user. The challenge here is that they may be "average web users" but are "expert user testers" and provide feedback that is deeper that you would expect from a general user. While this SEEMS good, it can lose focus on the areas that we're looking to target specifically.
  • For the users, UserTesting does require installing software. They mention using your own users via their platform, but it essentially goes through another tool and becomes a moderated test. While this is ok, it's something that we do ourselves anyway and don't really have use of their platform for.
  • There's no way to manage your team's permissions. Everyone gets added as an administrator. That doesn't have major impacts within our organization at current scale, but being able to limit permissions would be nice so we could better track and control who has access to doing user tests.
  • We're able to work more efficiently as a company because we don't need to always devote resources to recruiting users for testing or research. This allows us to devote our resources elsewhere
  • We can get real people's reactions to our current and new features on our website quickly and easily. This allows us to be agile in how we adapt our strategies and tactics to best suit what will be right for our users, leading to better first releases and a better starting point for iteration
  • We release better products because we're able to vet them out ahead of time efficiently. This leads to an increases across all levels of our KPIs because we have a sense of how users respond and can feel more confident putting an iteration out into the wild.
We're currently using UserTesting on a pay-as-you-go format where we purchase credits when we get low and use them fluidly to run tests. This format works great for us since our testing cadence waxes and wanes through the phases of any given project. With this low barrier to us using the service, where we can use it as much or as little as we like, it's a no brainer that we would keep buying credits as they come up.
UserTesting allows you to put your concepts, mock ups, wireframes, prototypes or live code in front of users for unmoderated feedback. This is great if you have either questions so specific that you can ask direct questions to answer them, or so broad that you only need general information. For the middle ground where you want to get some broad information with the ability to drill down in response to what users encounter, UserTesting's unmoderated platform falls short and can show you HOW people use things but often misses the details of why.

Using UserTesting

20 - UserTesting is used by everyone in our E-commerce department from our EVP of Ecommerce all the way to product development support staff. The primary users of the platform do it in relation to our User Experience Design function to perform usability testing. It's also used by Product Managers to determine business value of new concepts and ideas. Other folks use it to do competitive analysis and determine how we stack up against comparable sites.
4 - We have coordinators on deck that do a majority of the actual setup and running of our test. Since UserTesting is a third-party web application, via their website and tools, it requires very little overhead on our end. We have one person in place to do most of the account management, including talking to UserTesting about upgrades to the account and ensuring it stays stocked with credits but the platform has an incredibly low amount of effort devoted to supporting it on our end.
  • Usability testing of prototypes and mockups to ensure users are able to perform the desired functions
  • Competitive analysis to determine how we stack up against competitive and comparable websites
  • Opportunity assessment to put concepts in front of users to determine how valuable it may be for them
  • UserTesting has allowed us to test and refine question types for user testing when we do them in-person with users. It allows us to assess our methodology with less overhead so we can ensure we get the most value out of these in-person tests, which require much more investment.