Item: UserTesting
Rating: 1
Author: Alexander Kerschhofer

Designers use UT in order to validate their designs.

The UI doesn't have a full screen mode. That means that the user can see all sorts of other elements that don't belong to the prototype. In one case the user clicked the browser's back button which tainted the test.
The panel with the instructions partially obstructs elements of the prototype. That might taint the test.
In one case, during the test, a feedback form for UserTesting popped up, also tainting the test.
The UI also has dedicated back/next buttons. One user used them which also might taint the tests.

Negative impact since inexperienced designers thought their designs are good but in reality the design was flawed. This could cost us millions of dollars in lost revenue.

As mentioned, it's bad. Really bad - which is ironic for a company that aims to help you developing great user experiences.

The UI has significant flaws which prevents it from conducting accurate tests.

Sterling Jackson
(Vendor Representative) commented 3 years ago
Alex, sorry to hear your experience. Interacting with web apps in a typical browser view is how our customers almost always want participants to test their experiences to mimic how they'll use them in the wild. However, your concern about task instructions potentially obscuring the app is valid. It's why the instructions modal is collapsible for participants. There's no avoiding participants' need to be able to bring the instructions back if they need them -- imagine the study disruption if the participant forgot the instructions and couldn't retrieve them. But we're always looking at how we can make this even less intrusive. I believe the pop-up you mentioned is one that reminds the user to speak up if we detect their audio may be low. If so, it's one of those tradeoffs we've decided to make to ensure that the session audio is usable for customers. We'd love to confirm the use-case you mentioned to understand the need better. Feel free to contact us at support@usertesting.com

Poor test UX leads to tainted test and misleading results

Overall Satisfaction with UserTesting