What users are saying about
<a href='https://www.trustradius.com/static/about-trustradius-scoring#question3' target='_blank' rel='nofollow noopener noreferrer'>Customer Verified: Read more.</a>
Top Rated
129 Ratings
8 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener noreferrer'>trScore algorithm: Learn more.</a>
Score 10 out of 100

UserTesting

<a href='https://www.trustradius.com/static/about-trustradius-scoring#question3' target='_blank' rel='nofollow noopener noreferrer'>Customer Verified: Read more.</a>
Top Rated
129 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener noreferrer'>trScore algorithm: Learn more.</a>
Score 8.9 out of 100

Likelihood to Recommend

AWS Device Farm

This is the best solution I have found to trail run software before it is deployed across a large number of users and a large variety of device types. We have Windows, Mac OS, Android, and iOS devices. Android alone has a ton of OSs. This helps save time and test functionality across the board.
Anonymous | TrustRadius Reviewer

UserTesting

I find UserTesting is great if you are struggling to prove out or establish user research at a company. It is more cost-effective than hiring a full-time UX Researcher and helps speed up the process of testing with users. Before UserTesting we were maybe doing 2-3 tests a quarter and now we do 3-4 a month. It's a flat rate for tests so I do not have to worry about expensing and then getting reimbursed every time I need to run a test. I can run multiple tests a week or even in a day and be charged the same amount
Alex Gardner | TrustRadius Reviewer

Pros

AWS Device Farm

  • 100% compatibility with the installed user base. No simulation, but real results.
  • Integration with Jenkins and other CI/CD tools.
  • A considerable amount of device types available. Again, real devices in the cloud!
Anonymous | TrustRadius Reviewer

UserTesting

  • Simple product. Although less features than some key competitors, [UserTesting] centers on its core features and does it well.
  • Relationship management is great. One feels supported. Prior signing with [UserTesting], sales team truly tried to understand our need and showcase their product based on that. Once we were users, tremendous effort were displayed to onboard us and offer support
  • Whether it's using [UserTesting's] panel or our own, there's a lot of flexibility there to find the right audience for our test.
Anonymous | TrustRadius Reviewer

Cons

AWS Device Farm

  • Cost and price is always a concern
  • Can be a bit clunky for the staff using the service.
  • Sometimes, misses issues with iOS devices.
Anonymous | TrustRadius Reviewer

UserTesting

  • I wish the interface was not implemented as "pages" where you have to save your work or risk losing it if your connection drops. I would prefer to have my work autosave or use an application I download to my computer.
  • It can sometimes be hard to tell where people are tapping when watching the recording of people using their phone
  • Video clips are very easy to make, but there's no way to rename clips after you save them
Craig Scull | TrustRadius Reviewer

Likelihood to Renew

AWS Device Farm

No score
No answers yet
No answers on this topic

UserTesting

UserTesting 10.0
Based on 3 answers
UserTesting has drastically changed the way we research. We are now conducting research at a scale and pace that would be impossible to replicate using our traditional recruitment methods. Additionally, the ease of sharing information and findings out of UserTesting has made our research more dynamic and engaging to stakeholders. We can't imagine a situation in which we step backwards now that we have demonstrated the expansion capability UserTesting provides to an in-house UX research practice.
Anonymous | TrustRadius Reviewer

Usability

AWS Device Farm

AWS Device Farm 10.0
Based on 1 answer
The users have no issue. The IT staff have learned to work with the software with the help of AWS support. I find it very effective. Could it be easier? Of course. That doesn't mean this isn't a great solution. I would recommend it to anyone based on usability. Overall, it is easy enough to use.
Anonymous | TrustRadius Reviewer

UserTesting

UserTesting 8.9
Based on 55 answers
Their platform has good UI elements and information is easy to find, especially when it comes to finding templates and setting up tests accordingly. However, there needs to be a bit more [of] a safety net or control when launching tests. This is not given to the user. I would love to cancel or put a test on hold when it's launched.
Zeeshan Ali | TrustRadius Reviewer

Support Rating

AWS Device Farm

AWS Device Farm 10.0
Based on 3 answers
We had the enterprise support with AWS, so overall support experience was good with great engineers on the back providing answers. As you may know, overall AWS support is different and this is not different. Responses through the regular web support channel came easily, fast and accurate. We had questions/issues which were solved fast. Documentation is good as well, especially around the test automation pieces.
Anonymous | TrustRadius Reviewer

UserTesting

UserTesting 8.6
Based on 64 answers
Support is quick, professional, and always resolves any issue we have. The support team often responds to emails shortly after a request is sent, and we have a dedicated representative that is eager to share new features, help us get the most out of the platform, and even provide resources to help new team members learn how to use it.
Anonymous | TrustRadius Reviewer

Implementation Rating

AWS Device Farm

No score
No answers yet
No answers on this topic

UserTesting

UserTesting 10.0
Based on 2 answers
From a technical perspective, the implementation was extremely smooth. Most of the change management / implementation hurdles were clearing use of the tool through our various security, legal, and information privacy teams. Once these concerns were addressed (UserTesting.com was very helpful in providing all the needed documentation), the implementation process was very simple and we were able to get going right away.
Anonymous | TrustRadius Reviewer

Alternatives Considered

AWS Device Farm

This is the only solution I have used from device farming. From that perspective, I would purchase it again. I think this is a great solution for a large, diverse organization looking to mass deploy and manage software and devices. We selected this software because we have had great luck with other AWS services.
Anonymous | TrustRadius Reviewer

UserTesting

UserTesting is by far superior to other competitors. The amount of features and non-limitations is much better than what the competitors have. It's also much more pleasing to look at. Perhaps it's because it is well organized in comparison. It still have some annoying quirks, but much better than what I have seen before.
Anonymous | TrustRadius Reviewer

Return on Investment

AWS Device Farm

  • Reduced the amount we spent on physical devices by > 90%.
  • Enabled real CI integration even when running E2E test scripts using mobile phones.
  • Helped us to improve overall App stability and rating - fewer crashes (80% to 99% of sessions crashless) and 75% rating increases on apps.
Anonymous | TrustRadius Reviewer

UserTesting

  • UserTesting has allowed us to verify hypotheses qualitatively before we make UX changes.
  • UserTesting has also allowed us to verify UX solutions before they go live which allows us to go faster to market.
  • Often following positive UserTesting verbatims has led to increased conversion and free trial rates.
Anonymous | TrustRadius Reviewer

Pricing Details

AWS Device Farm

General

Free Trial
Free/Freemium Version
Premium Consulting/Integration Services
Entry-level set up fee?
No

AWS Device Farm Editions & Modules

Edition
Pay-As-You-Go$0.171
Unlimited Testing$250.002
Private Devices$200.002
Pay as You Go - Desktop Browsers$0.0053
  1. per device minute
  2. per month
  3. per instance minute
Additional Pricing Details

UserTesting

General

Free Trial
Yes
Free/Freemium Version
Premium Consulting/Integration Services
Entry-level set up fee?
No

UserTesting Editions & Modules

Additional Pricing Details

Add comparison