AWS Device Farm vs. HeadSpin vs. QAComplete

AWS Device Farm is perfect for small teams who don't have enough resources to conduct large scale testing scenarios or even large teams who would like an easier time with testing on various devices. [AWS] Device Farm makes it very easy to interact with a multitude of devices right from the browser.

Incentivized

We have been using HeadSpin for our web and mobile application testing. It is an easy-to-use solution that provides detailed and in-depth analytics from the testing results. We also get a wide range of live networks and devices to perform the testing from HeadSpin. It has an inbuilt artificial intelligence engine that helps us to get real-time scenarios simulated, which is very helpful for finding performance issues and improving the overall quality of the applications.

I think QAComplete is a great way to organize the literal hundreds of development bugs that are discovered throughout an agile-based project. Teams aren't always able to hotfix items on the fly and this tool provides a great repository and facility for storing and prioritizing those technical debt items that may get overlooked from sprint to sprint. It also let's everyone on the team get a clear understanding for project progress.

Incentivized

• 100% compatibility with the installed user base. No simulation, but real results.
• Integration with Jenkins and other CI/CD tools.
• A considerable amount of device types available. Again, real devices in the cloud!

Incentivized

• Keep track of each browser we have tested and when we lastly updated them.
• Remove any deprecated code or plugins that may cause issues with other browsers.
• Add browser information to our site’s meta tags so that the site displays correctly in different browsers.
• Test our website on all browsers to ensure compatibility.

• Allows us to create individual tests then group them into comprehensive test plans.
• Allows me to specify custom fields that match our product features to categorize both tests and defects.
• Allows me to create a series of requirements then match tests against them for validation.

Incentivized

• To be Frank, I have not found any room where it needs to improve. As whatever we used, It is useful for us

Incentivized

• UI needs to be improved
• Graphs on the dashboard are a bit laggy and need detailing for a better understanding of the user

Incentivized

• The user interface can be somewhat perplexing and difficult to navigate at times.
• Naming conventions aren't always succinct and additional training is needed by those who have more experience using the program.
• A modern refresh of the UI would help keep the product from looking dated.

Incentivized

I gave QAComplete this rating because though I do feel that it is a powerful tool, it's definitely not designed for the everyday user. Obviously the target demographic is QA specialists who should be team members that are very technical and methodical in their approaches. However, for the rest of the team, it can often time prove difficult to navigate. Also, the UI is in need of a serious refresh.

Incentivized

We had the enterprise support with AWS, so overall support experience was good with great engineers on the back providing answers. As you may know, overall AWS support is different and this is not different. Responses through the regular web support channel came easily, fast and accurate. We had questions/issues which were solved fast. Documentation is good as well, especially around the test automation pieces.

Incentivized

I have not had to work with their customer service directly yet. Our client has been utilizing this tool for years, so I would assume that the support they have received would be good enough to keep them wanting to continue to use the product.

Incentivized

We haven't used anything like AWS Device Farm before. I am familiar with Amazon Web Services and when we had our MVP ready to test, we turned to AWS for a solution. AWS Device Farm was exactly what we were looking for as we have a really small team and limit resources.

Incentivized

Though LambdaTest was cheaper, it did not offer the deal devices and we wanted to be 100% sure that our app offers a perfect experience to all users. Simulators and emulators are not the technology we wanted to rely on.

A comprehensive solution

Incentivized

• It impact more and more. Not in terms of ROI but in time as well

Incentivized

• Simplified the approach to test the mobile application on multiple devices from different vendors. This helped us to ensure the stability of the application in different environments.
• In-depth analysis to understand the loopholes in the application design function. This helped us to improve the application design and overall user experience.

Incentivized

• Helped us in increase the quality of our firmware releases.
• Helps us continuously improve our testing

Incentivized