AWS Device Farm vs. Akamai CloudTest

AWS Device Farm is perfect for small teams who don't have enough resources to conduct large scale testing scenarios or even large teams who would like an easier time with testing on various devices. [AWS] Device Farm makes it very easy to interact with a multitude of devices right from the browser.

Incentivized

We were designing an Airlines reservation system, where we wanted to check the maximum number if reservation our system can handle, this tool helped us to get the test results but the issue was from set up point of view, the initial set up took too much time. But we were able to achieve our Reservation system process.

Incentivized

• Integration with CI/CD Pipeline
• Diversity in Real Devices over the Cloud
• Pay as you go option
• Reproduce Issues and easily collect all kinds of logs

Incentivized

• Integration Call's like REST
• Integration Call's like SOAP
• Performance Testing
• Load Testing on Load Balancer

Incentivized

• Interface (CLI) not very easy. Competition has better options.
• Integration with IDE for test case coding.
• Reporting for the test scripts executed.

Incentivized

• Report generation is difficult
• Some graphs are not displaying accurate results
• More information are missing with respect to load testing

Incentivized

We had the enterprise support with AWS, so overall support experience was good with great engineers on the back providing answers. As you may know, overall AWS support is different and this is not different. Responses through the regular web support channel came easily, fast and accurate. We had questions/issues which were solved fast. Documentation is good as well, especially around the test automation pieces.

Incentivized

I got more features and handy within AWS Device farm than AWS Cloud9

Incentivized

The bundle of features that other tools provide is much higher when compared to the one provided by Akamai CloudTest, this is something they need to focus on Priority. Most tools comes with default setup and ends to end testing scenario that can be covered, such as Regression testing. From Akamai CloudTest this is a missing features that can be added up soon.

Incentivized

• Quicker testing leads to faster product to market times
• Can deploy updates much more regularly by testing across many devices quickly

Incentivized

• System Hangs as where severe load testing is required
• Authentication is an issue when using OAUTH
• Scalability can be improved and also an on premise tool can be of great help

Incentivized