AWS Device Farm vs. IBM Rational Quality Manager

AWS Device Farm is perfect for small teams who don't have enough resources to conduct large scale testing scenarios or even large teams who would like an easier time with testing on various devices. [AWS] Device Farm makes it very easy to interact with a multitude of devices right from the browser.

Incentivized

IBM Rational Quality Manager is well suited to be used by the Quality Team in any Software Project. The Application provides good features to track Test Cases and Test Execution Results. On the other hand, the Application suffers from a very Old User Interface and a Very Bad overall User Experience. You have to go through very long and complicated steps to something which is not intuitive at all. Also, some features depend on having IBM RTC integrated with it.

Incentivized

• Integration with CI/CD Pipeline
• Diversity in Real Devices over the Cloud
• Pay as you go option
• Reproduce Issues and easily collect all kinds of logs

Incentivized

• It's part of Rational CLM suite so provides seamless integration with the Change Management module (RTC) and Requirement Management module (Rational DOORS).
• It can be integrated with IBM Rational automation, performance and security testing tools which provides an easy way to schedule or execute the scripts from RQM console. Results are linked with test plans
• Provides Lab management facility, which helps to maintain, allocate and monitor your testing assets like test servers, devices etc.
• It provides good way to manage multiple test execution cycles. all execution cycles are linked with test plans.

Incentivized

• Interface (CLI) not very easy. Competition has better options.
• Integration with IDE for test case coding.
• Reporting for the test scripts executed.

Incentivized

• It does not provide any functionality for defect logging. For defects, you will have to have Rational Team Concert. So it's useful if you have the complete suite.
• Similar to defects it also does not provide builtin functionality for requirements. Requirement module needs to be purchased separately .
• It does not have query builder like in RTC, where a user could define its criteria and see the results easily. There is some mechanism to filter the results while you browse test plans and test cases but it's not very user-friendly and sufficient.

Incentivized

We had the enterprise support with AWS, so overall support experience was good with great engineers on the back providing answers. As you may know, overall AWS support is different and this is not different. Responses through the regular web support channel came easily, fast and accurate. We had questions/issues which were solved fast. Documentation is good as well, especially around the test automation pieces.

Incentivized

I got more features and handy within AWS Device farm than AWS Cloud9

Incentivized

We have used TestLink before using IBM Rational Quality Manager and there is no comparison between them as TestLink will lose in every aspect against IBM Rational Quality Manager. I think it was a very good decision to migrate to it instead of TestLink despite the problems in the User Experience.

Incentivized

• Quicker testing leads to faster product to market times
• Can deploy updates much more regularly by testing across many devices quickly

Incentivized

• As a part of Rational CLM suite, it provides end to end traceability.
• It offers coverage report to managers and test leads so that they can identify the gaps in testing.
• If testers are familiar writing test cases in Excel sheets, it can be a little challenging to ask them to write all their testing artifacts in the tool, because they have to write a test plan, test cases and test scripts separately and then combine according to need. This requires some level of enforcement.

Incentivized