Item: AWS Device Farm
Rating: 10
Author: Amr Saad

Overall Satisfaction with AWS Device Farm

Use Cases and Deployment Scope

AWS Device Farm is mainly used by the Research and Development Department in our organization. We use it before going live with our product to test and make sure that our applications are working properly through different Mobile Devices with different Operating Systems, OS Versions, Screen Resolutions and different Mobile Devices. Instead of building a full devices lab along with maintaining all kinds of devices like Mobiles and PCs and Tablets which can be very expensive, we can use AWS Farm as per our need and pay as we go. Also, sometimes we use it to conduct performance tests on our application by simulating different users using different devices.

Pros and Cons

Pros

Integration with CI/CD Pipeline
Diversity in Real Devices over the Cloud
Pay as you go option
Reproduce Issues and easily collect all kinds of logs

Cons

It can be a little bit costly if you use it heavily
Test Execution results report can be better
User Interface needs some Improvements to make it easier to use

Most Important Features

Pay as you go
Diversity in the real devices which covers customers devices easily
Ease of Integration with CI/CD Pipelines so we can utilize Automated Scripts

Return on Investment

Builds more confidence in the End Product as you validate how it behaves across different devices
The only downside is that it can be expensive if you use it without caution

Alternatives Considered

Sauce Labs and BrowserStack

In my opinion, AWS Device Farm is a very good overall choice but it lacks some tester-friendly features when compared with the competition, for example, having built-in Appium Inspector, Testing using Emulators, Headless Testing, and Visual Testing. But AWS Farm is better in terms of the Real Devices diversity with an Option to Pay as you go. I would highly recommend using AWS Device Farm as your lab over the cloud.

Key Insights

Do you think AWS Device Farm delivers good value for the price?

Yes

Are you happy with AWS Device Farm's feature set?

Yes

Did AWS Device Farm live up to sales and marketing promises?

Yes

Did implementation of AWS Device Farm go as expected?

Yes

Would you buy AWS Device Farm again?

Yes

Likelihood to Recommend

In my opinion AWS Device Farm is a very good choice for any organization that needs to execute tests on large number of real devices before their production deployment. They have one of the biggest devices farms I have seen and you can get your test results and integrate your automated tests easily. Also, you can add them to your CI/CD Pipeline using Jenkins or any other tool with ease. In my opinion don't use it in your day to day testing at the beginning because it will be costly, I would recommend using it at the end as the last couple of test cycles to make sure everything is working properly accross different devices.

Comments

Please log in to join the conversation