AWS Device Farm vs Selenium | TrustRadius

What users are saying about

Score 10 out of 10

Based on 8 reviews and ratings

Score 8.9 out of 10

Based on 216 reviews and ratings

Pros

100% compatibility with the installed user base. No simulation, but real results.
Integration with Jenkins and other CI/CD tools.
A considerable amount of device types available. Again, real devices in the cloud!

Read full review

Verified User

Executive in Information Technology

Consumer Goods Company, 10,001+ employees

View all 2 answers on this topic

The efficiency in testing is beneficial with Selenium because without the questioning the potential usefulness of software, we wouldn't really be able to confirm that certain web applications will perform the way we want them to.
Quality Assurance has worked better with Selenium since this program helps eliminate manual QA testing, which itself could lead to exponential error, and cost.
A strength of Selenium is also the array of tools as well as cross-platform capability. The remotely controlled server acts as a device platform for JavaScript, PHP, Perl, and Ruby.

Read full review

Verified User

Manager in Quality Assurance

Civil Engineering Company, 51-200 employees

View all 51 answers on this topic

Cons

Cost and price is always a concern
Can be a bit clunky for the staff using the service.
Sometimes, misses issues with iOS devices.

Read full review

Verified User

Analyst in Other

Government Administration Company, 201-500 employees

View all 2 answers on this topic

Unfortunately, there's no way to run tests with playback on a single monitor. For those who simply do not have additional screens on-hand, Selenium is impossible to use. Those who do have multiple monitors may choose to use 1 monitor for any specified reason, therefore making Selenium a non-option.
As a program designed to function within Firefox, users tend to experience technical issues with opposing browsers. Although Selenium has been improved over the years, it still has not mastered cross-browser compatibility.

Read full review

QA Lead

Inhance DigitalMarketing and Advertising, 51-200 employees

View all 51 answers on this topic

Likelihood to Renew

No score

No answers yet

No answers on this topic

Selenium 9.2

Based on 6 answers

We love this product mainly because of its high customization abilities and the ease of use. Moreover, its free and can be learned easily through online communities and videos. The tests are more consistent and reliable as compared to Manual tests. It has enabled us to test a large number of features all in one go, which would have impossible through manual tests. The reports generated at the end of the tests are really helpful for the QA and the development teams to get a fair view of the application.

Read full review

Siddharth Walia

Test Engineer

NexJ SystemsComputer Software, 201-500 employees

View all 6 answers on this topic

Usability

AWS Device Farm 10.0

Based on 1 answer

The users have no issue. The IT staff have learned to work with the software with the help of AWS support. I find it very effective. Could it be easier? Of course. That doesn't mean this isn't a great solution. I would recommend it to anyone based on usability. Overall, it is easy enough to use.

Read full review

Verified User

Analyst in Other

Government Administration Company, 201-500 employees

View all 1 answers on this topic

Selenium 8.7

Based on 6 answers

As I mentioned earlier, the reason I use Selenium is because there is a fairly widespread community of users, and user support services are at a good level. because the application is open source, it works on many platforms (Windows, Linux, IOS) without any problems. In addition, it gives us a lot of options for writing functional tests. For errors that we receive through the application, we can easily find the reasons for errors in the forums.

Read full review

Business Intelligence & Database Consultant

EMO A.ŞInformation Technology & Services, 501-1000 employees

View all 6 answers on this topic

Support Rating

AWS Device Farm 10.0

Based on 3 answers

We had the enterprise support with AWS, so overall support experience was good with great engineers on the back providing answers. As you may know, overall AWS support is different and this is not different. Responses through the regular web support channel came easily, fast and accurate. We had questions/issues which were solved fast. Documentation is good as well, especially around the test automation pieces.

Read full review

Verified User

Executive in Information Technology

Consumer Goods Company, 10,001+ employees

View all 3 answers on this topic

Selenium 8.1

Based on 21 answers

The tool can be used across organizations by various roles. It's free of cost and integrates well with most of the sophisticated or opensource development, build management, CI/CD, or management tools. Selenium has outstanding community support and is continuously updated to support newer technologies. The availability of lots of plugins/add-ons makes it a tool that can talk to almost any other tool available in your organization.

Read full review

Sachin Gulmadagkar

Senior QA & Test Manager, Head of Test Automation

Bravura SolutionsBanking, 201-500 employees

View all 21 answers on this topic

Implementation Rating

No score

No answers yet

No answers on this topic

Selenium 9.0

Based on 6 answers

We did everything we needed to use it.Now we can execute our tests on different operational systems and browsers running few tests simultaneously.We also implemented Appium framework to execute our tests on mobile devices, such as iPhones, iPads, Android phones and tablets.We use SauceLabs for our test execution and Jenkins for continuous integration.

Read full review

Dmitry Shyshkin

Lead QA Automation Engineer‏

FareCompare.comMarketing and Advertising, 11-50 employees

View all 6 answers on this topic

Alternatives Considered

This is the only solution I have used from device farming. From that perspective, I would purchase it again. I think this is a great solution for a large, diverse organization looking to mass deploy and manage software and devices. We selected this software because we have had great luck with other AWS services.

Read full review

Verified User

Analyst in Other

Government Administration Company, 201-500 employees

View all 2 answers on this topic

HQ UFT, it is one of the best and has more abilities but it is too much expensive while Selenium is free.SmartBear TestComplete, same reason as UFT.Watir, it is a Selenium-like open source project but has less features and limited documentation.

Read full review

Payam Pirooznia

Senior Automation Test Engineer

ViaSat Inc.Telecommunications, 1001-5000 employees

View all 51 answers on this topic

Return on Investment

Reduced the amount we spent on physical devices by > 90%.
Enabled real CI integration even when running E2E test scripts using mobile phones.
Helped us to improve overall App stability and rating - fewer crashes (80% to 99% of sessions crashless) and 75% rating increases on apps.

Read full review

Verified User

Executive in Information Technology

Consumer Goods Company, 10,001+ employees

View all 2 answers on this topic

There hasn’t been a downside to using it yet other than we’ve got to update the programs we create for each change.
This has saved us hundreds of hours of manpower by allowing our automation engineer to rapid fire tests.
We are able to screenshot and save entire sites before and after launch with a program the automation engineer created
We can compare large volumes of data against data in excel docs with a program created using Selenium

Read full review

Janice Cruz (L.I.O.N.)

Quality Assurance Analyst

Deltak - Wiley Education ManagementMarketing and Advertising, 501-1000 employees

View all 51 answers on this topic

Pricing Details

General

Free Trial

—

Free/Freemium Version

—

Premium Consulting/Integration Services

—

Entry-level set up fee?

No

AWS Device Farm Editions & Modules

Edition

Pay-As-You-Go	$0.17¹
Unlimited Testing	$250.00²
Private Devices	$200.00²
Pay as You Go - Desktop Browsers	$0.005³

per device minute
per month
per instance minute

Additional Pricing Details

—

General

Free Trial

—

Free/Freemium Version

—

Premium Consulting/Integration Services

—

Entry-level set up fee?

No

Selenium Editions & Modules

—

Additional Pricing Details

—

Add comparison