Apache JMeter vs. Appium vs. BrowserStack

JMeter is well suited for Java applications where the user can script the scenario once and make changes to accommodate for as many numbers of users for load test execution. The image and selection of any files or exporting files scenario is handled well.

It is less appropriate to test Ajax applications where it is required to script click per use.

Incentivized

1. It's open source which supports range of languages, operating systems and languages. Well suited for Android and IOS mobile automation. Supports all kinds of apps, which makes it flexible and robust mobile testing tool 2. It is less appropriate where we need intercept network call to verify the API calls. Extensive coding experience is required to work Appium

Incentivized

If you need immediate testing (for example, from Azure pipelines agents), use BrowserStack instead of LambdaTest (which makes you wait in a "pool" until the device/platform combo you are querying is accessible). If you don't need your tests to be immediate and prefer a wider range of devices, then use Lambdatest.

• Easy of use - in generate load like HTTP requests, and processing/analyzing the responses. No coding is necessary at the basic level, just need to understand load testing and the infrastructure being tested.
• Automatic management of things like cookies to help with session state support - so you don't specifically have to worry about it or handle it
• Lots of testing/configuration options to suit your needs in making the right load generation (sampling requests), and analyzing the results, including any pre and post processing of the results first. Things like the Beanshell/BSF pre/post processors, response assertion, regular expression extractor, XPath extractor, CSV data set config
• There is a JMeter cloud service called BlazeMeter that I think would be useful for those that need to scale up high load without provisioning their own systems. I've not personally tried it though, but I recently attended a meetup presentation that highlighted nice useful features that BlazeMeter provides. One should evaluate the service if they are considering JMeter and need to expand beyond existing hardware resources.

• It uses WebDriver API so it makes it easy to use for former web test automation engineers.
• It can be managed via the command line via an extensive set of parameters.
• It handles implicit waits at the server side that is especially valuable in distributed infrastructure.

Incentivized

• Reliable cross-browser, cross-device testing
• integration with JIRA to add screenshots directly to bugs etc
• Live mobile device testing for an extra layer of confidence
• Customer support has been very consistent and reliable
• User interface is incredibly simple to use and easy to train new users on

Incentivized

• Jmeter requires many tweaks with respect to its configuration file and thread properties. users need to edit theses files themselves. There could be some interface where we can edit this fields.
• Jmeter cannot handle more threads and hangs up when we increase the number of threads. This causes lot of inconvenience. In these situations, user can be notified that such change would be lead to slow performance so that user can do as required. The same appears when we try to view huge files on graph listener.
• Jmeter should optimize the read and write access to output csv since it acts as overhead to the I/O performance. This affects our test results for the application which we are testing.

• Element browser sometimes is unreliable and has sporadic fails.
• Appium running is a bit slow, compared to tests written with Appium and with Espresso or XCTest.

Incentivized

• The parallel execution is limited and is still in beta state. Once it starts to work properly, it will be more helpful
• The paid subscription can be a bit of an issue for some individuals or organizations as the charges are a bit too much
• Some of the devices do not open or load properly and are not available to test at times

Incentivized

Price, Wiki and user sharing. Having access to the information provided by the developers and other open source providers is key for me. The ability to share information and get answers directly is very important to success in software testing. And the price of this product currently is amazing. Too many companies charge way too much money for products that are far behind in their value and pertinence

I am rating 10/10 because Appium can use with multiple programming languages with multiple Test engines. Also, it works for both Android and iOS.

Incentivized

It's almost the 3rd year for us and it's renewal time for us. So yes, we are already discussing how many licenses we need to increase as users are increasing internally. So it's 100% sure that we are already planning renewal this year as well BrowserStack with live and app automate.

Incentivized

The purpose related to performance and load testing through Apache JMeter works fine but the usability of the tool should be improved quite a lot. If someone starts with the Jmeter fresh without prior experience, they need to put more efforts in understanding the tool. The UI is not that great which is the main reason not to give high rating on usability.

Incentivized

I would like to give 9/10 rating to Appium because of it can easily integrate with popular frameworks and CI/CD tools, as well as it is reliable, flexible and easy to use. The setup can bit complex in initial step, but once on configured it's very easy to use and enables stable and scalable mobile automation for real and cloud devices.

Incentivized

So many options that it can be a little overwhelming, but the core functions are easy to find and use and it's usually not too hard to figure things out for the more complex tasks. Very easy to boot up a device and a specific browser from the dashboard to begin a manual website test.

Incentivized

I rated BrowserStack's availability a 10 because it is consistently reliable, with minimal to no downtime or unplanned outages. The platform is accessible whenever needed, ensuring uninterrupted testing. Its robust infrastructure and proactive monitoring ensure a seamless experience, allowing us to meet deadlines without delays caused by availability issues and all

The tests are fast considering the fact that they're Appium tests. I've seen tests reliably pass or fail when they're supposed to, with next to zero issues on the BrowserStack side of things. Tests launch only seconds after I kick off them off from my CLI.

Incentivized

I have been using JMeter for the last year. By using this tool, you can make sure the system will work under varied loads. It helps us to simulate real time scenarios by creating required virtual users and make sure the application will work under load. Perform load, stress, and stability testing using JMeter.

Incentivized

I've not had much direct interaction with the BrowserStack support team. The help and community are great and we've not run into any issue that has really required us to reach out. I guess having a stable and easy to use system means you may never need to contact support.

Incentivized

Yes, it was online training on meet, and trainer looks like skilled and technical strong, he has covered end to end all the features and he has answers all the queries. because of this trainings we are able to implement it by our own in the organization, thank you for support and training.

Incentivized

It was a quick training from the support of browserstack, it was nice and easy to understand, thanks again for the support given by the team. and regularly I used to receive mails for training from support for any new feature they launch, I was able to spread same training to all my team and dev.

Incentivized

I rated the implementation satisfaction an 8 because while it went smoothly overall, there were some challenges during the initial learning phase and integration with existing tools. Key insights include the importance of providing sufficient training upfront and ensuring seamless integration with other systems to minimize disruptions and improve adoption speed.

I have used LoadRunner and Silkperformer, and so far Jmeter turns out be the easiest to use of all these. While each of them have their own ROI, Jmeter can be picked by anyone in hours and start testing within a day. While with other tools, we need to get license, install them (takes a while) and setup tests and firewalls, etc.

If you're an Apple developer, you use Xcode. It's practically a forced necessity. For system testing though, it doesn't have to be. You can have your development team focus on unit and integration tests in their platform and another team automate acceptance tests with a language they are more familiar with.

Incentivized

BrowserStack products has been found better for low code automations and visual regression techniques. We have been struggling to maintain the API endpoint sanity tests and writing a lot of code for them while releasing the builds, while we chose BrowserStack accessibility solutions, we found it a way easier than we thought and worked it up.

Incentivized

No Vendors were involved, directly dealt with Browserstack

Incentivized

I may not be the best person to answer this as I am only using it for 1 department and at 1 site but will still try my best As far as Scalability for Devices for Mobile Automation is concerned, it gets a Solid 10, as the users can run cases on upto 10 device parallel and also have the best choices of devices to choose

Incentivized

Not used

Incentivized

• Good ROI on improving the performance of the application.
• Finding issues in the performance.
• Benchmark the performance results.
• CON: Need skillset to create and maintain the scripts in Java.
• Scripts are reusable and it is executed by any user.
• Need Client and Server setup to execute the scripts.

Incentivized

• Appium is open source, so it's free. That's budget friendly right there.
• The ability to write mobile automation tests has saved considerable time for our manual test team, but that is true with most automation tests.
• We use Sauce Labs with our other automation, but Appium works great with Sauce Labs, as well, if I needed to run on emulators and simulators.

Incentivized

• It reduced the cost of buying multiple devices as we have multiple devices access in it
• Easily accessible from anywhere due to it cloud availability
• Introducing this to our client also helped us in building trust as well as winning their confidence during deliverables

Incentivized