What users are saying about
119 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring#question3' target='_blank' rel='nofollow noopener noreferrer'>Customer Verified: Read more.</a>
Top Rated
236 Ratings
119 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener noreferrer'>trScore algorithm: Learn more.</a>
Score 8.6 out of 100

Sauce Labs

<a href='https://www.trustradius.com/static/about-trustradius-scoring#question3' target='_blank' rel='nofollow noopener noreferrer'>Customer Verified: Read more.</a>
Top Rated
236 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener noreferrer'>trScore algorithm: Learn more.</a>
Score 7.3 out of 100

Likelihood to Recommend

JMeter

JMeter is well suited for Java applications where the user can script the scenario once and make changes to accommodate for as many numbers of users for load test execution. The image and selection of any files or exporting files scenario is handled well.
It is less appropriate to test Ajax applications where it is required to script click per use.
Anonymous | TrustRadius Reviewer

Sauce Labs

Well suited:
  • You know which set of core devices you need as a company which would then become your private devices. Also the ability to swap devices for new ones annually is most useful and cost efficient.
  • Other devices that are used less often can then be used from the public set of devices.
Not quite suited for the scenario where speed and framerate are the utmost importance.
Anonymous | TrustRadius Reviewer

Feature Rating Comparison

Load Testing

JMeter
8.7
Sauce Labs
End to end performance management
JMeter
9.4
Sauce Labs
Integrated performance data
JMeter
9.4
Sauce Labs
Deployment model flexibility
JMeter
9.4
Sauce Labs
Real time monitoring
JMeter
8.4
Sauce Labs
Automated anomaly detection
JMeter
7.0
Sauce Labs

Pros

JMeter

  • Easy of use - in generate load like HTTP requests, and processing/analyzing the responses. No coding is necessary at the basic level, just need to understand load testing and the infrastructure being tested.
  • Automatic management of things like cookies to help with session state support - so you don't specifically have to worry about it or handle it
  • Lots of testing/configuration options to suit your needs in making the right load generation (sampling requests), and analyzing the results, including any pre and post processing of the results first. Things like the Beanshell/BSF pre/post processors, response assertion, regular expression extractor, XPath extractor, CSV data set config
  • There is a JMeter cloud service called BlazeMeter that I think would be useful for those that need to scale up high load without provisioning their own systems. I've not personally tried it though, but I recently attended a meetup presentation that highlighted nice useful features that BlazeMeter provides. One should evaluate the service if they are considering JMeter and need to expand beyond existing hardware resources.
David Luu | TrustRadius Reviewer

Sauce Labs

  • Gets good people to work for it. These guys and gals are freed and just want you to succeed.
  • Sauce Labs listens to what the customers want. (Though I could still use an AWS East-1, hint-hint.)
  • They come to us when we ask, so that is pretty cool.
Anonymous | TrustRadius Reviewer

Cons

JMeter

  • Jmeter requires many tweaks with respect to its configuration file and thread properties. users need to edit theses files themselves. There could be some interface where we can edit this fields.
  • Jmeter cannot handle more threads and hangs up when we increase the number of threads. This causes lot of inconvenience. In these situations, user can be notified that such change would be lead to slow performance so that user can do as required. The same appears when we try to view huge files on graph listener.
  • Jmeter should optimize the read and write access to output csv since it acts as overhead to the I/O performance. This affects our test results for the application which we are testing.
Ameeta Muralidharan | TrustRadius Reviewer

Sauce Labs

  • In Sauce University trainings, it would be nice we could receive more in-depth information, as opposed to a short paragraph of information. As a beginner in the technological field, I would definitely find that helpful. Maybe they could do something where you can create a link to show more details, so that more advanced readers don't have to open it, but beginners can actually open up that link to read more.
  • Also, it would be nice if in the trainings you would include the videos as well because a lot of people are visual learners.
  • I wish it were easier to search through the Sauce Labs help page. Sometimes it takes quite a few searches to get to the right result. For instance, I'm not sure if I should click on Knowledge Base or Documentation or Support- there's too much to click-through. There should just be a Search All button that would search across the whole Sauce Labs page, so that we can easily find the answer that we need.
  • Our team especially loves the challenges, because again that fosters friendly competition and encourages us to continue to learn more about the product and the tech, as well as hone previous skills we've learned through Sauce Labs. We definitely hope that more of those classes come out every week, so we can dive deeper into each concept. I think there should be a feedback page where we can submit ideas for the classes that we would like to see.
Anonymous | TrustRadius Reviewer

Likelihood to Renew

JMeter

JMeter 8.6
Based on 12 answers
Price, Wiki and user sharing. Having access to the information provided by the developers and other open source providers is key for me. The ability to share information and get answers directly is very important to success in software testing. And the price of this product currently is amazing. Too many companies charge way too much money for products that are far behind in their value and pertinence
Charlie Harris | TrustRadius Reviewer

Sauce Labs

Sauce Labs 8.0
Based on 17 answers
Sauce Labs has become an integral part of our entire continuous delivery pipeline. Our architectural vision for CI/CD simply would not be possible without Sauce Labs given the resource constraints our company has to develop and maintain a solution such as this internally.
Brook Beam | TrustRadius Reviewer

Usability

JMeter

JMeter 9.0
Based on 1 answer
I can jump right into a new test plan and start building from scratch. The natural progression from test plan to thread group and then designing the basic format of the process is very streamlined and smooth. With only slight modifications I can build out a very complex model from a very basic beginning.
Charlie Harris | TrustRadius Reviewer

Sauce Labs

Sauce Labs 8.6
Based on 14 answers
I think Sauce Labs is great and I've been using them since late 2014. With that said, I'm very used to how their interface works and how to get what I need. I think if a brand new user would come in and use it, it would still be easy, but maybe not as easy as it is for me. That's why I rate this area a 9 instead of a 10. Still great though!
Randall Kelley | TrustRadius Reviewer

Reliability and Availability

JMeter

JMeter 1.0
Based on 2 answers
No answer on this topic is available.

Sauce Labs

Sauce Labs 9.1
Based on 6 answers
Yes, Sauce labs is always there, and it is easy to troubleshoot when you are having any connectivity issue, they always keep you informed when they plan to perform any type of maintenance window on their side in advance, so you can plan and will not affect your current work. I do not recall any outage.
Anonymous | TrustRadius Reviewer

Performance

JMeter

JMeter 8.0
Based on 2 answers
No answer on this topic is available.

Sauce Labs

Sauce Labs 9.0
Based on 6 answers
The execution of automated tests in Sauce Labs virtual machines is slightly slower than its main competition. Furthermore, the spin-up time of virtual machines (particularly device simulators or emulators) is significantly slower in our experience than its primary competition. The Sauce Connect Proxy further adds to performance decrease when used in our experience (around 10%-20% decrease when used), such that each call to the remote WebDriver instance in your test scripts can take upwards of 100-200 milliseconds.
Brook Beam | TrustRadius Reviewer

Support Rating

JMeter

JMeter 9.6
Based on 4 answers
I have been using JMeter for the last year. By using this tool, you can make sure the system will work under varied loads. It helps us to simulate real time scenarios by creating required virtual users and make sure the application will work under load. Perform load, stress, and stability testing using JMeter.
Anonymous | TrustRadius Reviewer

Sauce Labs

Sauce Labs 8.3
Based on 22 answers
While initial response time is quick, the initial question is the only thing that happens fast.
That's a standard request like adding one more error log or make sure I use the latest version of libs.
All further cmmunication goes really slow.
I am not sure my requests reach proper addresses quick enough.
In 50% of cases I receive a response in a few months (in half of cases stating that the issue can't be addressed).
It's often would be enough just to know if the fix is on the roadmap, but slow responses don't allow any planning.
Anonymous | TrustRadius Reviewer

Implementation Rating

JMeter

No score
No answers yet
No answers on this topic

Sauce Labs

Sauce Labs 8.0
Based on 14 answers
I am not sure if it's my company that makes getting Sauce Labs integrated into the team difficult or is it Sauce Labs. The process for getting Sauce Labs for a project is quite a tedious process and the information for using Sauce Labs initially is quite lacking. There is little support for getting started
Nancy Cheng | TrustRadius Reviewer

Alternatives Considered

JMeter

I have used LoadRunner and Silkperformer, and so far Jmeter turns out be the easiest to use of all these. While each of them have their own ROI, Jmeter can be picked by anyone in hours and start testing within a day. While with other tools, we need to get license, install them (takes a while) and setup tests and firewalls, etc.
Anonymous | TrustRadius Reviewer

Sauce Labs

We have also tested out Browser Stack, which at the time was more geared towards manual testing. Although it appeared to support more mobile devices/browsers, we also wanted something that can plugin in easily with our existing Selenium test scripts. Sauce Labs was definitely more geared towards our goals at the moment which were to automation functional/regression testing and build it into our release pipeline.
Anonymous | TrustRadius Reviewer

Scalability

JMeter

No score
No answers yet
No answers on this topic

Sauce Labs

Sauce Labs 9.1
Based on 3 answers
this is a great tool to run automation testing, you will have a broad option of browsers and mobile devices, I am giving an 8 only because the overall performance is kind of slow, specially with the mobile devices
Anonymous | TrustRadius Reviewer

Return on Investment

JMeter

  • Gives liberty to developers and testers to perform testing. Easy to set up and run load tests, stress tests has improved testing.
  • Enhances the performance testing which ensures the delivery of well-suited applications.
  • The Enterprise messaging system, it's a great tool which helps in testing JMS based messages.
Anonymous | TrustRadius Reviewer

Sauce Labs

  • Monthly cost was a bit higher than expected, which had a slight negative impact
  • Immediate transparency into application performance on various mobile devices was a major positive impact
  • Ability to run tests on various versions as needed was a positive impact
Anonymous | TrustRadius Reviewer

Screenshots

Pricing Details

JMeter

General

Free Trial
Free/Freemium Version
Yes
Premium Consulting/Integration Services
Entry-level set up fee?
No

JMeter Editions & Modules

Additional Pricing Details

Sauce Labs

General

Free Trial
Yes
Free/Freemium Version
Yes
Premium Consulting/Integration Services
Yes
Entry-level set up fee?
Optional

Sauce Labs Editions & Modules

Edition
Live Testing$19.001
Virtual Cloud$149.001
  1. Per Month
Additional Pricing Details
Free service available for Open Source projects.

Rating Summary

Likelihood to Recommend

JMeter
9.5
Sauce Labs
8.8

Likelihood to Renew

JMeter
8.6
Sauce Labs
8.0

Usability

JMeter
9.0
Sauce Labs
8.6

Reliability and Availability

JMeter
1.0
Sauce Labs
9.1

Performance

JMeter
8.0
Sauce Labs
9.0

Support Rating

JMeter
9.6
Sauce Labs
8.3

Online Training

JMeter
1.0
Sauce Labs

Implementation Rating

JMeter
Sauce Labs
8.0

Scalability

JMeter
Sauce Labs
9.1

Add comparison