Skip to main content
TrustRadius
Sauce Labs

Sauce Labs

Overview

What is Sauce Labs?

Sauce Labs is a cloud-based platform for automated testing of desktop and mobile applications. It is designed to be instantly scalable, since it is optimized for continuous integration workflows. (The vendor says that when tests are automated and run in parallel on multiple…

Read more
Recent Reviews

TrustRadius Insights

Sauce Labs has proved to be a valuable tool for various teams and departments within organizations. Users appreciate the seamless …
Continue reading

Nice product overall!

9 out of 10
October 19, 2022
Incentivized
Use for E2E testing reporting. Use for accessing internal web app access with Saucelabs connect (tunnel). Use for debugging with the …
Continue reading

Sauce Labs review.

9 out of 10
October 12, 2022
Incentivized
I use SauceLabs for UI regression testing and API testing. A feature to show how many tests there are in total, not just test suites, on …
Continue reading

Sauce Labs Review

9 out of 10
October 11, 2022
Incentivized
Sauce Labs provides access to several cloud devices in Android and iOS with older OS versions as well, thus providing a wide range of …
Continue reading
Read all reviews

Awards

Products that are considered exceptional by their customers based on a variety of criteria win TrustRadius awards. Learn more about the types of TrustRadius awards to make the best purchase decision. More about TrustRadius Awards

Reviewer Pros & Cons

View all pros & cons

Video Reviews

1 video

Sauce Labs Provides Details for Android Developer and Engineer
02:10
Return to navigation

Pricing

View all pricing

Live Testing

$19.00

Cloud
per month

Virtual Cloud

$149.00

Cloud
per month

Entry-level set up fee?

  • Setup fee optional
For the latest information on pricing, visithttps://saucelabs.com/pricing

Offerings

  • Free Trial
  • Free/Freemium Version
  • Premium Consulting/Integration Services
Return to navigation

Product Demos

WebdriverIO + Appium + Sauce Labs = Success? - Live Stream

YouTube

Automated Mobile Testing with Sauce Labs

YouTube

Parallel Testing with Sauce Labs

YouTube

Cross Browser Testing with Selenium, Sauce & Node.js

YouTube
Return to navigation

Product Details

What is Sauce Labs?

Sauce Labs is a cloud-based platform for automated testing of desktop and mobile applications. It is designed to be instantly scalable, since it is optimized for continuous integration workflows. (The vendor says that when tests are automated and run in parallel on multiple virtual machines across many different browser, platform and device combinations, testing time is reduced and developer time is freed up from managing infrastructure.) The Sauce Labs testing cloud is intended to be paired with a CI system. According to the vendor, this combination allows developers to easily test desktop and hybrid, native and mobile web applications early on in their development cycles, continuously and affordably.

Sauce Labs provides enterprise-grade security via Sauce Connect™, its secure tunneling protocol for testing behind a firewall while maintaining control of proxy and access policies. Tests are run in the company’s secure data center and VMs are destroyed (not “wiped”) after each run, ensuring customer data is never exposed to future sessions. After tests are completed the Sauce Labs’ dashboard provides a unique build-oriented report with metadata, access to Selenium logs, screenshots, video recordings, and a complete list of commands and responses. Support for SSO lets customers provision new user accounts on the fly with centralized user account management, access control, and usage reporting.

Sauce Labs Features

  • Supported: Automated Cross-Browser Testing
  • Supported: Automated Mobile Testing
  • Supported: Manual Testing
  • Supported: Real device cloud
  • Supported: Integrations with all CI servers and JIRA
  • Supported: Enterprise Security
  • Supported: Proprietary Data Center
  • Supported: Support for Selenium, Appium and JUnit Testing
  • Supported: Professional Services & Onboarding
  • Supported: Instructor-led Selenium and Appium training

Sauce Labs Screenshots

Screenshot of Sauce Labs UI optimized for continuous integration workflows.

Sauce Labs Technical Details

Deployment TypesSoftware as a Service (SaaS), Cloud, or Web-Based
Operating SystemsUnspecified
Mobile ApplicationNo

Frequently Asked Questions

BrowserStack, Perfecto by Perforce, and TestingBot are common alternatives for Sauce Labs.

Reviewers rate Availability and Product Scalability highest, with a score of 10.

The most common users of Sauce Labs are from Enterprises (1,001+ employees) and the Computer Software industry.
Return to navigation

Comparisons

View all alternatives
Return to navigation

Reviews and Ratings

(287)

Community Insights

TrustRadius Insights are summaries of user sentiment data from TrustRadius reviews and, when necessary, 3rd-party data sources. Have feedback on this content? Let us know!

Sauce Labs has proved to be a valuable tool for various teams and departments within organizations. Users appreciate the seamless integration of the platform with Jenkins CI servers, enabling continuous testing of web applications in real browsers and Dockerized environments. The ability to easily configure proxy tunnels and access firewalled environments and desired browsers via the Sauce OnDemand Jenkins plugin has been a major selling point.

One key use case for Sauce Labs is running regression test suites against different OS and browser combinations, which saves time and effort in maintaining test environments. Developer teams utilize Sauce Labs for running end-to-end Selenium tests, while the testing team manages its usage across the organization. Customer Service and Email Marketing teams also benefit from Sauce Labs, using it to identify and address customer issues and bugs before or after deployment. Additionally, the QA team relies on Sauce Labs for executing daily automation test cases on various platforms including mobile and web.

Best solution for deep automated browser testing: Users find Sauce Labs to be the best solution for integrating deep automated browser testing in a CI/CD pipeline, with multiple reviewers stating this as a key advantage of using the platform.

Ability to run test runs faster: Many users appreciate the ability to run test runs faster by using as many VMs as required, which is particularly helpful for CD/CI processes. This feature has been praised by multiple reviewers for its contribution to improving efficiency and reducing execution time.

Sauce Connect Proxy enables testing in firewalled environments: The Sauce Connect Proxy feature is highly valued by users as it allows companies to test in firewalled environments and localhost. Several reviewers have mentioned how this feature enables easy testing of applications behind a firewall, making it a valuable option for conducting tests securely.

Confusing and Difficult User Interface: Users have consistently found the user interface of the tool to be confusing and difficult to navigate, which has made it challenging for them to perform tasks efficiently. Several reviewers have expressed frustration with the complexity of the setup and configuration process, indicating a lack of user-friendliness.

Stability Issues: Some users have experienced stability issues with Sauce Labs, reporting that the tool is not reliable. These instances of instability can negatively impact the testing process for users, causing delays and uncertainties.

Limited Reporting and Customization Options: The lack of comprehensive reporting and customization options for dashboards is a drawback for users who require more detailed analytics and integration with APIs. This limitation has been mentioned by multiple reviewers, highlighting a need for improved functionality in this area.

Based on user reviews, users commonly recommend the following for Sauce Labs:

  1. Request a trial and take advantage of fast support. Users highly recommend asking for a trial of Sauce Labs. They praise the company's fast support, which is beneficial in getting started with the tool.

  2. Use Sauce Labs for multi-browser and multi-device automation. Sauce Labs is considered the best tool for cloud testing, particularly for testing web applications in different browsers, devices, and operating systems. Users mention its easy adoption and interface as strong points.

  3. Consider test architecture and logging for effective automation testing. Users suggest putting at least 2 retries in the test configuration when performing parallel mobile testing with Sauce Labs. They also advise considering test architecture and test logging to enhance the automation testing process.

Overall, users highly recommend Sauce Labs as a top service provider in the market, especially for continuous automated testing and serious mobile or desktop testing. It is praised for its secure nature and integration with real devices on the cloud. However, users also suggest trying other services like BrowserStack for comparison purposes.

Attribute Ratings

Reviews

(1-4 of 4)
Companies can't remove reviews or game the system. Here's why
Score 8 out of 10
Vetted Review
Verified User
Incentivized
Allows testing across all phases of development by all members of a cross functional team. Easy to access and use by all with little knowledge of Sauce Labs via the UI. Solves: Maintenance costs of devices including upgrades, software updates, malfunctions. Device shortage via use of public devices. Device sharing and collaboration. Access to devices during pandemic.
  • Range of devices are almost endless.
  • Combination of OS, Browser and browser versions are more than satisfactory.
  • Logs of every type at your fingertips.
  • Updates to the core could be better communicated.
  • Documentation can be outdated in certain places. Like broken links or now irrelevant.
  • Chat function would be nice as support even if it's a bot.
Well suited: You know which set of core devices you need as a company which would then become your private devices. Also the ability to swap devices for new ones annually is most useful and cost efficient. Other devices that are used less often can then be used from the public set of devices. Not quite suited for the scenario where speed and framerate are the utmost importance.
  • Eventually, our actual in-house devices would no longer be needed, where we’ve already stopped purchasing new devices.
  • A bit extra investigation needed. Device usage metrics must be gathered both on the Saucelabs side as well as our users so we can get a clear understanding of what devices we need so that each time we refresh our private devices, we get the ones in high demand
Yes
We had Saucelabs replace TestObject. Simple reason is that Saucelabs acquired them and made them part of the Sauce platform. All the features were migrated onto the Sauce platform (took a while longer than I'd wanted to be honest) with the added ability to access Saucelabs' desktop browsers via their virtual machines. It was a no brainer really as the cost stayed very close to what we had with TestObject originally.
  • Price
  • Product Features
  • Product Usability
  • Product Reputation
For us, we came from TestObject, which was acquired by Saucelabs. Pricing staying very close to what we originally paid for TestObject alone, but with the added benefit of desktop browsers.

Saucelabs is very well known in the industry, so with a community backing and great documentation, it was worthwhile making the jump.

As mentioned before in the previous parts, features are constantly being added to Saucelabs and they always have a habit of letting you know when they are available (via their UI), which is great!
At the time, we tried and tested many of competitors. But given that most of the competitors had less features with a higher cost, it was a no brainer for us to migrate from TestObject to Saucelabs since there was no added learning curve (due to the acquisition) and not much of a change in cost.
30
Cross functional development teams which consist of a mixture of QAs, backend developer and frontend developers. As you can imagine, Saucelabs is more utilised by QA, Frontend developers and backend developers, in that order. External of the teams like product owners or directors would make more use of the video features, like for example, sharing a test run which contains gameplay and/or tests running on a browser.
3
We have a small QA only team which supports with all things QA across the business. This team drives the direction of QA across all departments and provide not only Saucelabs support but support of our in house QA frameworks and technology. Saucelabs is fairly straightfoward to use, especially for manual QAs, so support is usually for automation and/or admin.
  • to replace real devices - no maintenance, purchasing, selling required
  • the ability to visually show test runs to stakeholders and non technicals
  • the ability to easily debug failed test runs
  • monitoring systems which spawn up a browser/device to ensure uptime is as expected
  • Not known as of yet. More features added in the future by Saucelabs may change this
As we currently know, there's nothing on the market with a big feature set like saucelabs at their current price point. Along with the business not having to learn a whole new tool to use again and the ability to refresh our private devices and the continuously growing number of public devices available and features.
  • Implemented in-house
No
Change management was minimal
Immediate support via an appointed sales rep via a Slack chat. But be prepared that they are not technical and can only help with basic support. Anything technical would be directed to a dedicated support team, but the solutions architect is usually available via Slack, so quite easy to get a hold of him for quick fire questions.
No. Did not need it since we are technical ourselves and are able to debug most of the issues we come across.
No
Probably a good thing, but haven't needed urgent support.

The times that we did raise a few things, the response time is fairly fast (within 24 hours). But then again, it was never for anything major.
The main use case of browser/device testing, be it manual or automation, is fairly easy to get started. The front end UI for manual testing is very intuitive and a non technical can easily just spawn up their needed browser/device. Automation is easy to setup, be it Java or Javascript where existing sauce bindings can be imported to quickly get setup.
  • Launching a browser/device manually or through code
  • Reviewing test runs through video and Webdriver comms list
  • Sharing a session through the UI via a link
  • USB remote debugging isn't straightforward
  • device usage metrics is non existent or I can't find it
  • Individual level usage is fine. But would like to see usage at a team level. i.e. ability to generate a team access key
Yes, but I don't use it
With private devices, you have full reign over usage of them, so no complaints there. Public devices are available if no one else is using it, which is understandable. Browser VMs are based on number of open sessions and Saucelabs give you a certain number depending on what you need. If you need more, then you pay for more. It is as simple as that. You need more devices, then you can pay for more private ones too.

A workaround for public devices is to pick the first available one and not be too picky, that's if you are able to of course.
I think there was only one occasion where we couldn't access our devices, but other than that, Saucelabs has always been up and running for us.
The time where they acquired TestObject and were trying to integrate their services would probably be the most annoying time. Annoying as features were in two separate places (websites) for example. But since the introduction of their unified platform, we haven't run into any issues as of yet and we've used them for at least 5-6 years now.
We are assigned our own sales representative or contact point, so any questions we had we could go to them. They were also placed in the same country as our business, so before the pandemic hit, we were able to always meet up to discuss things like our renewals or changes to our devices etc.
As stated before, our owns contact point and slack chat.
Since we were the original users of TestObject, of which they acquired, we were able to migrate our devices over from when we had a contract with TestObject as well as gain usage of Saucelabs' VMs for a very little increase in price. Turns out we use VMs more than we thought we would.
Just lay out your requirements as a business. Explain what you'd need devices/VMs for and at what scale and the Saucelabs rep would be able to recommend what's best for your in terms of browser sessions and devices that you'd need.
No
  • The two VM and real devices side being viewable on one platform (Unified platform)
  • Easier navigation and browsing through both real devices and VM browser runs
No
Yes
Not that it was a different edition, but it was their unified platform that integrated together their own VM browser service with their acquisition of TestObject (real devices). This took a while to fully roll out as we were dependent on certain APIs that were originally only on TestObject, but they eventually got there.
Score 8 out of 10
Vetted Review
Verified User
Incentivized
We use Sauce Labs in our engineering and QA departments to cover browser compatibility testing regressions, and it also helps with validating our Android and iOS apps through different mobile devices and OS versions.
  • Variety of browsers and versions
  • Variety of mobile brands and versions (Android and iOS)
  • The final report of each run is very useful
  • User-friendly
  • Testing on mobile phones interfaces is very slow
  • Sometimes we get disconnected from the cloud
  • Increase mobile phone options
Amazing to perform browser compatibility tests, and automation tests through Selenium and Jenkins integration. If you need to perform heavy tests on mobile phones, consider that it will be time-consuming.
  • ability to catch bugs or simply test specific browsers
  • you can run huge test runs against several browsers
  • definitely cheaper than maintaining a physical mobile device lab
  • keynote device anywhere
Sauce labs is faster and integrates with a lot of technologies. Sauce Labs user interface is very user friendly and easy to use. Once you define the parameters for your test runs it is very simple to let that go and just wait for the results, everything gets recorded so you can easily go to any specific test that requires more analysis.
It is great to have a very complete browser compatibility tool that is friendly with automation frameworks like Selenium, debugging options are very helpful too. The overall management is not complicated. Even when the Mobile devices test part is slow, having available all the different mobile operative systems with newer and older versions is amazing.
I can say that Saucelabs is reliable, in around 3 years we only used their support a couple of times, once you define your tunnel their service availability is really stable
Yes
Device anywhere
  • Price
  • Product Features
  • Product Usability
  • Product Reputation
  • Prior Experience with the Product
  • Third-party Reviews
we were looking for tools that cover Browsers and Mobile OS compatibility, Sauce labs have both! And you have a lot of different newer and older versions for browsers and mobile devices. Another plus: it is really use to integrate Sauce labs with other automation frameworks like Selenium or Jenkins. The debugging options are very helpful
we would choose Sauce labs again, we had been working with them for more than 3 years. It is reliable, user friendly, integrates easily with other technologies and keeps improving. Also the technical and administrative provided by Sauce labs is very professional and they always have an answer in a really short time.
25
Developers, Quality Assurance, Business Analyst, Product managers, Product Owners and Scrum Masters
2
We really don't have in-house Saucelabs support, just a couple of guys with Admin rights to add, delete or modify permissions for the rest of the team.
  • Browser compatibility tests
  • Android OS compatibility
  • Automated Selenium tests
  • we improve a lot in our browser compatibility, we had in the past special defect cases that occurred in production and we didn't had a way to see the problem. We use to release "fixes" hoping to fix that problems. Now we can replicate, debug and analyze the issue improving our code quality.
  • Android have a lot of different OS versions floating on the market, that was a blocker for us before, we use to test only on the newest Android OS version, but older versions had compatibility issues that we weren't able to debug and fix, Saucelabs improved this field a lot for us.
  • We have a dedicated team in charge to plan and implement a whole new UI automation framework using mainly Selenium and Saucelabs.
Saucelabs keep being a really important tool for our incoming new automation framework
Not Available
No
We contacted the Sauclebas support team when our cloud/tunnel was showing offline and they provide us a really quick tutorial about how to restart it by ourselves, being this even better because they show us a way to solve this issue without the need to call or reach them.
  • setting up a test session
  • selecting different browsers/versions/mobileUI
  • troubleshooting tunnel connectivity is very easy
  • users administration, access level
  • selenium integration
  • overall performance is slow
  • Mobile UI testing is super slow
this is a great tool to run automation testing, you will have a broad option of browsers and mobile devices, I am giving an 8 only because the overall performance is kind of slow, specially with the mobile devices.
Yes, Sauce labs is always there, and it is easy to troubleshoot when you are having any connectivity issue, they always keep you informed when they plan to perform any type of maintenance window on their side in advance, so you can plan and will not affect your current work. I do not recall any outage.
Yes, sauce labs coulb be a little slow, but being honest... the value that sauce labs provide us is worth it. We connect Sauce labs to call our web apps and it doesn't affect them it is only that it's performance is not that fast, specially when you are working with mobile testing
It is really nice to work with Sauce labs, reps and tech support. All of them are very friendly and supportive, also their response time is very quick.
They do not try to sell you unnecessary things or push you to take enhancements that you don't need, they will hear what are your needs and they will offer what better could work for you.
Again, Sauce labs customer appreciation is very nice, even after signing a contract to get their services, they constantly check on us to confirm that we are doing good with their product and to check if there is any concern on our side that they need to take action to solve it.
we negotiate the number of active connections, the different types of browser testing and mobile UI devices availability.
Just be clear on what your needs are and be sure that Sauce labs will cover these needs, this is a great company to work with.
March 31, 2019

Amazing website

Score 10 out of 10
Vetted Review
Verified User
Incentivized
At this time we are currently using Sauce Labs across the entire organization. Within 1 to 2 months of being hired, we all get an invite to join Sauce Labs from our email and then we're set a couple of reminder emails as well. Our organization is B2B software, and our goal is that no matter what department you're in, you'll be educated on the technology we all ultimately work with. We're all encouraged to take additional training to get ourselves up to speed about different kinds of software and technology, as well as troubleshooting devices and codes. What my team specifically uses is Sauce Labs University, which is where we can build our knowledge of IT and coding that we can apply to our job. The benefit that we get from this is that we can better understand what goes on in the back end, and we can therefore work more efficiently and better address customer-facing questions.
  • One thing that Sauce Labs does very well is continuing to provide education for technological products, so that we stay up to speed and are even able to learn new things that we never knew before. I definitely saw myself applying a lot of the new concepts that I learned, such as automated testing.
  • I also love the Secret Sauce where we can learn about new product updates for Sauce Labs, as well as learn about upcoming events such as SauceCon. I love being able to stay abreast of a company that I love, as well as be able to learn of more interactive meetings where I can meet tech-minded people, as well as the Sauce company, i.e. Saucecon.
  • I really love the customer service. Once in the Forum, I had a question, and Alyssa the VP actually gave me a reply, and was very quick to respond as well as very friendly. I've never had a company where the leadership was so responsive and so eager to dive in with customers on issues and questions that they have. This definitely shows that Sauce isn't just about getting profits, and that they truly deeply care about making the customers happy and successful.
  • Lastly, I love how Sauce University has a stash which encourages friendly competition among team members. It shows who has the most points, which encourages us to continue to actively participate in those trainings and challenges and quizzes. This in turn adds an element of fun to our work.
  • In Sauce University trainings, it would be nice we could receive more in-depth information, as opposed to a short paragraph of information. As a beginner in the technological field, I would definitely find that helpful. Maybe they could do something where you can create a link to show more details, so that more advanced readers don't have to open it, but beginners can actually open up that link to read more.
  • Also, it would be nice if in the trainings you would include the videos as well because a lot of people are visual learners.
  • I wish it were easier to search through the Sauce Labs help page. Sometimes it takes quite a few searches to get to the right result. For instance, I'm not sure if I should click on Knowledge Base or Documentation or Support- there's too much to click-through. There should just be a Search All button that would search across the whole Sauce Labs page, so that we can easily find the answer that we need.
  • Our team especially loves the challenges, because again that fosters friendly competition and encourages us to continue to learn more about the product and the tech, as well as hone previous skills we've learned through Sauce Labs. We definitely hope that more of those classes come out every week, so we can dive deeper into each concept. I think there should be a feedback page where we can submit ideas for the classes that we would like to see.
I would say that Sauce Labs is best suited for troubleshooting internal issues that we do have customers. Currently we do use Sauce Labs with JIRA integration which easily allows us to keep track of the tickets that we are sending to a technical team on the half of our clients that utilize our software. It also allows our technical team to use that interface to investigate what's going on in the background for the issues that our clients AK and users are facing. It's also well suited for educational purposes, because you get to use Sauce University as well as attend any boot camps to continue to broaden your knowledge and become a stronger technological resource in the company.
Previously, we were just using internal tracking systems and internal software in order to track bug issues as well as work on bug issues. With Sauce, there was definitely learning curve as everyone was used to working on the internal systems for over 10 years so it was definitely a hard change to break away from tradition. In the end, however, we did decide to go with Sauce because they had great customer service and were able to answer all our questions throughout the purchase process, and were able to demonstrate the value that they would provide our company through demos and case studies. Not only that, but ultimately we felt that Sauce Labs would increase our resolution time to solve those bugs that were such a headache for our team.
Not Sure
  • Product Features
  • Product Usability
  • Product Reputation
We were reccomended by peers to use it.
We put a lot of trust into word of mouth.
I would also love to have read testimonials
90
We have people in Customer Service and IT. As well as quality assurance folks.
20
These people who use on an ongoing basis are our IT folks.
  • Ease of use
  • Pricing
  • Security
  • Security
  • Sauce Labs Challenges to promote learning
  • N/A
I love how it's super easy to use. Not only that but it ensures security and it always has great updates to ensure that it continues to be awesome.
Also, I love how nice the people there are too. Their support tea is amazing. Anytime, I have a question, they are super speedy to answer.
  • Don't know
Change management was minimal
The lesson was to take it slow and give enough time for everyone to catch up to learning curve.
  • people had learning curve
  • some people only wanted to do things the old way
Not really. I personally thought it was good. Some people though were stuck on doing things the old way but eventually were won over.
  • Self-taught
Yes. I looked at the webstie and Sauce Labs University. I learned the basics.
no, just right
no
a time saver
The people here are just so friendly and personable.
For instance, Tristan Lombard answered every single email with a very cheery tone and not only did he diagnose my issue, he also made sure to ask how my day was going. Keep it up.
Yes,
Tristan Lombard was super quick to help with one of my questions. He immediately responded back in the same business day. I did not expect that but I really appreciated how quick he did it. Not only that but he made his response back very personable.
  • sauce lab challenges
  • AI
  • N/a, they provide great training so I didn't find anything difficult
Their training is amazing so I already felt up to speed when using itl
Yes it has been very reliable. Easy to use and no bugs
Yes, no outages.
Reports do complete in a reasonable time frame, given their complexity. Sauce Labs does not tend to slow other systems down.
  • none
n/a
  • salesforce
n/a
  • File import/export
none
It was super easy to integrate.
Just make sure they have that integration and then do it. :) Very user friendly
Vendor was very friendly
Vendor was nice and kept following up
not applicable
Just ask as many questions as you need to.They are super patient and friendly
  • increased efficiency
  • more trainings
Brook Beam | TrustRadius Reviewer
Score 8 out of 10
Vetted Review
Verified User
Incentivized
As we began architecting a CI/CD pipeline in our SDLC, a need quickly arose for left-shifted, continuous testing of our web applications in real browsers. Specifically, a service that could integrate seamlessly with our Jenkins CI servers and executing tests continuously against our Dockerized applications and firewalled environments. Furthermore, our company does not have the resources to implement and maintain a proprietary Selenium Grid for test distribution. Sauce Labs addresses all of these concerns, and furthermore offers a platform for manually debugging applications in any combination of platforms, browsers, and devices needed, either from an automated test script or from a manual test session. The largest selling point for our needs is its relatively seamless integration with our Jenkins CI servers, including detailed playback and test reports per build job, as well as easy configuration of the proxy tunnel used to access our firewalled environments and desired browsers-under-test, right from the Jenkins UI via their Sauce OnDemand Jenkins plugin.
  • Sauce Labs offers the single best solution for integrating deep automated browser testing in a CI/CD pipeline, and integrates best with Jenkins over their competition.
  • The Sauce Connect Proxy is a necessary technology for companies who have firewalled testing environments, as well as localhost testing.
  • The Jenkins UI plugin offers a way to easily configure browser settings for tests being executed continuously.
  • Browser support is quick to be updated as new versions of browsers are released.
  • The custom capabilities that can be provided to Sauce Labs VMs during automated testing sessions are a valuable option for experimental or niche testing.
  • The Jenkins Sauce OnDemand plugin could have more options available to tap into more of the custom capabilities Sauce Labs actually supports. Currently, in order to tap into these capabilities, a programatic solution is required from test code, rather than being able to configure them via the Jenkins plugin UI. For example, things like desktop resolution, browser automation binary versions, remote Selenium version, and browser console logs.
  • Some issues with the Sauce Connect Proxy on our Jenkins servers failing to start, which makes the optimal CI/CD pipeline come to a halt. Better documentation, logging, and more intuitive configuration around the Sauce Connect Proxy would take what is already an industry leading feature to the next level.
  • User account management needs an overhauls, allowing for user groups, rather than just a hierarchy structure.
  • Test job queuing is first-come-first-serve, meaning there is no in-built way to queue batches of jobs from, say, different Jenkins servers/accounts.
If your need is architecting a CI/CD pipeline with left-shifted continuous real browser testing, then there is simply no better service available than Sauce Labs to integrate with this desired model. They are a smaller company than their leading competitor, but they are rapidly innovating in the right direction, and lead the charge in continuous automated testing. If, however, your need is specific to manual test sessions, or manually-executed automated test scripts, then their competition becomes more appealing given price points, market solidification, and support.
BrowserStack captures more market share than Sauce Labs does as of this writing, and thus has a much larger support network. During testing, it was also a little faster to spin-up virtual machines, and substantially quicker spinning-up mobile simulators and emulators. Pricing models were very similar in the enterprise tier. However, while BrowserStack may have been a better choice for manual testing or manually-executed automated tests, Sauce Labs offered a better product and better vision for tests that were left-shifted in a continuous delivery pipeline, which is the architectural direction or company has gone. Their service was proven to better integrate with this pipeline than BrowserStack.
No
  • Price
  • Product Features
  • Product Usability
  • Positive Sales Experience with the Vendor
  • Third-party Reviews
The features of Sauce Labs over its primary competition are ultimately what drove the purchase decision. Sauce Labs offers a far-better solution for integrating deep user interface testing into a continuous delivery pipeline, that takes care to allow for testing of firewalled environments. The Jenkins support, specifically, informed our decision quite a bit, since we use Jenkins CI servers for our CI/CD model.
Reaching out to Sauce Labs representatives to get a timeline estimate on development of new features might have been an addition to the process we considered. Furthermore, a lengthier trial implementation to better determine some of the shortcoming of features we liked might have been beneficial, such as relatively restrictive user account management.
30
Software Architects, Software Development Engineers, Business Analysts, and Software Test Engineers all leverage the Sauce Labs service in some way.
2
Most support required for Sauce Labs involves coding test automation solutions that interface with Sauce Labs and Jenkins in the ways that are expected of our CI/CD model, and evolve along with it. There are also the administrative tasks surround user account, including creation of new users in the Sauce Labs service, as well as managing their privileges.
  • Functional and Regression testing of our web applications via the user interface
  • A VM grid supporting all necessary platforms, devices, and browsers that tests can be executed in from our CI servers that is maintenance-free
  • Ability to debug browsers, platforms, or devices that are not readily available in-office
  • Integration with our continuous delivery pipeline
  • Detailed test reports and playback
  • The ability to test the responsiveness of our user interfaces against smaller resolutions (either desktop or mobile)
  • The ability to test multiple independent browser sessions within the execution of a single test, such as for testing session independence for Single Sign-On cases, or for testing concurrent usage of an application across multiple browser sessions
  • The ability to execute automated or manual tests in Sauce Labs against localhost
  • Continuous and left-shifted performance testing of the User Experience
  • Real Device testing, rather than mobile simulators or emulators
  • More extensive mobile testing
Sauce Labs has become an integral part of our entire continuous delivery pipeline. Our architectural vision for CI/CD simply would not be possible without Sauce Labs given the resource constraints our company has to develop and maintain a solution such as this internally.
  • Implemented in-house
No
Change management was minimal
N/A
  • Ensuring that implemented automated test scrips could interface with Sauce Labs from either a local workstation or a Jenkins CI server
  • Implementing solutions that allowed for all of Sauce Labs capabilities to be configured either from a local workstation or a Jenkins CI server, since the Jenkins Sauce OnDemand plugin offers less than the service provides in its user interface configuration
  • Determining a solution for test job queuing. Sauce Labs does not support batch job queuing, but rather test jobs are queuing by first-come-first-server. What this means is that if your organization has multiple CI servers, each with their own Sauce Labs account and each wanting to execute a batch of test jobs, your organization must internally implement a solution to ensure that those concurrent batches of tests do not collide with each other.
By far the largest deduction for satisfaction is the lack of complete capability support in the Jenkins Sauce OnDemand plugin's user interface, as well as the lack of being able to queue test jobs by batches. Our organization has multiple Jenkins servers, and each has its own Sauce Labs account for executing tests at built time. If multiple Jenkins servers kick off test batches at the same time, we have to develop a solution internally to manage those collisions.
  • Self-taught
Product is extremely easy to learn on one's own, and requires little technical knowledge unless you are attempting to interface with the service from an in-house test automation framework. The documentation is well done, and updated with an acceptable frequency. Furthermore, they have a knowledge base and community that can be used to research more niche issues or questions.
The configurability is quite extensive, but to leverage it in a CI/CD model from a CI server requires a programmatic solution to bridge the gap between source code configuration and Jenkins Sauce OnDemand plugin's UI configurations, since the Jenkins plugin's configurations are far more limited than what Sauce Labs actually supports (yet are a preferred way of configuration since they do not require code or configuration file changes).
Ensure that automated test source code and leverage all of Sauce Labs features from either a local workstation or a CI server, and that (if applicable) the Jenkins Sauce OnDemand plugin's configurations take priority over source code configurations, unless source code configs explicitly override the Jenkins plugin configs. This is the best way to ensure that specific test cases can have specific configurations without blanketing them with a configuration for the entire suite, as well as ensure that capabilities Sauce supports that aren't reflected in the Jenkins plugin can be leveraged.
No - there is no facility to customize the interface
No - the product does not support adding custom code
An internal solution needs to be implemented that organized batches of automated tests from multiple CI servers, such that if batches are executing concurrently, their individual tests do not collide with one another in the Sauce Labs service and unexpectedly subtract from the available concurrency pool. Sauce Labs does not support such batch queuing natively.
Yes
Premium support comes with entry into the enterprise license tier, which our company is a part of for legal reasons.
On the enterprise license tier, support tickets are answered very quickly. Furthermore, when issues are detected in the service itself, status pages are updated frequently, and notifications sent out with updates on the status of those issues. It would be nice, however, to see issues that are the result of missing features be taken more seriously, or a more seamless integration between support tickets and feature requests, rather than having the consumers of the service have to submit their own feature requests.
No
Our trial period before we decided to purchase Sauce Labs was extensive. At this time, we were still entertaining the idea of their main competitor as well. We mostly needed to vet what service would best work with our continuous delivery pipeline, and we needed time to prove this out. The Sauce Labs representative we were working with at the time was more than helpful and went above and beyond to ensure that our trial period was extended to as long as we needed to make an informed decision.
  • Manual test sessions are very easy to spin up, and customize your platform, device, and browser and its version
  • Basic Jenkins integration is incredibly easy via the Sauce OnDemand Jenkins plugin. Configuration is done completely via its user interface
  • Test playback and metadata viewing is incredibly intuitive, for both manual and automated testing sessions
  • Sometimes the Sauce Connect Proxy on a Jenkins server can be inexplicable flaky and fail to start during a build job. When this occurs, debugging the cause is difficult, and it breaks the intention of the CI/CD model.
  • To execute a test in Sauce Labs from a local workstation against an application/environment that is firewalled requires starting the Sauce Connect Proxy utility on the local machine. This is a command line utility that may be unwieldy for non-technical people.
  • The Jenkins Sauce OnDemand plugin is, as of this writing, relatively restrictive in what capabilities it can set for test execution. If your automated tests wish to tap into more granular capabilities that Sauce Labs supports (such as desktop resolutions, Selenium version in remote VM, automated driver binary version, etc), then a programatic solution is required, as these capabilities cannot be configured from the Jenkins plugin user interface.
Yes, but I don't use it
It is an incredibly easy service to use for what its primary intention is. The only reason a point is deducted is because more feature enrichment can be done around the Sauce Connect Proxy utility and the Jenkins Sauce OnDemand plugin. User Account administration also needs more work, such as the addition of user groups, rather than a simple hierarchy of users.
The product is as scalable as your company's wallet can afford, since you are charged per unit of concurrency, where each new unit of concurrency represents another concurrent virtual machine you can spin up.
Outages with the virtual machine services or the Sauce Connect Proxy service are more frequent than would be ideal, but updates to resolutions are communicated frequently, and resolutions usually come quick.
The execution of automated tests in Sauce Labs virtual machines is slightly slower than its main competition. Furthermore, the spin-up time of virtual machines (particularly device simulators or emulators) is significantly slower in our experience than its primary competition. The Sauce Connect Proxy further adds to performance decrease when used in our experience (around 10%-20% decrease when used), such that each call to the remote WebDriver instance in your test scripts can take upwards of 100-200 milliseconds.
Slack integration is communication of Sauce Labs outages et al, and was easy to achieve via Slack configuration. Basic Jenkins configuration is very simple via Sauce OnDemand Jenkins plugin that offers a configuration user interface.
N/A
  • API (e.g. SOAP or REST)
Basic Jenkins integration is simple. A big exception, however, is if leveraging all of Sauce Labs capabilities is desired, which requires a programmatic approach.
N/A
Our company required signing of our own Non-Disclosure Agreement, which Sauce Labs was initially unwilling to do given the entry-level enterprise tier we were purchasing. This required more back-and-forth than we would have liked and delayed our purchasing process (and implementation) more than was ideal.
During purchasing talks for additional services after initial purchase, the sales staff worked with was extremely rude and pushy, and seemed annoyed by inquiries and requests.
N/A
Know that the enterprise pricing is flexible and can be negotiated.
No
  • N/A, cloud-based
  • Hopefully more configurations for user account administration
  • More features implemented for the Jenkins Sauce OnDemand plugin
  • More fleshed-out Sauce Connect Proxy utility
No
No
Return to navigation