TrustRadius Insights
Sauce Labs has proved to be a valuable tool for various teams and departments within organizations. Users appreciate the seamless …
Starting at $19 per month
View PricingOverview
What is Sauce Labs?
Sauce Labs is a cloud-based platform for automated testing of desktop and mobile applications. It is designed to be instantly scalable, since it is optimized for continuous integration workflows. (The vendor says that when tests are automated and run in parallel on multiple…
Recent Reviews
Awards
Products that are considered exceptional by their customers based on a variety of criteria win TrustRadius awards. Learn more about the types of TrustRadius awards to make the best purchase decision. More about TrustRadius Awards
Reviewer Pros & Cons
Video Reviews
1 video
Sauce Labs Provides Details for Android Developer and Engineer
02:10
Pricing
Live Testing
$19.00
Cloud
per month
Virtual Cloud
$149.00
Cloud
per month
Entry-level set up fee?
- Setup fee optional
For the latest information on pricing, visithttps://saucelabs.com/pricing
Offerings
- Free Trial
- Free/Freemium Version
- Premium Consulting/Integration Services
Product Demos
WebdriverIO + Appium + Sauce Labs = Success? - Live Stream
YouTube
Automated Mobile Testing with Sauce Labs
YouTube
Parallel Testing with Sauce Labs
YouTube
Cross Browser Testing with Selenium, Sauce & Node.js
YouTube
Product Details
- About
- Integrations
- Competitors
- Tech Details
- Downloadables
- FAQs
What is Sauce Labs?
Sauce Labs is a cloud-based platform for automated testing of desktop and mobile applications. It is designed to be instantly scalable, since it is optimized for continuous integration workflows. (The vendor says that when tests are automated and run in parallel on multiple virtual machines across many different browser, platform and device combinations, testing time is reduced and developer time is freed up from managing infrastructure.) The Sauce Labs testing cloud is intended to be paired with a CI system. According to the vendor, this combination allows developers to easily test desktop and hybrid, native and mobile web applications early on in their development cycles, continuously and affordably.
Sauce Labs provides enterprise-grade security via Sauce Connect™, its secure tunneling protocol for testing behind a firewall while maintaining control of proxy and access policies. Tests are run in the company’s secure data center and VMs are destroyed (not “wiped”) after each run, ensuring customer data is never exposed to future sessions. After tests are completed the Sauce Labs’ dashboard provides a unique build-oriented report with metadata, access to Selenium logs, screenshots, video recordings, and a complete list of commands and responses. Support for SSO lets customers provision new user accounts on the fly with centralized user account management, access control, and usage reporting.
Sauce Labs Features
- Supported: Automated Cross-Browser Testing
- Supported: Automated Mobile Testing
- Supported: Manual Testing
- Supported: Real device cloud
- Supported: Integrations with all CI servers and JIRA
- Supported: Enterprise Security
- Supported: Proprietary Data Center
- Supported: Support for Selenium, Appium and JUnit Testing
- Supported: Professional Services & Onboarding
- Supported: Instructor-led Selenium and Appium training
Sauce Labs Screenshots
Sauce Labs Integrations
Sauce Labs Competitors
Sauce Labs Technical Details
| Deployment Types | Software as a Service (SaaS), Cloud, or Web-Based |
|---|---|
| Operating Systems | Unspecified |
| Mobile Application | No |
Sauce Labs Downloadables
- Sauce Labs Automated Testing Platform Datasheet
- Sauce Connect - Security in the Sauce Labs Cloud
- Sharecare Scales from Manual to Mobile with Sauce Labs - Case Study
- OKTA Increases Developer Productivity 80% with Sauce Labs - Case Study
- Campus Explorer Reduces Testing Time from 72 Hours to 72 Minutes with Sauce Labs - Case Study
Frequently Asked Questions
Reviewers rate Availability and Product Scalability highest, with a score of 10.
The most common users of Sauce Labs are from Enterprises (1,001+ employees) and the Computer Software industry.
Comparisons
Compare with
Reviews and Ratings
(285)
Community Insights
TrustRadius Insights are summaries of user sentiment data from TrustRadius reviews and, when necessary, 3rd-party data sources. Have feedback on this content? Let us know!
- Business Problems Solved
- Pros
- Cons
- Recommendations
Sauce Labs has proved to be a valuable tool for various teams and departments within organizations. Users appreciate the seamless integration of the platform with Jenkins CI servers, enabling continuous testing of web applications in real browsers and Dockerized environments. The ability to easily configure proxy tunnels and access firewalled environments and desired browsers via the Sauce OnDemand Jenkins plugin has been a major selling point.
One key use case for Sauce Labs is running regression test suites against different OS and browser combinations, which saves time and effort in maintaining test environments. Developer teams utilize Sauce Labs for running end-to-end Selenium tests, while the testing team manages its usage across the organization. Customer Service and Email Marketing teams also benefit from Sauce Labs, using it to identify and address customer issues and bugs before or after deployment. Additionally, the QA team relies on Sauce Labs for executing daily automation test cases on various platforms including mobile and web.
Best solution for deep automated browser testing: Users find Sauce Labs to be the best solution for integrating deep automated browser testing in a CI/CD pipeline, with multiple reviewers stating this as a key advantage of using the platform.
Ability to run test runs faster: Many users appreciate the ability to run test runs faster by using as many VMs as required, which is particularly helpful for CD/CI processes. This feature has been praised by multiple reviewers for its contribution to improving efficiency and reducing execution time.
Sauce Connect Proxy enables testing in firewalled environments: The Sauce Connect Proxy feature is highly valued by users as it allows companies to test in firewalled environments and localhost. Several reviewers have mentioned how this feature enables easy testing of applications behind a firewall, making it a valuable option for conducting tests securely.
Confusing and Difficult User Interface: Users have consistently found the user interface of the tool to be confusing and difficult to navigate, which has made it challenging for them to perform tasks efficiently. Several reviewers have expressed frustration with the complexity of the setup and configuration process, indicating a lack of user-friendliness.
Stability Issues: Some users have experienced stability issues with Sauce Labs, reporting that the tool is not reliable. These instances of instability can negatively impact the testing process for users, causing delays and uncertainties.
Limited Reporting and Customization Options: The lack of comprehensive reporting and customization options for dashboards is a drawback for users who require more detailed analytics and integration with APIs. This limitation has been mentioned by multiple reviewers, highlighting a need for improved functionality in this area.
Based on user reviews, users commonly recommend the following for Sauce Labs:
-
Request a trial and take advantage of fast support. Users highly recommend asking for a trial of Sauce Labs. They praise the company's fast support, which is beneficial in getting started with the tool.
-
Use Sauce Labs for multi-browser and multi-device automation. Sauce Labs is considered the best tool for cloud testing, particularly for testing web applications in different browsers, devices, and operating systems. Users mention its easy adoption and interface as strong points.
-
Consider test architecture and logging for effective automation testing. Users suggest putting at least 2 retries in the test configuration when performing parallel mobile testing with Sauce Labs. They also advise considering test architecture and test logging to enhance the automation testing process.
Overall, users highly recommend Sauce Labs as a top service provider in the market, especially for continuous automated testing and serious mobile or desktop testing. It is praised for its secure nature and integration with real devices on the cloud. However, users also suggest trying other services like BrowserStack for comparison purposes.
Attribute Ratings
Reviews
(1-5 of 5)Companies can't remove reviews or game the system. Here's why
August 03, 2022
Maintenance? What’s that? No such thing with Sauce Labs!
Allows testing across all phases of development by all members of a cross functional team. Easy to access and use by all with little knowledge of Sauce Labs via the UI. Solves: Maintenance costs of devices including upgrades, software updates, malfunctions. Device shortage via use of public devices. Device sharing and collaboration. Access to devices during pandemic.
- Range of devices are almost endless.
- Combination of OS, Browser and browser versions are more than satisfactory.
- Logs of every type at your fingertips.
- Updates to the core could be better communicated.
- Documentation can be outdated in certain places. Like broken links or now irrelevant.
- Chat function would be nice as support even if it's a bot.
- Eventually, our actual in-house devices would no longer be needed, where we’ve already stopped purchasing new devices.
- A bit extra investigation needed. Device usage metrics must be gathered both on the Saucelabs side as well as our users so we can get a clear understanding of what devices we need so that each time we refresh our private devices, we get the ones in high demand
Yes
We had Saucelabs replace TestObject. Simple reason is that Saucelabs acquired them and made them part of the Sauce platform. All the features were migrated onto the Sauce platform (took a while longer than I'd wanted to be honest) with the added ability to access Saucelabs' desktop browsers via their virtual machines. It was a no brainer really as the cost stayed very close to what we had with TestObject originally.
- Price
- Product Features
- Product Usability
- Product Reputation
For us, we came from TestObject, which was acquired by Saucelabs. Pricing staying very close to what we originally paid for TestObject alone, but with the added benefit of desktop browsers.
Saucelabs is very well known in the industry, so with a community backing and great documentation, it was worthwhile making the jump.
As mentioned before in the previous parts, features are constantly being added to Saucelabs and they always have a habit of letting you know when they are available (via their UI), which is great!
Saucelabs is very well known in the industry, so with a community backing and great documentation, it was worthwhile making the jump.
As mentioned before in the previous parts, features are constantly being added to Saucelabs and they always have a habit of letting you know when they are available (via their UI), which is great!
At the time, we tried and tested many of competitors. But given that most of the competitors had less features with a higher cost, it was a no brainer for us to migrate from TestObject to Saucelabs since there was no added learning curve (due to the acquisition) and not much of a change in cost.
30
Cross functional development teams which consist of a mixture of QAs, backend developer and frontend developers. As you can imagine, Saucelabs is more utilised by QA, Frontend developers and backend developers, in that order. External of the teams like product owners or directors would make more use of the video features, like for example, sharing a test run which contains gameplay and/or tests running on a browser.
3
We have a small QA only team which supports with all things QA across the business. This team drives the direction of QA across all departments and provide not only Saucelabs support but support of our in house QA frameworks and technology. Saucelabs is fairly straightfoward to use, especially for manual QAs, so support is usually for automation and/or admin.
- to replace real devices - no maintenance, purchasing, selling required
- the ability to visually show test runs to stakeholders and non technicals
- the ability to easily debug failed test runs
- monitoring systems which spawn up a browser/device to ensure uptime is as expected
- Not known as of yet. More features added in the future by Saucelabs may change this
- Implemented in-house
No
Change management was minimal
No. Did not need it since we are technical ourselves and are able to debug most of the issues we come across.
No
Probably a good thing, but haven't needed urgent support.
The times that we did raise a few things, the response time is fairly fast (within 24 hours). But then again, it was never for anything major.
The times that we did raise a few things, the response time is fairly fast (within 24 hours). But then again, it was never for anything major.
- Launching a browser/device manually or through code
- Reviewing test runs through video and Webdriver comms list
- Sharing a session through the UI via a link
- USB remote debugging isn't straightforward
- device usage metrics is non existent or I can't find it
- Individual level usage is fine. But would like to see usage at a team level. i.e. ability to generate a team access key
Yes, but I don't use it
Since we were the original users of TestObject, of which they acquired, we were able to migrate our devices over from when we had a contract with TestObject as well as gain usage of Saucelabs' VMs for a very little increase in price. Turns out we use VMs more than we thought we would.
Just lay out your requirements as a business. Explain what you'd need devices/VMs for and at what scale and the Saucelabs rep would be able to recommend what's best for your in terms of browser sessions and devices that you'd need.
No
- The two VM and real devices side being viewable on one platform (Unified platform)
- Easier navigation and browsing through both real devices and VM browser runs
No
Yes
Not that it was a different edition, but it was their unified platform that integrated together their own VM browser service with their acquisition of TestObject (real devices). This took a while to fully roll out as we were dependent on certain APIs that were originally only on TestObject, but they eventually got there.
July 11, 2022
Get Saucy!
We are using it across many teams for test automation across multiple browsers. It makes our lives much easier as we do not have to maintain multiple browsers, drivers, and platforms for testing.
- SauceConnect is great to allow tunnelling to test systems that are behind a firewall. There are many great features and they are adding more all the time.
- The fact that they maintain all WebDrivers and every version of browsers in invaluable. If we have a need for a specific platform, they have it.
- They have been having increased unexpected downtime lately. This is interruptive at times.
- Cut down on environment setup immensely
- Reduce the need for manual testing, which is expensive and slower
Sauce Labs' platform seemed to be the most flexible and I really haven't used much else. Haven't had a need to.
2
Software QA
2
development/devOps/QA
- cross-browser testing
- user flow validations
- automating webdriver maintenance
- Nothing yet
- Device emulation
No
- Price
- Product Features
- Product Usability
- Existing Relationship with the Vendor
Ease of use
Evaluate a few more
- Implemented in-house
Yes
POC, single test, then suite, then adding all tests
Change management was a big part of the implementation and was well-handled
Sauce concepts will help drive us to have more automation
- Nothing yet!
- no training
It was depending on your level of experience. The resources on Saucelabs' website and the Internet were sufficient for me. However, I'm a senior resource and testing manager, so your experience may vary!
Definitely follow their documentation on the website. Between the setup docs and the forums, you can find anything you need! Their support is also top-notch and is very responsive.
No - we have not done any custom code
Not Sure
No
They provide personalized support - they want to know YOUR use case and setup and work within it. This type of support is not common.
- The UI for representing test results is great
- Integration with CI tools is tight and seamless
- Nothing with the product itself - just learning the concepts
Yes
May 10, 2019
A Sauce Review
Helps us with our user acceptance tests for multiple browser analysis.
- spin up environments
- Sauce connect! For running on my local machine
- review archives of runs across environments
- concurrent sessions
- logs can be more descriptive
- tests run significantly slower when pointed at SL
NA
Not Sure
- Product Usability
Ease to write up automation and train new engineers
Perform interviews with other teams about their processes. I would also seek to demo competitive solutions.
10
Customer facing software to validate user workflows and acceptance.
5
Developers, QA, and architect engineers.
- Nightly regression runs.
- Validating defects do not return.
- Confirming new feature user flows.
- to assist with manual testing by spinning up a selected browser and starting point.
- Increasing boundary test limits such as capturing 1000 scenarios versus 10.
- Finding inconsistencies in JS between browsers.
- Video capture to show new hires the overhead view of functionality.
- Showing stakeholders the run count for confidence.
- Jr engineers can write automation before diving into feature development.
- Implemented in-house
No
Change management was minimal
NA
- Transfer of Knowledge.
- Increasing learning curve.
- Implementing with CI.
NA
- Integrating with Jenkins
- Integrating with Testrail
- Running automation.
- Adding a new framework.
- Sauce Connect
- Determining if failures are a system issue or flaky test.
We use Sauce Labs when we're running our regression test suite against a multitude of OS and browser combinations for our core web product. We are able to easily spin up virtual machines that contain a specific OS along with either the latest, beta or older version of the browsers we support with our app. This saves us a lot of maintenance and support issues with our test environments.
- Greatest strength of Sauce Labs to me is the ease of spinning up a clean virtual machine with the desired OS and browser combination.
- Sauce Labs support has been great! When I've run into issues whether it's from my own code issue or something on their side, they have been very responsive and helpful.
- The ability to scale up or down our needs from Sauce Labs has been great. As we build out more automation, we have a larger need of Sauce Labs services and it's been so simple to do. Need more parallel VMs? No problem. Need more automation time each month? No problem. It's all done almost instantly.
- My main area of concern that could be improved is the cost of the real mobile device farm offered. I have tried it out and it's great! I want to use it for most of my mobile automation, but can't justify the cost.....yet. Once more and more tests are created on my side along with a need to cover more devices then maybe it will work.
- A minor area that could be improved would be the responsiveness of the video stream of active tests. Sometimes I need to refresh the browser to get the video to come on or update.
- Another area of improvement would be the speed that tests can run. This has improved since I first started using Sauce Labs, but don't we all want everything, right now, immediately? I'm asking for the world, but you asked!
I evaluated Browser Stack at the same time I tried Sauce Labs and the ease of use of Sauce Labs made it an instant winner to me. I found Browser Stack (in early 2015) to be a little cumbersome to use and navigation wasn't always very intuitive to me where Sauce Labs excelled in these areas.
No
- Price
- Product Features
- Product Usability
Product Usability was the clear cut reason why we chose Sauce Labs. It came down to Sauce Labs and Browser Stack and Sauce Labs clearly had the edge in usability. This was back in early 2015, so things may have changed, but Sauce Labs has always been easy and intuitive to use.
I don't think I would have changed much of anything. I tried out Sauce Labs and Browser Stack and attempted to do the same type of automation on both. Sauce Labs was clearly easier to use and I believe their price was even better than Browser Stack, so it was a no brainer.
3
The three of us represent the automation team for our core web product and our mobile application that supports it.
3
All three of us who use Sauce Labs equally support for each other and any other team members who may want to use it on occasion. While it doesn't take extra technical skills to use Sauce Labs, it's good to have an understanding the concept of firewalls and how to pass in desired capabilities for Selenium Webdriver. If your test environment is behind a firewall then you will need to make sure you are running Sauce Labs secured tunnel before starting, so your tests will be able to access your environment. Knowing the various desired capabilities will also help with running the correct configuration on Sauce Labs, too.
- Number one use case is being able to test on many different OS and browser combinations on a clean machine. Even if you're able to spin up multiple virtual machines with your choice of configuration, it still takes quite a bit of time to create and maintain them.
- Automatic reporting of testing. Running automation with Sauce Labs keeps all of your tests in one place with easy filtering to get quick access to test results.
- Running on mobile emulators and simulators. You will not always have every device supported to test on, so you will need to rely on emulators and simulators for some of your testing. Like not having to spin up virtual machines for every desktop OS and browser combo, you don't need to create every emulator or simulator either. Run directly on Sauce Labs!
- Using Sauce Labs has increased our test coverage which is the biggest win for us. Previously, we were limited by resources to create all the environments, but now we can test across many different types of configurations. This has been a huge boost to our testing!
- Running tests in parallel! Before, we mostly ran our tests one by one which took a long time especially as we developed more and more scripts. Being able to seamlessly run many tests in parallel has greatly sped up testing and keeping our coverage at the same time.
- The use of mobile emulators and simulators. Before we were limited by what devices we had on hand and only using a browser's emulator for our testing which is very limited and doesn't cover many features of a mobile device. Using Sauce Labs for emulator and simulator testing has helped us with greater coverage than we had before.
- Sauce Labs real device farm! I've only test driven their real device farm, but it worked great! I was able to run my automation on native android and ios apps on real devices that we didn't have on hand. How can you beat that?
- The plan is to run many tests in parallel, so I'm expecting to need many more virtual machines to run at the same time.
- Real device farm: I've tried it out and now I want to use it on a regular basis. As more mobile scripts are developed, there will be a need at some point to move away from managing our own inventory and use Sauce Labs real device farm.
- Implemented in-house
No
- We were lucky. I knew Sauce Labs before starting at a new job that had no automation in place, so I was able to convince management to get Sauce Labs at the very start of building out automation.
No
The support we receive with the subscription we have has been more than adequate.
Yes
Yes. They have always been very responsive to support issues and just questions. I can't say enough good things about how great they've been in supporting us.
When I was first trying to use Appium for mobile testing, I struggled quite a bit getting it up and running. I was attempting to use Sauce Labs, as well, while trying to get going. I ran into many issues (self made), but many times I didn't know if it was on my side or Sauce Labs, so there were a number of times that I sent them my problem. Not once did they kick it back to me saying it was on my side and not theirs. They always helped and would even escalate to a senior member to offer assistance with Appium to help out. Like I've said before, I can't say enough great things about the support I've received at Sauce Labs. They truly make your success their success.
- The ease of spinning up a new virtual machine with your choice of OS and browser
- Being able to retrieve and review test results
- Adding team members to their own sub account to separate testing
- While spinning up a new virtual machine is easy, it can be tricky if you're not familiar with the need of creating a secured tunnel to access a test environment behind a firewall. Easy after you realize it though.
- Getting live video to update while a test is running can be cumbersome. I've had to refresh the browser sometimes to get the video to load.
- I wish tests would run faster especially with mobile emulators and simulators. I hate to put this down because I know how difficult this must be, but if I could magically make something better it would be the speed of tests.
As we began architecting a CI/CD pipeline in our SDLC, a need quickly arose for left-shifted, continuous testing of our web applications in real browsers. Specifically, a service that could integrate seamlessly with our Jenkins CI servers and executing tests continuously against our Dockerized applications and firewalled environments. Furthermore, our company does not have the resources to implement and maintain a proprietary Selenium Grid for test distribution. Sauce Labs addresses all of these concerns, and furthermore offers a platform for manually debugging applications in any combination of platforms, browsers, and devices needed, either from an automated test script or from a manual test session. The largest selling point for our needs is its relatively seamless integration with our Jenkins CI servers, including detailed playback and test reports per build job, as well as easy configuration of the proxy tunnel used to access our firewalled environments and desired browsers-under-test, right from the Jenkins UI via their Sauce OnDemand Jenkins plugin.
- Sauce Labs offers the single best solution for integrating deep automated browser testing in a CI/CD pipeline, and integrates best with Jenkins over their competition.
- The Sauce Connect Proxy is a necessary technology for companies who have firewalled testing environments, as well as localhost testing.
- The Jenkins UI plugin offers a way to easily configure browser settings for tests being executed continuously.
- Browser support is quick to be updated as new versions of browsers are released.
- The custom capabilities that can be provided to Sauce Labs VMs during automated testing sessions are a valuable option for experimental or niche testing.
- The Jenkins Sauce OnDemand plugin could have more options available to tap into more of the custom capabilities Sauce Labs actually supports. Currently, in order to tap into these capabilities, a programatic solution is required from test code, rather than being able to configure them via the Jenkins plugin UI. For example, things like desktop resolution, browser automation binary versions, remote Selenium version, and browser console logs.
- Some issues with the Sauce Connect Proxy on our Jenkins servers failing to start, which makes the optimal CI/CD pipeline come to a halt. Better documentation, logging, and more intuitive configuration around the Sauce Connect Proxy would take what is already an industry leading feature to the next level.
- User account management needs an overhauls, allowing for user groups, rather than just a hierarchy structure.
- Test job queuing is first-come-first-serve, meaning there is no in-built way to queue batches of jobs from, say, different Jenkins servers/accounts.
BrowserStack captures more market share than Sauce Labs does as of this writing, and thus has a much larger support network. During testing, it was also a little faster to spin-up virtual machines, and substantially quicker spinning-up mobile simulators and emulators. Pricing models were very similar in the enterprise tier. However, while BrowserStack may have been a better choice for manual testing or manually-executed automated tests, Sauce Labs offered a better product and better vision for tests that were left-shifted in a continuous delivery pipeline, which is the architectural direction or company has gone. Their service was proven to better integrate with this pipeline than BrowserStack.
No
- Price
- Product Features
- Product Usability
- Positive Sales Experience with the Vendor
- Third-party Reviews
The features of Sauce Labs over its primary competition are ultimately what drove the purchase decision. Sauce Labs offers a far-better solution for integrating deep user interface testing into a continuous delivery pipeline, that takes care to allow for testing of firewalled environments. The Jenkins support, specifically, informed our decision quite a bit, since we use Jenkins CI servers for our CI/CD model.
Reaching out to Sauce Labs representatives to get a timeline estimate on development of new features might have been an addition to the process we considered. Furthermore, a lengthier trial implementation to better determine some of the shortcoming of features we liked might have been beneficial, such as relatively restrictive user account management.
30
Software Architects, Software Development Engineers, Business Analysts, and Software Test Engineers all leverage the Sauce Labs service in some way.
2
Most support required for Sauce Labs involves coding test automation solutions that interface with Sauce Labs and Jenkins in the ways that are expected of our CI/CD model, and evolve along with it. There are also the administrative tasks surround user account, including creation of new users in the Sauce Labs service, as well as managing their privileges.
- Functional and Regression testing of our web applications via the user interface
- A VM grid supporting all necessary platforms, devices, and browsers that tests can be executed in from our CI servers that is maintenance-free
- Ability to debug browsers, platforms, or devices that are not readily available in-office
- Integration with our continuous delivery pipeline
- Detailed test reports and playback
- The ability to test the responsiveness of our user interfaces against smaller resolutions (either desktop or mobile)
- The ability to test multiple independent browser sessions within the execution of a single test, such as for testing session independence for Single Sign-On cases, or for testing concurrent usage of an application across multiple browser sessions
- The ability to execute automated or manual tests in Sauce Labs against localhost
- Continuous and left-shifted performance testing of the User Experience
- Real Device testing, rather than mobile simulators or emulators
- More extensive mobile testing
- Implemented in-house
No
Change management was minimal
N/A
- Ensuring that implemented automated test scrips could interface with Sauce Labs from either a local workstation or a Jenkins CI server
- Determining a solution for test job queuing. Sauce Labs does not support batch job queuing, but rather test jobs are queuing by first-come-first-server. What this means is that if your organization has multiple CI servers, each with their own Sauce Labs account and each wanting to execute a batch of test jobs, your organization must internally implement a solution to ensure that those concurrent batches of tests do not collide with each other.
- Self-taught
Product is extremely easy to learn on one's own, and requires little technical knowledge unless you are attempting to interface with the service from an in-house test automation framework. The documentation is well done, and updated with an acceptable frequency. Furthermore, they have a knowledge base and community that can be used to research more niche issues or questions.
Ensure that automated test source code and leverage all of Sauce Labs features from either a local workstation or a CI server, and that (if applicable) the Jenkins Sauce OnDemand plugin's configurations take priority over source code configurations, unless source code configs explicitly override the Jenkins plugin configs. This is the best way to ensure that specific test cases can have specific configurations without blanketing them with a configuration for the entire suite, as well as ensure that capabilities Sauce supports that aren't reflected in the Jenkins plugin can be leveraged.
No - there is no facility to customize the interface
No - the product does not support adding custom code
An internal solution needs to be implemented that organized batches of automated tests from multiple CI servers, such that if batches are executing concurrently, their individual tests do not collide with one another in the Sauce Labs service and unexpectedly subtract from the available concurrency pool. Sauce Labs does not support such batch queuing natively.
Yes
Premium support comes with entry into the enterprise license tier, which our company is a part of for legal reasons.
No
Our trial period before we decided to purchase Sauce Labs was extensive. At this time, we were still entertaining the idea of their main competitor as well. We mostly needed to vet what service would best work with our continuous delivery pipeline, and we needed time to prove this out. The Sauce Labs representative we were working with at the time was more than helpful and went above and beyond to ensure that our trial period was extended to as long as we needed to make an informed decision.
- Manual test sessions are very easy to spin up, and customize your platform, device, and browser and its version
- Test playback and metadata viewing is incredibly intuitive, for both manual and automated testing sessions
- Sometimes the Sauce Connect Proxy on a Jenkins server can be inexplicable flaky and fail to start during a build job. When this occurs, debugging the cause is difficult, and it breaks the intention of the CI/CD model.
- To execute a test in Sauce Labs from a local workstation against an application/environment that is firewalled requires starting the Sauce Connect Proxy utility on the local machine. This is a command line utility that may be unwieldy for non-technical people.
- The Jenkins Sauce OnDemand plugin is, as of this writing, relatively restrictive in what capabilities it can set for test execution. If your automated tests wish to tap into more granular capabilities that Sauce Labs supports (such as desktop resolutions, Selenium version in remote VM, automated driver binary version, etc), then a programatic solution is required, as these capabilities cannot be configured from the Jenkins plugin user interface.
Yes, but I don't use it
N/A
- API (e.g. SOAP or REST)
N/A
N/A
Know that the enterprise pricing is flexible and can be negotiated.
No
- N/A, cloud-based
- Hopefully more configurations for user account administration
- More features implemented for the Jenkins Sauce OnDemand plugin
- More fleshed-out Sauce Connect Proxy utility
No
No