- Customer communication - My account team and support are always responsive and helpful!
- Staying up to date including day one support for some things.
- Receiving feedback. I've had several meetings with Product to discuss my pain points, future wants, etc.
- Sometimes things can be finicky
- Connection speeds haven't always been great
- Greatest strength of Sauce Labs to me is the ease of spinning up a clean virtual machine with the desired OS and browser combination.
- Sauce Labs support has been great! When I've run into issues whether it's from my own code issue or something on their side, they have been very responsive and helpful.
- The ability to scale up or down our needs from Sauce Labs has been great. As we build out more automation, we have a larger need of Sauce Labs services and it's been so simple to do. Need more parallel VMs? No problem. Need more automation time each month? No problem. It's all done almost instantly.
- My main area of concern that could be improved is the cost of the real mobile device farm offered. I have tried it out and it's great! I want to use it for most of my mobile automation, but can't justify the cost.....yet. Once more and more tests are created on my side along with a need to cover more devices then maybe it will work.
- A minor area that could be improved would be the responsiveness of the video stream of active tests. Sometimes I need to refresh the browser to get the video to come on or update.
- Another area of improvement would be the speed that tests can run. This has improved since I first started using Sauce Labs, but don't we all want everything, right now, immediately? I'm asking for the world, but you asked!
- Always up and running: I'd say 99.9% it's up and running, as I've rarely experienced any down time with their VM farm machines.
- Reliable: we run our automated tests on their VMs at least every hour, around the clock, and their service is very reliable and working properly.
- Community: good online community of other people using Sauce Labs with tips/tricks; often see Sauce Labs at conferences and it's nice to talk with them about what they have planned (new features).
- Reporting: overall reporting over a time period used to be an issue, but there new feature to support this is getting there and improving.
- Logs: I don't find their logs very useful for front-end web tests, I usually refer to my own logs if I have a bug/issue/error that needs investigation.
- Cost: it's a little bit pricey, but I don't pay the bill so I'm not complaining too much there. Overall, I'd say the price isn't ridiculous but it's not inexpensive.
- Real device cloud is a much needed feature, and Sauce delivers.
- We are in the process of evaluating Sauce Labs for real device testing.
- The biggest strength of Sauce Labs is very easy to configure.
- Many time performance of the execution is really slow, especially when connected with Sauce Connect.
- need to support video playback on
- No real device cloud access for open source projects.
- Eliminate testing as a bottleneck
- Improve quality by increasing browser/OS/device coverage
- Move from waterfall to agile methodology
- Many versions (includes older versions) of popular browsers like (IE, Firefox, Chrome, Edge etc.) are available
- Great collection of real mobile devices available for testing
- Works well with CI tools like Jenkins, CircleCI etc.
- Has support for many programming languages like C#, Java, Python, Ruby etc.
- Identifying flakey tests, slow tests can prove to be really useful
- Live testing on VMs is slow i.e. there is a lag for the browser to perform actions after actually performing them
- The metrics provided are very basic. This could be improved
- They recorded the execution videos and you could play back and check the failure in time
- Their help desk and customer support team are very helpful, always ready to answer any questions/issues you have while using their product
- It's very easy to integrate Sauce Labs to almost very CI tools you are using
- If you are using Selenium as the main automation tool in your organization, their VMs might not support the latest Selenium3 code
- It's not that easy for debugging failures on Sauce Labs side, though all the test scripts are passing through our local machine
- Willow is generally compatible with the latest version of Appium, pretty fast
- We use Sauce Labs to run a test on multiple devices (simulators).
- We use Sauce Labs to execute automation scripts in different browsers in the cloud
- There is not much information on what is causing the issue but it works when I restart it. It would also be nice if the tests are a little faster since the time for the tests to complete is a bit slow
- The main point that I have actually dealt with was the cloud service is a bit sluggish. Often when I run it, it takes a substantially long period of time to finish executions. It is bit sluggish and took at some point to carry out action
- Reproduce user's bugs or problems in a certain platform, OS version, Browser, and version.
- Cross-browsing tests.
- Run in a remote browser our E2e tests, in an isolated context, not our CI content. Sauce helps us to improve our CI build duration and let us forget about Selenium grid maintenance.
- Test the same feature in multiple devices and browsers
- Test feature in older versions and even older OS versions are present which make it awesome.
- Save time to upgrade your real device. Emulators in Sauce Labs are readily available
- Test dashboard and capture manual execution video
- Less real time devices
- Some features are better to test in real device
- As an advocate for the unit and functional testing, I can safely say that Sauce Labs has been the reason we have been successful in catching several major and minor regression issues before they ever made their way into production.
- There is still some integration to go with Github and several other similar tools.. but Sauce is already appearing to be on that track. With all of the innovations they have made over the past year and a half, I am really looking forward to the changes they have planned for the future.
Sauce Labs Review: "Why we chose Saucelabs as our Cross-browser testing platform within our mission critical team."
- Multi-browser testing scenarios.
- Real mobile device testing
- Enterprise grade tool.
- Easy to run a Selenium test on a device. Add 3-4 capabilities, pass it into a driver instance, and boom. You are connected to Sauce Labs.
- Screenshots, videos, network logs, build monitoring, it's got a lot of tools that make it easy to debug a test and figure out what in the AUT is causing test failure.
- Uptime is high, we rarely have problems with their services.
- Mobile and tablet devices use emulators, which are extremely slow. The reason why they are slow is because they boot up a fresh VM on each test run. This adds minutes to a test, even if the test is 10s long. If there was a way whereby we could run a test on VM's which are already booted and ready to go, it will save immense amounts of time.
- Cookie view for a test: we manipulate areas of the cookies at times for debugging. It makes life easier if a snapshot of cookies with their key:values are stored in the Sauce Labs test run dashboard for viewing. This would aid us in saving much time.
- You can have a licence of 50 virtual machines and 5 accounts who share these 50 VMs. However, when you are logged into one of these accounts, and run i.e. 49 tests and then log in to another account, it still shows 50 VM's free when that is not the case. This causes other tests on other accounts to fail, which makes teams lose faith not only in Sauce Labs but in the test frameworks. I wish this area was more visible to all accounts and handled better in the console output.
- Mobile testing with Appium
- Recording of all test runs
- Adopting new technologies like XCUITest and Espresso allow for the use of industry standard tools in test automation
- Having access to browsers and devices in beta would be ideal
- They have been having increased unexpected downtime lately. This is interruptive at times.
- One thing that Sauce Labs does very well is continuing to provide education for technological products, so that we stay up to speed and are even able to learn new things that we never knew before. I definitely saw myself applying a lot of the new concepts that I learned, such as automated testing.
- I also love the Secret Sauce where we can learn about new product updates for Sauce Labs, as well as learn about upcoming events such as SauceCon. I love being able to stay abreast of a company that I love, as well as be able to learn of more interactive meetings where I can meet tech-minded people, as well as the Sauce company, i.e. Saucecon.
- I really love the customer service. Once in the Forum, I had a question, and Alyssa the VP actually gave me a reply, and was very quick to respond as well as very friendly. I've never had a company where the leadership was so responsive and so eager to dive in with customers on issues and questions that they have. This definitely shows that Sauce isn't just about getting profits, and that they truly deeply care about making the customers happy and successful.
- Lastly, I love how Sauce University has a stash which encourages friendly competition among team members. It shows who has the most points, which encourages us to continue to actively participate in those trainings and challenges and quizzes. This in turn adds an element of fun to our work.
- In Sauce University trainings, it would be nice we could receive more in-depth information, as opposed to a short paragraph of information. As a beginner in the technological field, I would definitely find that helpful. Maybe they could do something where you can create a link to show more details, so that more advanced readers don't have to open it, but beginners can actually open up that link to read more.
- Also, it would be nice if in the trainings you would include the videos as well because a lot of people are visual learners.
- I wish it were easier to search through the Sauce Labs help page. Sometimes it takes quite a few searches to get to the right result. For instance, I'm not sure if I should click on Knowledge Base or Documentation or Support- there's too much to click-through. There should just be a Search All button that would search across the whole Sauce Labs page, so that we can easily find the answer that we need.
- Our team especially loves the challenges, because again that fosters friendly competition and encourages us to continue to learn more about the product and the tech, as well as hone previous skills we've learned through Sauce Labs. We definitely hope that more of those classes come out every week, so we can dive deeper into each concept. I think there should be a feedback page where we can submit ideas for the classes that we would like to see.
It's also well suited for educational purposes, because you get to use Sauce University as well as attend any boot camps to continue to broaden your knowledge and become a stronger technological resource in the company.
- Infrastructure as a service - we do not need to worry about spinning up machines or VMs.
- Scalable based on demands of the company.
- View recordings of the automated tests.
- Live manual test support.
- Mobile device support.
- UI not the most responsive.
- SauceConnect tunnel sometimes fails for no reason.
- Comprehensive platform coverage including all the browsers and operating systems we were looking for. We did not test anything on mobile.
- It saved time on the regression test cycle time and enabled quicker releases to production.
- Video recordings of the test execution were very helpful to triage failures and resolve.
- Cost could be lower - some of the team members felt Sauce Labs was a little expensive but I do not have more details on that.
- The devices are really up to date. For example, the latest iPhone became part of the list almost as soon as it was released.
- Ability to tunnel to a different location so you can run tests on a specific region.
- Customer support is quite responsive.
- Testobject analytics - we've had a hard time ordering tests
- Test start time - sometimes takes a while to upload an app to a device and start test execution
- Price - it's a bit on the expensive side
- Wide and deep offerings of browser and OS combinations
- Quick support for new releases versions of browsers
- Includes mobile emulators
- Supports both manual and automated testing
- Recent stability issues and outages
- Lack of deep technical support for Windows-based Sauce Connect Tunnels
- The online dashboard provides basic data, but in-depth analysis and drill down is not possible. We run over 10,000 tests a day and can be difficult to find patterns on errors and failed tests
- Multiple browsers, Multiple OS available for running tests.
- Difficult to maintain these server farm in-house.
- Someone maintains the server farm for us and provides as SaaS offering.
- Tests take longer to run as compared to in-house. This makes overall tests suite completing slow, slowing our deployment time.
Least suited - where your test suite used to run for hours, it will now take days.
- Sauce Labs enables easy debugging by providing video for the test case, Selenium log and step by step command.
- Accessing Sauce Labs is easy, I mean good documentation
- Email notifications when in Sauce Labs have maintenance activity
- Customer support ticket should be responded faster
- Mostly every weekend has maintenance activity, if it is reduced it will be good
- spin up environments
- Sauce connect! For running on my local machine
- review archives of runs across environments
- concurrent sessions
- logs can be more descriptive
- tests run significantly slower when pointed at SL
The main business problems that it addresses is helping us to understand customers' problems or bugs and make a visual and functionality check before or after a deployment.
- Cross Browsing: It's a large list of OS, Browsers and Versions really updated.
- Really aaS: Sauce is really an as a service product, you just use it when you want with really a few clicks.
- Documentation rocks: They have a large knowledge base documented for all their customers.
- Analytics: The have a nice analytics service to understand you test trends.
- Allow CIs stateless: With Sauce Connect it's possible to get a CI-CD environment really stateless without needing to install and maintain a lot of tools and infrastructure.
- Speed: We've some speed issues with "Live Testing" it's a bit slow and in our agile environment is a little sad.
- Sauce and Testobject: It's wired to use 2 platforms when it's the same company. I wish it was all Saucelabs...It'd be more useful to explain to all the people that it's another service of the same company.
* You need to run your tests in a lot of browsers and versions. It's very useful to be used for cross-browsing and visual pixel perfect tests.
* You need to delegate the maintenance of the selenium infrastructure. It's awesome to forget about browsers or OS releases and all the problems involved with it.
SauceLabs doesn't suit if:
* You need a fast environment to run manual tests.
* You are looking for a fast UI.
- Applications behind a firewall can be easily tested using the sauce tunnel. This helps us not expose our applications to public for testing on a cloud platform
- Platform versions are frequently updated and can be configured easily using desired capabilities.
- Debugging becomes easy as you could watch a replay of the test or debug using command tool or even get the selenium logs.
- Integrate with QASymphony for test management.
- Use extended debugging tool at run time instead of uploading metadata for test run
- Increase test speed by reducing OS startup
- Sauce Labs offers the single best solution for integrating deep automated browser testing in a CI/CD pipeline, and integrates best with Jenkins over their competition.
- The Sauce Connect Proxy is a necessary technology for companies who have firewalled testing environments, as well as localhost testing.
- The Jenkins UI plugin offers a way to easily configure browser settings for tests being executed continuously.
- Browser support is quick to be updated as new versions of browsers are released.
- The custom capabilities that can be provided to Sauce Labs VMs during automated testing sessions are a valuable option for experimental or niche testing.
- The Jenkins Sauce OnDemand plugin could have more options available to tap into more of the custom capabilities Sauce Labs actually supports. Currently, in order to tap into these capabilities, a programatic solution is required from test code, rather than being able to configure them via the Jenkins plugin UI. For example, things like desktop resolution, browser automation binary versions, remote Selenium version, and browser console logs.
- Some issues with the Sauce Connect Proxy on our Jenkins servers failing to start, which makes the optimal CI/CD pipeline come to a halt. Better documentation, logging, and more intuitive configuration around the Sauce Connect Proxy would take what is already an industry leading feature to the next level.
- User account management needs an overhauls, allowing for user groups, rather than just a hierarchy structure.
- Test job queuing is first-come-first-serve, meaning there is no in-built way to queue batches of jobs from, say, different Jenkins servers/accounts.
Sauce Labs Review: "Great for seasoned software testers, but may be a bit overwhelming to those newer to running [automated] tests"
- The many possible desktop/mobile/tablet browser-platform combos
- More thorough than other competitive solutions as a means of testing cross-browser functionality
- Versatile in that it supports different languages for automated tests, from C#, PHP & Java to even JS (Node.js)
- Has an extensive wiki at https://wiki.saucelabs.com - very helpful!
- Can feel a bit overwhelming when first integrating in a project - that is to say, may seem a bit complex to implement and navigate at first, however, and in fairness, as a front-end software developer and web designer less seasoned than say a veteran software test engineer, a more experienced tester might not share the same experience
- Login credentials have been problematic (could be do to some SSO glitch on my end)
- User interface is somewhere between "good" and "mediocre", relying more on the assumed knowledge of the tester (could be intentional design choice)
May be overkill for: - smaller websites and apps
Sauce Labs Scorecard Summary
About Sauce Labs
Sauce Labs is a cloud-based platform for automated testing of desktop and mobile applications. It is designed to be instantly scalable, since it is optimized for continuous integration workflows. (The vendor says that when tests are automated and run in parallel on multiple virtual machines across many different browser, platform and device combinations, testing time is reduced and developer time is freed up from managing infrastructure.) The Sauce Labs testing cloud is intended to be paired with a CI system. According to the vendor, this combination allows developers to easily test desktop and hybrid, native and mobile web applications early on in their development cycles, continuously and affordably.
Sauce Labs provides enterprise-grade security via Sauce Connect™, its secure tunneling protocol for testing behind a firewall while maintaining control of proxy and access policies. Tests are run in the company’s secure data center and VMs are destroyed (not “wiped”) after each run, ensuring customer data is never exposed to future sessions. After tests are completed the Sauce Labs’ dashboard provides a unique build-oriented report with metadata, access to Selenium logs, screenshots, video recordings, and a complete list of commands and responses. Support for SSO lets customers provision new user accounts on the fly with centralized user account management, access control, and usage reporting.
Sauce Labs Screenshots
Sauce Labs Integrations
Sauce Labs Competitors
Free service available for Open Source projects.
Sauce Labs Support Options
|Free Version||Paid Version|
|Video Tutorials / Webinar|
Sauce Labs Technical Details