Get on the Sauce! Sauce Labs will make your automation life A LOT easier!
Updated July 25, 2018
Get on the Sauce! Sauce Labs will make your automation life A LOT easier!
Score 10 out of 10
Vetted Review
Verified User
Overall Satisfaction with Sauce Labs
We use Sauce Labs when we're running our regression test suite against a multitude of OS and browser combinations for our core web product. We are able to easily spin up virtual machines that contain a specific OS along with either the latest, beta or older version of the browsers we support with our app. This saves us a lot of maintenance and support issues with our test environments.
- Greatest strength of Sauce Labs to me is the ease of spinning up a clean virtual machine with the desired OS and browser combination.
- Sauce Labs support has been great! When I've run into issues whether it's from my own code issue or something on their side, they have been very responsive and helpful.
- The ability to scale up or down our needs from Sauce Labs has been great. As we build out more automation, we have a larger need of Sauce Labs services and it's been so simple to do. Need more parallel VMs? No problem. Need more automation time each month? No problem. It's all done almost instantly.
- My main area of concern that could be improved is the cost of the real mobile device farm offered. I have tried it out and it's great! I want to use it for most of my mobile automation, but can't justify the cost.....yet. Once more and more tests are created on my side along with a need to cover more devices then maybe it will work.
- A minor area that could be improved would be the responsiveness of the video stream of active tests. Sometimes I need to refresh the browser to get the video to come on or update.
- Another area of improvement would be the speed that tests can run. This has improved since I first started using Sauce Labs, but don't we all want everything, right now, immediately? I'm asking for the world, but you asked!
I evaluated Browser Stack at the same time I tried Sauce Labs and the ease of use of Sauce Labs made it an instant winner to me. I found Browser Stack (in early 2015) to be a little cumbersome to use and navigation wasn't always very intuitive to me where Sauce Labs excelled in these areas.
Evaluating Sauce Labs and Competitors
- Price
- Product Features
- Product Usability
Product Usability was the clear cut reason why we chose Sauce Labs. It came down to Sauce Labs and Browser Stack and Sauce Labs clearly had the edge in usability. This was back in early 2015, so things may have changed, but Sauce Labs has always been easy and intuitive to use.
I don't think I would have changed much of anything. I tried out Sauce Labs and Browser Stack and attempted to do the same type of automation on both. Sauce Labs was clearly easier to use and I believe their price was even better than Browser Stack, so it was a no brainer.
Using Sauce Labs
3 - The three of us represent the automation team for our core web product and our mobile application that supports it.
3 - All three of us who use Sauce Labs equally support for each other and any other team members who may want to use it on occasion. While it doesn't take extra technical skills to use Sauce Labs, it's good to have an understanding the concept of firewalls and how to pass in desired capabilities for Selenium Webdriver. If your test environment is behind a firewall then you will need to make sure you are running Sauce Labs secured tunnel before starting, so your tests will be able to access your environment. Knowing the various desired capabilities will also help with running the correct configuration on Sauce Labs, too.
- Number one use case is being able to test on many different OS and browser combinations on a clean machine. Even if you're able to spin up multiple virtual machines with your choice of configuration, it still takes quite a bit of time to create and maintain them.
- Automatic reporting of testing. Running automation with Sauce Labs keeps all of your tests in one place with easy filtering to get quick access to test results.
- Running on mobile emulators and simulators. You will not always have every device supported to test on, so you will need to rely on emulators and simulators for some of your testing. Like not having to spin up virtual machines for every desktop OS and browser combo, you don't need to create every emulator or simulator either. Run directly on Sauce Labs!
- Using Sauce Labs has increased our test coverage which is the biggest win for us. Previously, we were limited by resources to create all the environments, but now we can test across many different types of configurations. This has been a huge boost to our testing!
- Running tests in parallel! Before, we mostly ran our tests one by one which took a long time especially as we developed more and more scripts. Being able to seamlessly run many tests in parallel has greatly sped up testing and keeping our coverage at the same time.
- The use of mobile emulators and simulators. Before we were limited by what devices we had on hand and only using a browser's emulator for our testing which is very limited and doesn't cover many features of a mobile device. Using Sauce Labs for emulator and simulator testing has helped us with greater coverage than we had before.
- Sauce Labs real device farm! I've only test driven their real device farm, but it worked great! I was able to run my automation on native android and ios apps on real devices that we didn't have on hand. How can you beat that?
- The plan is to run many tests in parallel, so I'm expecting to need many more virtual machines to run at the same time.
- Real device farm: I've tried it out and now I want to use it on a regular basis. As more mobile scripts are developed, there will be a need at some point to move away from managing our own inventory and use Sauce Labs real device farm.
Sauce Labs Implementation
- Implemented in-house
- We were lucky. I knew Sauce Labs before starting at a new job that had no automation in place, so I was able to convince management to get Sauce Labs at the very start of building out automation.
Sauce Labs Support
| Pros | Cons |
|---|---|
Quick Resolution Good followup Knowledgeable team Problems get solved Kept well informed No escalation required Immediate help available Support understands my problem Support cares about my success Quick Initial Response | None |
No - The support we receive with the subscription we have has been more than adequate.
Yes - Yes. They have always been very responsive to support issues and just questions. I can't say enough good things about how great they've been in supporting us.
When I was first trying to use Appium for mobile testing, I struggled quite a bit getting it up and running. I was attempting to use Sauce Labs, as well, while trying to get going. I ran into many issues (self made), but many times I didn't know if it was on my side or Sauce Labs, so there were a number of times that I sent them my problem. Not once did they kick it back to me saying it was on my side and not theirs. They always helped and would even escalate to a senior member to offer assistance with Appium to help out. Like I've said before, I can't say enough great things about the support I've received at Sauce Labs. They truly make your success their success.
Using Sauce Labs
| Pros | Cons |
|---|---|
Like to use Relatively simple Easy to use Technical support not required Well integrated Consistent Convenient Feel confident using Familiar | None |
- The ease of spinning up a new virtual machine with your choice of OS and browser
- Being able to retrieve and review test results
- Adding team members to their own sub account to separate testing
- While spinning up a new virtual machine is easy, it can be tricky if you're not familiar with the need of creating a secured tunnel to access a test environment behind a firewall. Easy after you realize it though.
- Getting live video to update while a test is running can be cumbersome. I've had to refresh the browser sometimes to get the video to load.
- I wish tests would run faster especially with mobile emulators and simulators. I hate to put this down because I know how difficult this must be, but if I could magically make something better it would be the speed of tests.