Sauce Labs? Situationally Lovely
Anonymous | TrustRadius Reviewer
January 12, 2018

Sauce Labs? Situationally Lovely

Score 5 out of 10
Vetted Review
Verified User
Review Source

Overall Satisfaction with Sauce Labs

Sauce Labs is used by my organization to automate testing with certain CI/CD pipelines. It is used across several teams in the organization, but isn't terribly widespread.

My team used Sauce Labs as our CI/CD pipeline went through Bamboo, which did not have any browsers on their servers. Since our UI tests required a browser, we used Sauce Labs to spin up the browsers we needed in a situation where we otherwise didn't have access to browsers.

  • Allows testing on browsers where your setup normally couldn't.
  • Once it is setup, it is incredibly easy to add additional browsers to test on as well as different versions of browsers.
  • Sauce Labs captures your tests in screenshots, video, and commands, which makes it very handy for tracking down exactly where a test messed up [you can see where it happened in real-time!]
  • Sauce Labs is incredibly difficult to set up and start using. Be prepared to browse forums, contact support, and do a lot of trial and error.
  • Sauce Labs doesn't tell you by default if a test passes/fails, only if it completes or doesn't complete. This means more setup time to get Sauce Lab working with a test reporter.
  • Since you run tests over a connection, running tests on Sauce Labs is incredibly finicky. Sometimes a test will fail just because it wants to; I've literally had tests that were working for days suddenly fail despite no code being changed to change them, and tests that were failing just start passing for seemingly no reason.
  • Achieve continuous delivery

Using Sauce Labs, my team was able to achieve continuous delivery. Our tests have been automated so that whenever we try to deploy, we will know if something goes wrong. Given the situation my team was in, Sauce Labs was one of the few options we could turn to.

I know Sauce Labs has worked in several other teams across my organization as well. As far as I'm aware, it has helped them achieve a similar outcome.

We trust Sauce Labs to be compliant with our security and regulations. In my case, my team was using Sauce Labs for unit tests and UI tests, so the only information being sent was dummy information for testing purposes. There isn't much risk in sending such information to Sauce Labs, although your mileage may vary.
Sauce Labs helped us meet our customer experience requirements by making sure we had excellent code coverage. By being able to catch issues early, we were able to fix numerous issues before they hit production. The less bugs and issues that make it through, the better the experience for our customers.
My organization decided to buy instead of build because we use it for a particular purpose. Use of Sauce Labs isn't widespread at my organization; only a several teams use it and the organization is quite big. It was easier for the company to pay for the few times we'd need something like Sauce Labs than to devote a team (or multiple teams) to build our own version of it.

Sauce Labs is incredibly well-suited to a particular scenario - being able to run tests in the browser when you couldn't otherwise do so. It also makes running tests across multiple browsers and versions of browsers very easy, which could be quite valuable if your application prioritizes compatibility.

Even so, it's hard to recommend Sauce Labs if you have any kind of alternative solution. It's far more difficult to set up than it should be, and even when it is set up it seems rather hit-or-miss whether it will accurately run your tests or not. I've spent far too much time trying to track down issues that aren't there just because a test 'failed' on Sauce Labs.

Use it if you need to, avoid it if you don't.