Item: Sauce Labs
Rating: 5
Author: Verified User

Overall Satisfaction with Sauce Labs

Use Cases and Deployment Scope

Sauce Labs is used by my organization to automate testing with certain CI/CD pipelines. It is used across several teams in the organization, but isn't terribly widespread.

My team used Sauce Labs as our CI/CD pipeline went through Bamboo, which did not have any browsers on their servers. Since our UI tests required a browser, we used Sauce Labs to spin up the browsers we needed in a situation where we otherwise didn't have access to browsers.

Pros and Cons

Pros

Allows testing on browsers where your setup normally couldn't.
Once it is setup, it is incredibly easy to add additional browsers to test on as well as different versions of browsers.
Sauce Labs captures your tests in screenshots, video, and commands, which makes it very handy for tracking down exactly where a test messed up [you can see where it happened in real-time!]

Cons

Sauce Labs is incredibly difficult to set up and start using. Be prepared to browse forums, contact support, and do a lot of trial and error.
Sauce Labs doesn't tell you by default if a test passes/fails, only if it completes or doesn't complete. This means more setup time to get Sauce Lab working with a test reporter.
Since you run tests over a connection, running tests on Sauce Labs is incredibly finicky. Sometimes a test will fail just because it wants to; I've literally had tests that were working for days suddenly fail despite no code being changed to change them, and tests that were failing just start passing for seemingly no reason.

Other Software Used

Protractor, Bamboo, Bitbucket

Likelihood to Recommend

Sauce Labs is incredibly well-suited to a particular scenario - being able to run tests in the browser when you couldn't otherwise do so. It also makes running tests across multiple browsers and versions of browsers very easy, which could be quite valuable if your application prioritizes compatibility.

Even so, it's hard to recommend Sauce Labs if you have any kind of alternative solution. It's far more difficult to set up than it should be, and even when it is set up it seems rather hit-or-miss whether it will accurately run your tests or not. I've spent far too much time trying to track down issues that aren't there just because a test 'failed' on Sauce Labs.

Use it if you need to, avoid it if you don't.

Comments

Please log in to join the conversation

Sauce Labs? Situationally Lovely