Sauce Labs? Situationally Lovely
Overall Satisfaction with Sauce Labs
Sauce Labs is used by my organization to automate testing with certain CI/CD pipelines. It is used across several teams in the organization, but isn't terribly widespread.
My team used Sauce Labs as our CI/CD pipeline went through Bamboo, which did not have any browsers on their servers. Since our UI tests required a browser, we used Sauce Labs to spin up the browsers we needed in a situation where we otherwise didn't have access to browsers.
- Allows testing on browsers where your setup normally couldn't.
- Once it is setup, it is incredibly easy to add additional browsers to test on as well as different versions of browsers.
- Sauce Labs captures your tests in screenshots, video, and commands, which makes it very handy for tracking down exactly where a test messed up [you can see where it happened in real-time!]
- Sauce Labs is incredibly difficult to set up and start using. Be prepared to browse forums, contact support, and do a lot of trial and error.
- Sauce Labs doesn't tell you by default if a test passes/fails, only if it completes or doesn't complete. This means more setup time to get Sauce Lab working with a test reporter.
- Since you run tests over a connection, running tests on Sauce Labs is incredibly finicky. Sometimes a test will fail just because it wants to; I've literally had tests that were working for days suddenly fail despite no code being changed to change them, and tests that were failing just start passing for seemingly no reason.
- Achieve continuous delivery
Using Sauce Labs, my team was able to achieve continuous delivery. Our tests have been automated so that whenever we try to deploy, we will know if something goes wrong. Given the situation my team was in, Sauce Labs was one of the few options we could turn to.
I know Sauce Labs has worked in several other teams across my organization as well. As far as I'm aware, it has helped them achieve a similar outcome.
Sauce Labs is incredibly well-suited to a particular scenario - being able to run tests in the browser when you couldn't otherwise do so. It also makes running tests across multiple browsers and versions of browsers very easy, which could be quite valuable if your application prioritizes compatibility.
Even so, it's hard to recommend Sauce Labs if you have any kind of alternative solution. It's far more difficult to set up than it should be, and even when it is set up it seems rather hit-or-miss whether it will accurately run your tests or not. I've spent far too much time trying to track down issues that aren't there just because a test 'failed' on Sauce Labs.
Use it if you need to, avoid it if you don't.