Overall Satisfaction with Sauce Labs
We have used Sauce Labs on our CI server to run our JavaScript unit tests and Selenium functional tests. For unit tests, this ensured that our code will run without issues on any browser, for functional tests we were able to check UI in multiple environments easily including browsers not available on developer machines.
- Can be easily configured to run tests in parallel which makes testing many environments fast.
- Lots of environments available including phones, tablets, different operating systems etc. This help spot environment specific bugs which would be hard to find otherwise.
- Can be easily integrated with most of the major languages/frameworks.
- Running tests on Sauce Labs is not as stable as running them locally. Sometimes tests will fail due to networking issues which then produces false negative results and breaking CI builds.
- Running tests on Sauce Labs seems to be a bit slower than on a local machine. Although it is a bad practice, it is often that teams use arbitrary timeouts in their tests and it is not always trivial to update all tests to follow best standards. That makes tests passing locally and failing on Sauce Labs. Increasing those timeouts help but it slows them down.
- Some of the less known frameworks such as Intern.js might be harder to configure with Sauce Labs as there are fewer docs/examples available for them.