Thoughts on using Sauce Labs LabsUnspecified8.1209101
Bartosz Kaczorek profile photo
January 22, 2018

Thoughts on using Sauce Labs

Score 7 out of 101
Vetted Review
Verified User
Review Source

Overall Satisfaction with Sauce Labs

We have used Sauce Labs on our CI server to run our JavaScript unit tests and Selenium functional tests. For unit tests, this ensured that our code will run without issues on any browser, for functional tests we were able to check UI in multiple environments easily including browsers not available on developer machines.
  • Can be easily configured to run tests in parallel which makes testing many environments fast.
  • Lots of environments available including phones, tablets, different operating systems etc. This help spot environment specific bugs which would be hard to find otherwise.
  • Can be easily integrated with most of the major languages/frameworks.
  • Running tests on Sauce Labs is not as stable as running them locally. Sometimes tests will fail due to networking issues which then produces false negative results and breaking CI builds.
  • Running tests on Sauce Labs seems to be a bit slower than on a local machine. Although it is a bad practice, it is often that teams use arbitrary timeouts in their tests and it is not always trivial to update all tests to follow best standards. That makes tests passing locally and failing on Sauce Labs. Increasing those timeouts help but it slows them down.
  • Some of the less known frameworks such as Intern.js might be harder to configure with Sauce Labs as there are fewer docs/examples available for them.
  • Improve quality by increasing browser/OS/device coverage
  • Achieve continuous delivery
We were able to integrate Sauce Labs on our CI server and also enabled developers possibility to run tests on Sauce Labs directly from their machines. We have cut down the suite of tests running on Sauce Labs to only core features to keep them stable and by using Sauce Labs we were able to cover many environments which helped us find more bugs.
We had no issues around security with Sauce Labs.
Testing our application features in multiple environments using Sauce Labs helped us ensure that we will deliver high quality experience for our customers.
Sauce Labs is well suited for a set of well-defined UI tests where it is important to test application in many environments. The test suite should be kept small so that results can be obtained fast and they must be well written (i.e. not using arbitrary timeouts) to avoid false failures. I would not recommend it for testing lots of minor features as they will be brittle and not worth maintenance cost. I would also not recommend it for unit tests, as for them multiple environments are not that important and running them locally will be faster & more stable.