Simple and Scalable? How Sauce Labs fits into your automation roadmap!
May 03, 2017
Simple and Scalable? How Sauce Labs fits into your automation roadmap!
Score 10 out of 10
Vetted Review
Verified User
Overall Satisfaction with Sauce Labs
We've been successfully leveraging Sauce Labs, by our SDETs and DEVs to verify our end-to-end testing across the latest and "legacy" platforms across multiple organizations. Sauce Labs provides QA/DEV teams the ability to automate UI/UX with a focus on our automation and less on managing the infrastructure, e.g. Selenium Grid, while fitting in with our CI/CD methodologies. At times, the test execution can be "slow", however, the big-picture of executing in parallel and saved-time per resource management is a win.
In addition, our UI/UX teams require visual test coverage, not just "functional" - the seamless integration between SauceLabs/Selenium/Applitools' Eyes works really well for our team - since it requires very little effort to leverage visual testing in our existing automated tests!
In addition, our UI/UX teams require visual test coverage, not just "functional" - the seamless integration between SauceLabs/Selenium/Applitools' Eyes works really well for our team - since it requires very little effort to leverage visual testing in our existing automated tests!
- Documentation on using Sauce Labs has improved, specifically, the examples and excerpts for different programming languages are helpful and thoughtful.
- The Git repo (examples) have helped my team to quickly explore and experiment with the provided examples by Sauce Labs.
- The "Dashboard" has greatly improved .. helps us to manage and see how our teams are utilizing Sauce Labs and automation, including the device coverage that our automated tests are covering.
- Sauce Labs' plugin to Jenkins is improving, this helps our SDETs quickly and reliably add/update automated tests with less coding and boilerplate scripting!
- On occasion, Sauce Labs, was off-line due to unexpected outage (or planned outage). When our tests failed due to these outages, we weren't notified in a timely manner. For example, after 2 hours, we finally realized it wasn't our environment/tests that were failing, but Sauce Labs was down. We were reminded to visit the "status" page, however, that doesn't seem pro-active enough. It'll be great if "sauce-connect" provided immediate feedback of such circumstances.
- Selenium's success is tied directly with Sauce Labs, whereas "cross-browser" testing is one of the selling points to use Sauce. However, the differences and nuances of APIs per Selenium, with regards to, IE11, Edge, Firefox, and Chrome should be better highlighted in their documentation.
- For example, keyboard navigation and active-focus requires different approaches w.r.t browsers. This is especially important for testing scenarios with keyboard vs. mouse.
- Consistency in test execution.
- At times, without any changes to our automate testware, the execution results are not consistent, this places doubt on our existing automated tests. There should be a "performance-expectation" thermometer.
I've used Sauce Lab's competitors, including managing our own Selenium Grid during the past 5 years. Now, I can confidently share that Sauce Labs continues to ensure we are successful, while keeping their technology current. I appreciate that they're continuously promoting and leading the efforts to improve Selenium.