Likelihood to Recommend When you have to test the UI and how it behaves when certain actions are performed, you need something that can automate the browsers. This is where Selenium comes to the rescue. If you have to test APIs and not the frontend (UI), I would recommend going with other libraries that support HTTP Requests. Selenium is good only when you have no choice but to run the steps on a browser.
Read full review It is best suited for implementing the automated test cases in a human readable form so it's easy for non-technical members of the team and stakeholders to understand the test cases, features and the functionalities of the application. Automation of Integration tests and End to End tests are good use case. It is less appropriate or situations where the focus is only on the writing and maintenance of unit tests.
Read full review Pros For any web based UI automation, Selenium is the best tool out there to automate your tests. It supports multiple coding languages like Java, Python, Ruby, C# etc.. to choose from. There is a huge community of users and can get many answers on StackOverFlow. It has lot of other plugins to make your tests even more efficient. Read full review Versatility to be used in combination with different kinds of automated testing like automated performance testing, API testing, UI testing etc. I use JavaScript, Selenium, C#, email testing libraries, database testing libraries in combination with BDD with SpecFlow. I am able to use all these with SpecFlow to make my automation framework to be able to automate any kind of automated testing. It provides different widely used runner options like NUnit, XUnit etc. Before I started to work on establishing proper test automation in my workplace, the previous automation framework (non-BDD based) as well as unit tests used NUnit runner. The transition to using BDD was smooth because we could use the same runner and there were no compatibility issues. The auto-complete feature is good. I use it with Visual Studio as well as Rider and I don't have to recall the entire Gherkin statements. I just type a few words and the entire Gherkin statement implemented in framework is auto-suggested by SpecFlow. It saves time and context switching. Read full review Cons Selenium is pretty user-friendly but sometimes tests tend to flake out. I'd say roughly one out of twenty tests yields a false positive. Selenium software cannot read images. This is a minor negative because a free plug-in is available from alternate sources. Slowness may be a minor factor with Selenium, though this is an issue with basically any testing software since waiting on a site to execute JavaScript requires the browser to wait for a particular action. Read full review SpecFlow does not accepts optional input variables in the methods defined during Gherkin statement implementation. Cucumber supports optional input variables in the methods defined during Gherkin statement implementation. The tests identified while using SpecFlow with NUnit removes all white spaces in the scenario names. It makes the tests less readable. If the white spaces are not auto-removed, it would be much better for readability as well as their actual identification in the repository. Read full review Likelihood to Renew We love this product mainly because of its high customization abilities and the ease of use. Moreover, its free and can be learned easily through online communities and videos. The tests are more consistent and reliable as compared to Manual tests. It has enabled us to test a large number of features all in one go, which would have impossible through manual tests. The reports generated at the end of the tests are really helpful for the QA and the development teams to get a fair view of the application.
Read full review Usability As I mentioned earlier, the reason I use Selenium is because there is a fairly widespread community of users, and user support services are at a good level. because the application is open source, it works on many platforms (Windows, Linux, IOS) without any problems. In addition, it gives us a lot of options for writing functional tests. For errors that we receive through the application, we can easily find the reasons for errors in the forums.
Read full review Support Rating Selenium does not have technical support available easily. You have to go through forums to get the information you need. However, there are excellent forums out there that make it easy to troubleshoot. The open-source flexibility makes it difficult to have dedicated support.
Read full review Implementation Rating We did everything we needed to use it. Now we can execute our tests on different operational systems and browsers running few tests simultaneously. We also implemented Appium framework to execute our tests on mobile devices, such as iPhones, iPads, Android phones and tablets. We use SauceLabs for our test execution and Jenkins for continuous integration.
Read full review Alternatives Considered At the time of adoption, there were not many other alternatives that were even close to being competitive when it comes to browser testing. As far as I know now to this day, there is still little competition to Selenium for what it does. Any other browser-based testing still utilises Selenium to interact with the browser.
Read full review SpecFlow is .Net based which supports C#. Behave is Python based. Cucumber is Java based.
Ghost Inspector is no-code based but provides very limited testing features. We wanted to implement BDD so we rued out using
Ghost Inspector . Most of the developers in my team are C# experts so it was decided for everyone's comfort to go for SpecFlow rather than Behave or Cucumber. It's import to have technical experts in the language of the automation framework because there are many situations where the solutions to the test automation needs are not straightforward and implementing those requires expertise in the related programming language.
Read full review Return on Investment There hasn’t been a downside to using it yet other than we’ve got to update the programs we create for each change. This has saved us hundreds of hours of manpower by allowing our automation engineer to rapid fire tests. We are able to screenshot and save entire sites before and after launch with a program the automation engineer created We can compare large volumes of data against data in excel docs with a program created using Selenium Read full review Everyone stays on the same page regarding the behavior of existing functionalities whether it be technical or non-technical individuals. So there is less need for multiple people to get involved which saves time and thus money. Reusing the same code through the implemented Gherkin statement saves test automation time and thus reduces cost. We combine SpecFlow with other opensource testing technologies to make our automation framework more versatile which further saves costs for us. Read full review ScreenShots