Katalon vs. Selenium vs. Tricentis qTest

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

When you have to test the UI and how it behaves when certain actions are performed, you need something that can automate the browsers. This is where Selenium comes to the rescue. If you have to test APIs and not the frontend (UI), I would recommend going with other libraries that support HTTP Requests. Selenium is good only when you have no choice but to run the steps on a browser.

Incentivized

Tricentis qTest integrates seamlessly with Jira, making it ideal for teams that manage user stories and defects in Jira while keeping test cases and execution in qTest. When paired with automation tools like tosca, Selenium, or WebdriverIO, qTest is excellent for aggregating both manual and automated test results in one place.

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• For any web based UI automation, Selenium is the best tool out there to automate your tests.
• It supports multiple coding languages like Java, Python, Ruby, C# etc.. to choose from.
• There is a huge community of users and can get many answers on StackOverFlow.
• It has lot of other plugins to make your tests even more efficient.

Incentivized

• As a fresher, when I started using qTest it was very handy and easy to understand.
• It helps us trace the test cases that are used to test the quality in a single location
• The main thing is its integration with JIRA as soon as we create a ticket we would be getting all the requirements in the qTest so it became easy for me

• Integration with Azure is painstakingly slow as it takes too much time to upload reports to Azure. When raised with the support team i was advised to contact Microsoft who promptly and correctly replied that its an issue with the katalon plugin.
• Non Responsive Customer Support. Several mails were replied late.
• The Desktop app uses too much memory when booting up and running tests even on a 16gb RAM.

• Selenium is pretty user-friendly but sometimes tests tend to flake out. I'd say roughly one out of twenty tests yields a false positive.
• Selenium software cannot read images. This is a minor negative because a free plug-in is available from alternate sources.
• Slowness may be a minor factor with Selenium, though this is an issue with basically any testing software since waiting on a site to execute JavaScript requires the browser to wait for a particular action.

Incentivized

• In requirements , we can't add multiple test cases at once, or search multiple cases at once, need to do one by one. Here actually qtest needs to improve.
• Linking cloud hosted qtest and on-premise TOSCA is very difficult especially when you are working with client system with security wall. It requires tunnelling software which is not recommended.

Incentivized

We love this product mainly because of its high customization abilities and the ease of use. Moreover, its free and can be learned easily through online communities and videos. The tests are more consistent and reliable as compared to Manual tests. It has enabled us to test a large number of features all in one go, which would have impossible through manual tests. The reports generated at the end of the tests are really helpful for the QA and the development teams to get a fair view of the application.

Incentivized

This has been a core QA management tool for our organization and integrates well with our other SDLC platforms (Azure DevOps/Jira/Katalon).

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

For those who are unfamiliar with coding, there is a bit of a learning curve. There is plenty of helpful documentation and resources but it can take a little time to get the software up and running. Once you get the hang of how Selenium works, and what it can do, you realize how many things you can use it for, and how many processes you can automate.

Incentivized

It successfully balances deep enterprise functionality with a clean and modern interface. One of its strongest points is the intuitive, folder based navigation, which makes organizing test case and cycles feel natural. Learning curve is relatively shallow for the new team members to become productive without extensive onboarding. Also the deep integration with JIRA is a significant boost to its usability.

Incentivized

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

The Selenium app has a pretty fat community of users. For the problems we are experiencing, we are primarily receiving support from these communities. In addition, there is widespread service support. Instant support is given to the problems we experience when we need Online support. We and our team are happy to provide this support, especially before important deployment processes

The actual answer is 0. I have never experienced worse support, whether personal or professional

Incentivized

We did everything we needed to use it. Now we can execute our tests on different operational systems and browsers running few tests simultaneously. We also implemented Appium framework to execute our tests on mobile devices, such as iPhones, iPads, Android phones and tablets. We use SauceLabs for our test execution and Jenkins for continuous integration.

Incentivized

Again, supporting documentation could have been a lot better

Incentivized

Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.

Incentivized

At the time of adoption, there were not many other alternatives that were even close to being competitive when it comes to browser testing. As far as I know now to this day, there is still little competition to Selenium for what it does. Any other browser-based testing still utilises Selenium to interact with the browser.

Incentivized

All of them offer formidable solutions in the test management realm, but each one caters to different niche and need. qTest distinguishes itself with its deep integration capabilities, especially with Agile and DevOps tools, enabling streamlined CI/CD process. Its modern, user-centric interface contrasts with ALM's more dated appearance and complex setup. While TestRail provides a clean user experience and caters to a broad spectrum of business, qTest's scalability, from SMBs to large enterprises, stands out. PractiTest's cloud-based solution is geared towards mid-sized companies, but qTest's flexibility, advanced analytics, and robust reporting grant teams actionable insights. qTest' approach to a more holistic test management closely aligning with modern software development practices

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized

• There hasn’t been a downside to using it yet other than we’ve got to update the programs we create for each change.
• This has saved us hundreds of hours of manpower by allowing our automation engineer to rapid fire tests.
• We are able to screenshot and save entire sites before and after launch with a program the automation engineer created
• We can compare large volumes of data against data in excel docs with a program created using Selenium

Incentivized

• In comparison to other Testing tools, Tricentis qTest was the most expensive.
• We were able to quickly bring international users up to speed on how to use the tool.
• The ability to clone a project rather than create a new one from scratch was a valuable time saver.

Incentivized