Katalon vs. PractiTest vs. TestRail

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

PractiTest works GREAT as a test case repository. It is very easy to gather metrics, filter, and sort based on custom fields. We were able to work with the API to pull our automation results in as well. The support team is always very quick with their responses and monitors the "in-app chat." They are very open to answering questions, providing best practice materials, and looking for additional feedback. If you already have a central location for all of your test cases and testing needs, then I guess you probably wouldn't need to add another. However, PractiTest has high capability and potential, so if it's set up properly you can easily save time managing your tests.

Incentivized

The integration with other tools, and their user friendly layout and design. It's very comfortable to use and is way better than other tools with the Automation Testing tools, thanks to the great API that is included. Sometimes the integration with Jira is a little faulty, but the links to that tool usually work well. It could be cool if it had a better following for the bug items that were registered on other tools.

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• Effective Test case template.
• Can integrate with different Atlassian products.
• Report generation.

Incentivized

• Managing test plans and test cases
• Generation of test results
• Modularised way of storing tests

Incentivized

• Integration with Azure is painstakingly slow as it takes too much time to upload reports to Azure. When raised with the support team i was advised to contact Microsoft who promptly and correctly replied that its an issue with the katalon plugin.
• Non Responsive Customer Support. Several mails were replied late.
• The Desktop app uses too much memory when booting up and running tests even on a 16gb RAM.

• Linking the defect with Jira throws an error quite some times which can be improved in the future.

Incentivized

• Outstanding user bugs
• UI is clunky
• Things are not where you think they would be

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

As I am a tester, for me I found this tool to be new in terms of everything like the management of tests, plans, releases, reporting,etc. It is overall a good tool for test reporting and can be used for reference in the future. I liked several features of on the go placement of screenshots. Also I feel like the UI , the font, the color combination can be improved

Incentivized

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

The chat button is available to anyone who logs into PractiTest. In my experience, the support has always been very quick, very friendly, and very thorough. They make sure that your question is answered in a way that you understand it. They also provide documentation of best practices so you are never left hanging on what to do next.

Incentivized

Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.

Incentivized

I've used many different Test Case Repository tools, and while each of them has its perks, I like the capabilities of PractiTest best. When creating a test in qTest for example, you can only input information into the fields provided, and you have everything set up in a folder tree structure. With PractiTest, we are able to create custom fields and filter our tests based on those fields to provide more accurate information in a readily available format while quickly searching for the filter instead of through a folder tree. TestRail did not appear to meet our needs as a company. It just didn't have the potential that we found with PractiTest. Zephyr for example worked seamlessly with Jira, which is really nice since that is what we use for the most part. However since we cater to many different clients, we needed an external Test Case repository so we could use something that wasn't tied to 1 Jira instance.

Incentivized

TestRail definitely saves times. I work in a company that consists of several development teams, all of which have different processes. Some of the teams leverage test cases, some do not. I've noticed that the turn around time it takes for me to pick up a ticket, QA it, and then pass/fail/send feedback is much faster when there is a test case created as I'm not reading through ticket description/comments to figure out what needs to be QA'd.

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized

• Test Reporting
• Defect Management

Incentivized

• We are able to easily pull reports any time management would like to see them
• Jira integration makes it easy to attach to stories/defects and save time
• A quick overview at the system's health as a whole is easy to see from TestRail

Incentivized