Katalon vs. TestRail

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

The integration with other tools, and their user friendly layout and design. It's very comfortable to use and is way better than other tools with the Automation Testing tools, thanks to the great API that is included. Sometimes the integration with Jira is a little faulty, but the links to that tool usually work well. It could be cool if it had a better following for the bug items that were registered on other tools.

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• Managing test plans and test cases
• Generation of test results
• Modularised way of storing tests

Incentivized

• Integration with Azure is painstakingly slow as it takes too much time to upload reports to Azure. When raised with the support team i was advised to contact Microsoft who promptly and correctly replied that its an issue with the katalon plugin.
• Non Responsive Customer Support. Several mails were replied late.
• The Desktop app uses too much memory when booting up and running tests even on a 16gb RAM.

• Outstanding user bugs
• UI is clunky
• Things are not where you think they would be

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

As I am a tester, for me I found this tool to be new in terms of everything like the management of tests, plans, releases, reporting,etc. It is overall a good tool for test reporting and can be used for reference in the future. I liked several features of on the go placement of screenshots. Also I feel like the UI , the font, the color combination can be improved

Incentivized

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.

Incentivized

TestRail definitely saves times. I work in a company that consists of several development teams, all of which have different processes. Some of the teams leverage test cases, some do not. I've noticed that the turn around time it takes for me to pick up a ticket, QA it, and then pass/fail/send feedback is much faster when there is a test case created as I'm not reading through ticket description/comments to figure out what needs to be QA'd.

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized

• We are able to easily pull reports any time management would like to see them
• Jira integration makes it easy to attach to stories/defects and save time
• A quick overview at the system's health as a whole is easy to see from TestRail

Incentivized