Katalon vs. SpiraTest

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

I think SpiraTest is well suited as a test suite, but in situations where the team is already using multiple products from one particular provider, it may be better to go with that provider's test solution. This is because integrations are very important nowadays, and should be considered when picking your project management software. It should be noted that management selected SpiraTest primarily due to the low cost of the test software. I'm less familiar with other test software in the field, but if cost is an issue, you should take a look at SpiraTest.

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• A clean view of test case steps, expected results and the ability to record a result for each step quickly on one page per test set. This really helps testers work through executing manual test cases.
• Hierarchical structure of releases and builds, requirements, and test cases.
• Can quickly build test sets and/or test runs per build/release.
• Simple identification of each test case, requirement, test run, build, release, etc.

• Integration with Azure is painstakingly slow as it takes too much time to upload reports to Azure. When raised with the support team i was advised to contact Microsoft who promptly and correctly replied that its an issue with the katalon plugin.
• Non Responsive Customer Support. Several mails were replied late.
• The Desktop app uses too much memory when booting up and running tests even on a 16gb RAM.

• Training can be difficult to find at times.
• Basic reporting can become very verbose unless you set lots of filters and parameters.
• The ability to customize some of the verbiage in the application would help bridge the gap between translating what SpiraTest's testing terminology is and the company culture uses for testing terminology.

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

As promised during product selection, SpiraPlan support has always been quick and helpful. Replies almost always come back in hours (and often in an hour or two). And SpiraPlan's online tech support maintains all support conversations online so no need to look through emails to try to recall repllies.

Incentivized

Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.

Incentivized

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized

• SpiraTest can be improved in terms of its dashboard and reporting. This will be really helpful for the management as well.

Incentivized