Digital.ai Release vs. Katalon TestOps

Mainly used in release management where all deployments are well managed and processed further based on the approval system. Complete enterprise-level solution with minor difficulties which need to be added to product improvement features. Integration with other CI-CD tools makes it easier to perform tasks in terms of release and deployments.

Incentivized

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

• Cross-team release workflow control using email, texts, scripts allow our release management to be truly a 360 process.
• XL Release allowing our Jenkins toolchain to control the beginning of release trains which is very powerful.
• XL release allows us to expose the business process flow for anyone to read direct at the source which runs the process instead of a separate vision.

Incentivized

• One place for all report history
• Get count of tests automated each release
• Easy way to schedule and run test suites automatically
• Track test flakiness easily

Incentivized

• Pagination of data - across tool.
• User Roles Management API can be improved.
• Case insensitive ID's are treated differently making user face login and access issues.
• Dependency on Universal template/custom plugins creation should be reduced.
• Code versioning of templates is very difficult.
• Better error handling.
• Futures Timeout Issues.

Incentivized

• Very small community
• Requires learning curve
• Sometimes application crashes

Incentivized

The tool is easy to use, easy to navigate and learn. Manages releases with proper approvals in a systematic manner. Though it needs minor improvements in terms of pagination (data loading), access management, but, overall the tool helps in increasing productivity and less time for production deployments.

Incentivized

Support is not good at all. To this day, I have to mail my queries and their support site does not log in for me (me alone). But, upon contacting many times, no one helps with a proper response. Though good thing is, I get a proper response over mail too. But, being informative about the tool and not on the issues faced by users outside of the process to get support should also be addressed equally. Which is currently missing in support.

Incentivized

XL release is simpler to configure and deploy to the organization than other change management platforms I have used. That simplicity has minor drawbacks requiring you to fit into a limited set of control methods but that exercise helped us simplify a needlessly onerous process.

Incentivized

Initially, our oragnization had to integrate Katalon tests with the Jenkins to schedule the test runs. The reports were stored in company local database. To identify test flakiness or product issue one had to dig deep into the reports. This issue was all solved by Katalon TestOps. Katalon TestOps provides report history in one place and also helps in easy triage of the report by pointing out script flakiness

Incentivized

• XL release has improved our consistency of release process, removing multiple days worth of manual status checking and coordination.
• XL release has allowed us to increase the number of beta releases we can support due to simplifying our release actions.

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized