Katalon vs. Red Hat Ansible Automation Platform

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

For automating the configuration of a multi-node, multi-domain (Storage, VM, Container) cluster, Ansible is still the best choice; however, it is not an easy task to achieve. Creating the infrastructure layer, i.e., creating network nodes, VMs, and K8s clusters, still can't be achieved via Ansible. Additionally, error handling remains complex to resolve.

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• Debugging is easy, as it tells you exactly within your job where the job failed, even when jumping around several playbooks.
• Ansible seems to integrate with everything, and the community is big enough that if you are unsure how to approach converting a process into a playbook, you can usually find something similar to what you are trying to do.
• Security in AAP seems to be pretty straightforward. Easy to organize and identify who has what permissions or can only see the content based on the organization they belong to.

Incentivized

• Integration with Azure is painstakingly slow as it takes too much time to upload reports to Azure. When raised with the support team i was advised to contact Microsoft who promptly and correctly replied that its an issue with the katalon plugin.
• Non Responsive Customer Support. Several mails were replied late.
• The Desktop app uses too much memory when booting up and running tests even on a 16gb RAM.

• I can't think of any right now because I've heard about the Lightspeed and I'm really excited about that. Ansible has been really solid for us. We haven't had any issues. Maybe the upgrade process, but other than that, as coming from a user, it's awesome.
• Give out Lightspeed for free.

Incentivized

Even is if it's a great tool, we are looking to renew our licence for our production servers only. The product is very expensive to use, so we might look for a cheaper solution for our non-production servers. One of the solution we are looking, is AWX, free, and similar to AAP. This is be perfect for our non-production servers.

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

It's overall pretty easy to use foe all the applications I've mentioned before: configuring hosts, installing packages through tools like apt, applying yaml, making changes across wide groups of hosts, etc. Its not a 10 because of the inconveinience of the yaml setup, and the time to write is not worth it for something applied one time to only a few hosts

Incentivized

Great in almost every way compared to any other configuration management software. The only thing I wish for is python3 support. Other than that, YAML is much improved compared to the Ruby of Chef. The agentless nature is incredibly convenient for managing systems quickly, and if a member of your term has no terminal experience whatsoever they can still use the UI.

Incentivized

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

There is a lot of good documentation that Ansible and Red Hat provide which should help get someone started with making Ansible useful. But once you get to more complicated scenarios, you will benefit from learning from others. I have not used Red Hat support for work with Ansible, but many of the online resources are helpful.

Incentivized

I spoke on this topic today!

Incentivized

Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.

Incentivized

AAP compares favorably with Terraform and Power Automate. I don't have much experience with Terraform, but I find AAP and Ansible easier to use as well as having more capabilities. Power Platform is also an excellent automation tool that is user friendly but I feel that Ansible has more compatibility with a variety of technologies.

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized

• POSITIVE: currently used by the IT department and some others, but we want others to use it.
• NEGATIVE: We need less technical output for the non-technical. It should be controllable or a setting within playbooks. We also need more graphical responses (non-technical).
• POSITIVE: Always being updated and expanded (CaC, EDA, Policy as Code, execution environments, AI, etc..)

Incentivized