Katalon vs. WSO2 API Manager

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

It's free! No argument can win a fight with that! And it's the only reason I gave it a 5. If you have no money to spend, and a simple environment you'll have a nice product. But free does come with a price. After 5 years we're still struggling with ports, and analytics (it just won't work without any errors caused by some configuration somewhere). An API Manager should work out of the box. The only configuration expertise that any developer wants to invest in, is the configuration of API's. Not the product itself... Anyone who've seen the training material, just for installing this thing will agree that this is not the way to go. Of all the API Managers out there (we've tried 4), WSO2 is the only one were you need to know how this dragon of a java application works internally. Did I already mention the humongous amount of config files?

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• Authentication based on OAuth 2.0 and HTTP Basic Authentication.
• Rate Limiting applied at different levels like Subscriber, API, Resource and Backend.
• Monitoring by exporting the metrics in Prometheus and traces in Jaeger.
• Mediation to perform transformation, orchestration etc.

Incentivized

• Integration with Azure is painstakingly slow as it takes too much time to upload reports to Azure. When raised with the support team i was advised to contact Microsoft who promptly and correctly replied that its an issue with the katalon plugin.
• Non Responsive Customer Support. Several mails were replied late.
• The Desktop app uses too much memory when booting up and running tests even on a 16gb RAM.

• Better QA testing prior to releases rollout
• Better support needed

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.

Incentivized

Providing better capabilities comparing the overall API lifecycle management, especially the availability of API Integration layer and a strong identity layer of their own which provides an end-to-end API ecosystem that would be advantageous in terms of a large software development initiative.

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized

• We've moved away from legacy SOAP services where nobody knew what services was used by who. WSO2 eliminated at least 90% of time spend on any service.
• Creating API's (or actually creating the API Management layer...) is so simple that new developers can get away with it in no time. Again, real time gainer.
• Since creating API's is so simple, developers are very fast in adopting a kind of "Domain thinking". In comparison with Azure API Manager: Azure does not demand knowledge of "how" the product works, but it's definitely more difficult to get an API up and running in Azure. And for some reason, azure does not promote clean domain driven architecture. Domain Driven architecture is the greatest time saver strategy possible. And WSO2 fits nicely in there.

Incentivized