Katalon vs. Octopus Deploy

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

Octopus Deploy is well suited for our requirements of deploying across multiple environments with one consistent release. We are saving a lot of time by not having to package and move release files around. Our release process is more efficient and consistent with automation. There are some parts that we could probably perform with existing tools such as DevOps and there are one or two features we have to workaround to fit for our setup such as the step templates to install websites. These are minor in our opinion.

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• Octopus integrates nicely with TeamCity, our build server.
• Octopus uses the same deployment packages across environments and makes it easy to track their progress across those environments.
• Octopus makes it easy to handle configuration across environments which is usually a particularly difficult task.

Incentivized

• Integration with Azure is painstakingly slow as it takes too much time to upload reports to Azure. When raised with the support team i was advised to contact Microsoft who promptly and correctly replied that its an issue with the katalon plugin.
• Non Responsive Customer Support. Several mails were replied late.
• The Desktop app uses too much memory when booting up and running tests even on a 16gb RAM.

• In the past has been somewhat .net focused but that has been changing in recent times
• Would be great if community licenses for NFP organizations were perpetual - but in saying that I appreciate that Octopus does provide my organization with a community [license]. Not all organisations do
• I can’t think of any other

Incentivized

Would be a 10 except for the retirement of the free Community Edition.

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

Octopus Deploy has greatly helped us to improve our reliability and frequency of our deployments and given us the confidence to deploy much more often, with a direct benefit to customers. Cross-platform support and release to Cloud require more focus on the product side.

Incentivized

Octopus Deploy is a software that runs very effectively, is easy to use, does not require such a high learning curve, provides the necessary tools to carry out the functions it offers, making it a very flexible software, it also allows that can be configured according to the needs of the user and provides integrations with other very advantageous tools since they are carried out in a very favorable way.

Incentivized

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

Octopus Deploy support has always been there for us, even when using the free tier, we get responsive hands-on help. We haven't needed to use that level of support since the documentation is clearly written, and help is readily available within the interface itself. Using Octopus Deploy is a truly joyful experience.

Incentivized

Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.

Incentivized

There aren't really any competitors in the land of ASP.NET. Deployment is too ad-hoc. Other tools exist that have massive downsides, like Web Deploy. Most aren't even supported anymore. You could argue that containers (Docker) are a competitor, but containers cannot be used for everything and solve a somewhat different problem. Octopus Deploy is even able to help with containers. To us, Octopus Deploy was really the only really polished solution.

Incentivized

Great support from professional services.

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized

• Allows us to deploy to our fleet quickly and without interruption to service
• Can roll back to previous releases quickly, allowing us to back out of breaking changes in the worst-case scenarios.
• Integration with several targets has allowed us to explore new platforms for our products, such as Docker in AWS ECS.

Incentivized