Katalon vs. SpecFlow

Overview
ProductRatingMost Used ByProduct SummaryStarting Price
Katalon
Score 7.7 out of 10
N/A
Katalon Studio is provided by the vendor as a free and robust automation solution for API, Web and Mobile testing. It is designed to eliminate the complexities of building an automation framework by integrating all necessary test components with built-in keywords and project templates. Katalon Studio includes a full feature set for a complete test automation process covering multiple platforms and testing types. New users can get started with test automation using its easy-to-use…
$69
per month
SpecFlow
Score 10.0 out of 10
N/A
SpecFlow is an open source BDD for .NET. that aims to bridge the communication gap between domain experts and developers by binding readable behavior specifications to the underlying implementation.N/A
Pricing
KatalonSpecFlow
Editions & Modules
Node Locked License
$69.00
per month
Floating License
$1,529.00
per year
No answers on this topic
Offerings
Pricing Offerings
KatalonSpecFlow
Free Trial
YesNo
Free/Freemium Version
YesNo
Premium Consulting/Integration Services
NoNo
Entry-level Setup FeeNo setup feeNo setup fee
Additional DetailsKatalon Studio was originated from KMS Technology R&D. It has been a successful solution for numerous automation projects ranging from small team to enterprise client. Katalon Studio is free for use, product development is funded by the testing services provided by KMS Technology. To support future releases, please consider subscribing to our business support services.
More Pricing Information
Community Pulse
KatalonSpecFlow
Considered Both Products
Katalon
Chose Katalon
The decision to use Katalon was made by my organization before i joined it.
Chose Katalon
Initially, our oragnization had to integrate Katalon tests with the Jenkins to schedule the test runs. The reports were stored in company local database. To identify test flakiness or product issue one had to dig deep into the reports. This issue was all solved by Katalon …
Chose Katalon
Ready API is one of the most popular and widely used tool for API automation. Data driven testing and reporting are some of the features which makes this tool very useful. but if we compare this tool with Katalon Studio, this is only useful for API automation where as Katalon …
Chose Katalon
Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.
Chose Katalon
Katalon Studio License cost is reasonable and it suits the budget where as other tools are expensive and its difficult to get right ROI.
Capabilities are user friendly while using the Katalon Studio for api, mobile , web and desktop application automation where as expensive …
Chose Katalon
My company was using selenium until we shifted to Katalon Studio. The transition was pretty easy as this tool is built on top of selenium frameworks. Since the companies application grew from web to mobile, glad to have a single automation tool for all different testings. The …
Chose Katalon
I've used the Selenium IDE before. According to Katalon Studio, it is very slow and has little functionality.
Chose Katalon
Katalon has no License fee, a nice user-friendly UI to use, easy setup and quick automation, easy integration with other tools like JIRA, Jenkins, ALM, etc., and Katalon analytics tools.
SpecFlow
Chose SpecFlow
SpecFlow is .Net based which supports C#.
Behave is Python based.
Cucumber is Java based.
Best Alternatives
KatalonSpecFlow
Small Businesses
BrowserStack
BrowserStack
Score 8.4 out of 10
BrowserStack
BrowserStack
Score 8.4 out of 10
Medium-sized Companies
ReadyAPI
ReadyAPI
Score 6.1 out of 10
ReadyAPI
ReadyAPI
Score 6.1 out of 10
Enterprises
ignio AIOps
ignio AIOps
Score 8.1 out of 10
ignio AIOps
ignio AIOps
Score 8.1 out of 10
All AlternativesView all alternativesView all alternatives
User Ratings
KatalonSpecFlow
Likelihood to Recommend
8.1
(0 ratings)
9.0
(0 ratings)
Usability
9.0
(0 ratings)
-
(0 ratings)
Support Rating
9.9
(0 ratings)
-
(0 ratings)
User Testimonials
KatalonSpecFlow
Likelihood to Recommend
Free trial version is fine but you don't get the adequate support and VFM for the licensed version. Customer service needs to be improved a lot and any QA's with less technical background or less time to troubleshoot issues in the software on their own would find it difficult to use.
Read full review
It is best suited for implementing the automated test cases in a human readable form so it's easy for non-technical members of the team and stakeholders to understand the test cases, features and the functionalities of the application. Automation of Integration tests and End to End tests are good use case. It is less appropriate or situations where the focus is only on the writing and maintenance of unit tests.
Read full review
Pros
  • Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
  • Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
  • Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
  • Image-based testing: Katalon also provides inbuilt image based test automation.
Read full review
  • Versatility to be used in combination with different kinds of automated testing like automated performance testing, API testing, UI testing etc. I use JavaScript, Selenium, C#, email testing libraries, database testing libraries in combination with BDD with SpecFlow. I am able to use all these with SpecFlow to make my automation framework to be able to automate any kind of automated testing.
  • It provides different widely used runner options like NUnit, XUnit etc. Before I started to work on establishing proper test automation in my workplace, the previous automation framework (non-BDD based) as well as unit tests used NUnit runner. The transition to using BDD was smooth because we could use the same runner and there were no compatibility issues.
  • The auto-complete feature is good. I use it with Visual Studio as well as Rider and I don't have to recall the entire Gherkin statements. I just type a few words and the entire Gherkin statement implemented in framework is auto-suggested by SpecFlow. It saves time and context switching.
Read full review
Cons
  • Katalon Studio supports only grovvy language for now. It will be nice to see support for other scripting languages.
  • Performance issues; sudden crash of the tool.
  • Debug button is provided only to licensed owner since Katalon 7+ version.
  • New tool, few resources available to learn. So there is a mild learning curve here.
Read full review
  • SpecFlow does not accepts optional input variables in the methods defined during Gherkin statement implementation. Cucumber supports optional input variables in the methods defined during Gherkin statement implementation.
  • The tests identified while using SpecFlow with NUnit removes all white spaces in the scenario names. It makes the tests less readable. If the white spaces are not auto-removed, it would be much better for readability as well as their actual identification in the repository.
Read full review
Usability
Due to the slowness, there is an intermediate timeout. There is no other problem.
Read full review
No answers on this topic
Support Rating
I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.
Read full review
No answers on this topic
Alternatives Considered
Initially, our oragnization had to integrate Katalon tests with the Jenkins to schedule the test runs. The reports were stored in company local database. To identify test flakiness or product issue one had to dig deep into the reports. This issue was all solved by Katalon TestOps. Katalon TestOps provides report history in one place and also helps in easy triage of the report by pointing out script flakiness
Read full review
SpecFlow is .Net based which supports C#. Behave is Python based. Cucumber is Java based. Ghost Inspector is no-code based but provides very limited testing features. We wanted to implement BDD so we rued out using Ghost Inspector. Most of the developers in my team are C# experts so it was decided for everyone's comfort to go for SpecFlow rather than Behave or Cucumber. It's import to have technical experts in the language of the automation framework because there are many situations where the solutions to the test automation needs are not straightforward and implementing those requires expertise in the related programming language.
Read full review
Return on Investment
  • Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
  • Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
  • Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.
Read full review
  • Everyone stays on the same page regarding the behavior of existing functionalities whether it be technical or non-technical individuals. So there is less need for multiple people to get involved which saves time and thus money.
  • Reusing the same code through the implemented Gherkin statement saves test automation time and thus reduces cost.
  • We combine SpecFlow with other opensource testing technologies to make our automation framework more versatile which further saves costs for us.
Read full review
ScreenShots

Katalon Screenshots

Screenshot of Project setup - fast start with pre-built templates and scriptsScreenshot of Run Single Tests or Suites on multiple configurations and data sets
Multiple Platform Support so you can run your automation tests on Windows, OSX, Android, and iOSScreenshot of Maintain TestsScreenshot of View Reports