BlazeMeter vs. Katalon

It is well suited for applications that are mission-critical or applications that can receive high traffic/transactions at unscheduled time periods. Using the load testing feature of BlazeMeter, we can test and ascertain the capacity of the application without the drawbacks of the usual Apache JMeter load testing which depends heavily on the host system from where the load testing is performed.

Incentivized

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

• BlazeMeter takes the .jmx script that teams had written for JMeter. The transition from JMeter to BlazeMeter was seamless.
• BlazeMeter offers its own cloud and hence we did not have to set it up on-premise.
• BlazeMeter hits our application from various geographic locations that simulates real life users.

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• Blazemeter reporting is very basic and shallow. There is no way to drill down or correlate. I can get better reports by using JMeter for free.
• Blazemeter is very costly. Testing with volumes of more than 1K cu is expensive, and can be done for much cheaper if a company/team is willing to invest a bit of time to figure out how to use cloud instances and jmeter slaves, and to write a basic script to collect resulting xml output.

Incentivized

• Integration with Azure is painstakingly slow as it takes too much time to upload reports to Azure. When raised with the support team i was advised to contact Microsoft who promptly and correctly replied that its an issue with the katalon plugin.
• Non Responsive Customer Support. Several mails were replied late.
• The Desktop app uses too much memory when booting up and running tests even on a 16gb RAM.

Easy to setup to do API Performance Testing. Dashboard to showcase results Capability to showcase runtime results

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

In comparison with Blazemeter the closes competitor is JMeter but it has disadvantages like it is not a tool that can be use as a collaborative tool and works locally in a computer, Blazemeter is in the web so different people can access and run tests or collaborate do add, edit or delete the existent scripts.

Incentivized

Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.

Incentivized

• It helped positively by helping to identify the maximum capacity needed for high traffic periods
• Saved revenue by eliminating unwanted duplication of systems

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized