Apache JMeter vs. Layer7 API Security vs. IBM API Connect

JMeter is well suited for Java applications where the user can script the scenario once and make changes to accommodate for as many numbers of users for load test execution. The image and selection of any files or exporting files scenario is handled well.

It is less appropriate to test Ajax applications where it is required to script click per use.

Incentivized

Well suited for publishing internal services with very high security, protecting against attacks, and managing on-demand load. Also good for transform messages and SQL to Rest services. Not appropriate for business logic implementation.

Incentivized

Based on our experience, IBM API Connect clearly excels in large enterprises navigating complex B2B ecosystems, especially within the Fintech and Banking sectors. Its capabilities are particularly valuable for achieving Open Banking compliance, facilitating the monetization of data and services, and seamlessly operating in hybrid cloud environments. However, it's crucial to understand that IBM API Connect primarily functions as an API management and orchestration tool. This means that backend integrations with providers like databases are best handled by dedicated integration middleware, such as IBM App Connect, allowing API Connect to then securely and efficiently expose those functionalities to external entities.

• Easy of use - in generate load like HTTP requests, and processing/analyzing the responses. No coding is necessary at the basic level, just need to understand load testing and the infrastructure being tested.
• Automatic management of things like cookies to help with session state support - so you don't specifically have to worry about it or handle it
• Lots of testing/configuration options to suit your needs in making the right load generation (sampling requests), and analyzing the results, including any pre and post processing of the results first. Things like the Beanshell/BSF pre/post processors, response assertion, regular expression extractor, XPath extractor, CSV data set config
• There is a JMeter cloud service called BlazeMeter that I think would be useful for those that need to scale up high load without provisioning their own systems. I've not personally tried it though, but I recently attended a meetup presentation that highlighted nice useful features that BlazeMeter provides. One should evaluate the service if they are considering JMeter and need to expand beyond existing hardware resources.

• SOAP to Rest / Rest to SOAP web services transformation.
• Secure authorization and authetication (JWT, OpenIP Connect, OAuth, SAML and more). WASP Features as injection protection.
• Easy rule writing without coding.

Incentivized

• API discovery feature is very helpful to allow us to scope the work required before diving into things.
• The ability to set permissions and rulesets at a user level is essential to restrict access to specific datasets.
• Ability for external users (those from outside our organisation) to leverage the API in more restrictive manner.

Incentivized

• Jmeter requires many tweaks with respect to its configuration file and thread properties. users need to edit theses files themselves. There could be some interface where we can edit this fields.
• Jmeter cannot handle more threads and hangs up when we increase the number of threads. This causes lot of inconvenience. In these situations, user can be notified that such change would be lead to slow performance so that user can do as required. The same appears when we try to view huge files on graph listener.
• Jmeter should optimize the read and write access to output csv since it acts as overhead to the I/O performance. This affects our test results for the application which we are testing.

• It would be nice to have an integration with services like Github for managing projects and versions.

Incentivized

• Improvement can come in the interface of the tool, which is sometimes complex and requires a deep understanding of the tool.
• Improving the different levels of access to the playroom could be helpful because product owners and developers need to see other information.
• Publishing API and integration of API can be more straightforward.

Incentivized

Price, Wiki and user sharing. Having access to the information provided by the developers and other open source providers is key for me. The ability to share information and get answers directly is very important to success in software testing. And the price of this product currently is amazing. Too many companies charge way too much money for products that are far behind in their value and pertinence

The purpose related to performance and load testing through Apache JMeter works fine but the usability of the tool should be improved quite a lot. If someone starts with the Jmeter fresh without prior experience, they need to put more efforts in understanding the tool. The UI is not that great which is the main reason not to give high rating on usability.

Incentivized

The Layer7 API Management is a great solution, but, sometimes the documentation is not clear or has lack of examples.

Incentivized

I can honestly say that the availability of the platform has been incredible, with almost no disruptions during the two years of the implementation. Availability is provided through the distributed architecture that may be deployed in multiple zones. We have had almost no cases of unplanned outages, and most maintenance operations were done during planned downtimes.

Incentivized

I have been using JMeter for the last year. By using this tool, you can make sure the system will work under varied loads. It helps us to simulate real time scenarios by creating required virtual users and make sure the application will work under load. Perform load, stress, and stability testing using JMeter.

Incentivized

The support team is good and the response time is reasonable, but (there is always a but), when support inquiries are made by someone experienced, and the question is not trivial, it is boring and takes a long time to get the case escalated to the next level of support.

Incentivized

I have used LoadRunner and Silkperformer, and so far Jmeter turns out be the easiest to use of all these. While each of them have their own ROI, Jmeter can be picked by anyone in hours and start testing within a day. While with other tools, we need to get license, install them (takes a while) and setup tests and firewalls, etc.

We evaluated 3 tools and decided on Layer7 API Management because it had the shortest implementation times and fastest learning curve, without losing power of customization and scalability.

Incentivized

IBM API Connect and Apigee are both robust API management platforms. IBM API Connect was selected for its strong integration capabilities, hybrid cloud deployment options, and comprehensive analytics. It aligns well with organizations seeking flexibility and control over their API ecosystems, especially when dealing with complex integration scenarios across diverse environments.

Incentivized

• Good ROI on improving the performance of the application.
• Finding issues in the performance.
• Benchmark the performance results.
• CON: Need skillset to create and maintain the scripts in Java.
• Scripts are reusable and it is executed by any user.
• Need Client and Server setup to execute the scripts.

Incentivized

• Very fast implementation and is easy to learn & use.

Incentivized

• Our company benefited from IBM API Connect's assistance in establishing cooperation with our outside providers and efficiently managing the API. As a result, we were able to expand our vendor ecosystem.
• Additionally, creating APIs and implementing them on the goods was not too difficult. Additionally, we were able to ensure that only appropriate people could publish the APIs following evaluation by using role-based access control.
• Our teams were able to reuse and share departmental APIs and services with each other thanks to the simple, self-service developer site.

Incentivized