Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
WSO2 API Manager
Score 9.3 out of 10
Enterprise companies (1,001+ employees)
WSO2 API Manager makes it possible for developers to both develop and manage APIs of different types. Unlike solutions which focus only on managing API proxies, WSO2 API Manager provides tools to develop APIs by integrating different systems as well. It supports a variety of API types from REST, SOAP, GraphQL, WebSockets, WebHooks, SSEs and gRPC APIs with specialized policies and governance for each different type. Being fully open source, its architecture and extensibility…
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
It's free! No argument can win a fight with that! And it's the only reason I gave it a 5. If you have no money to spend, and a simple environment you'll have a nice product. But free does come with a price. After 5 years we're still struggling with ports, and analytics (it just won't work without any errors caused by some configuration somewhere). An API Manager should work out of the box. The only configuration expertise that any developer wants to invest in, is the configuration of API's. Not the product itself... Anyone who've seen the training material, just for installing this thing will agree that this is not the way to go. Of all the API Managers out there (we've tried 4), WSO2 is the only one were you need to know how this dragon of a java application works internally. Did I already mention the humongous amount of config files?
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
Providing better capabilities comparing the overall API lifecycle management, especially the availability of API Integration layer and a strong identity layer of their own which provides an end-to-end API ecosystem that would be advantageous in terms of a large software development initiative.
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
We've moved away from legacy SOAP services where nobody knew what services was used by who. WSO2 eliminated at least 90% of time spend on any service.
Creating API's (or actually creating the API Management layer...) is so simple that new developers can get away with it in no time. Again, real time gainer.
Since creating API's is so simple, developers are very fast in adopting a kind of "Domain thinking". In comparison with Azure API Manager: Azure does not demand knowledge of "how" the product works, but it's definitely more difficult to get an API up and running in Azure. And for some reason, azure does not promote clean domain driven architecture. Domain Driven architecture is the greatest time saver strategy possible. And WSO2 fits nicely in there.
