Skip to main content
TrustRadius
Apache Kafka

Apache Kafka

Overview

What is Apache Kafka?

Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical…

Read more

Learn from top reviewers

The versatile Apache Kafka

Rating: 8 out of 10
December 31, 2024
IncentivizedVetted ReviewVerified User
We use Apache Kafka for asynchronous communication. For any processing that we need to do on background, we use Apache Kafka. We also set the configurations in such a way so ...
Continue reading
Ankit Singh
Engineer - Information Technology
Senior Engineering Manager
Pharmeasy
Apache Kafka4 years of experience

Apache Kafka - Default Choice For Large Scale Messaging

Rating: 8 out of 10
August 23, 2023
Vetted ReviewVerified User
Apache Kafka is really the bedrock of all things streaming and data processing. I cannot imagine if there is any other product that does it better. My last 2 companies used it...
Continue reading
Victor Tay
Engineer - Product Management
Engineer
IronNet CyberSecurity
Apache Kafka5 years of experience

A Deep Dive into the Power and Potential of Apache Kafka

Rating: 10 out of 10
August 20, 2023
IncentivizedVetted ReviewVerified User
We use Apache Kafka as an event bus for all our async activities & Micro Service Communication, like sending emails, SMS, and notifications between services and consumers ...
Continue reading
Alok Pabalkar
C-Level Executive - Research & Development
Co-Founder & CTO
GIDE.AI
Apache Kafka5 years of experience

Kafka for tracking changes

Rating: 8 out of 10
May 30, 2023
Vetted ReviewVerified User
We use Apache Kafka to stream order information across systems. An order may go through certain updates through its lifecycle. These updates need to be communicated to the sys...
Continue reading
Animesh Kumar
Engineer - Engineering
Senior Member of Technical Staff
athenahealth
Apache Kafka3 years of experience
Verified on LinkedIn

Confluent Kafka for messaging.

Rating: 6 out of 10
May 22, 2023
IncentivizedVetted ReviewVerified User
Currently consulting and implementing for a bank, we use a cloud-native Kafka solution (Confluent Kafka) for brokering. The solution is well documented, and liked by the devel...
Continue reading
Verified user
Consultant
Apache Kafka2 years of experience
Log in with LinkedIn to see content from your network
Return to navigation

Product Details

What is Apache Kafka?

Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

Apache Kafka Technical Details

Operating SystemsUnspecified
Mobile ApplicationNo
Return to navigation

Comparisons

View all alternatives
Return to navigation

Reviews From Top Reviewers

View all reviews
(1-5 of 19)

Kafka for tracking changes

Rating: 8 out of 10
May 30, 2023
AK
Animesh Kumar
Senior Member of Technical Staff
athenahealth (Hospital & Health Care, 5001-10,000 employees)
Vetted Review
Verified User
Apache Kafka
3 years of experience
Verified on LinkedIn
Use Cases and Deployment Scope
We use Apache Kafka to stream order information across systems. An order may go through certain updates through its lifecycle. These updates need to be communicated to the systems in near real time and we rely on Kafka for this.Our business use case is to take these orders up with the insurance companies for approval and thus the order information need to be up to date. Kafka has been excellent at doing this so far.
Pros
  • Receiving messages from publisher and sending to consumer in FIFO manner
  • Handling of errors using Dead Letter Queue when message could not be consumed on the consumer end
  • Fault tolerance
Cons
  • Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
  • Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
  • Learning curve around creation of broker and topics could be simplified
Likelihood to Recommend
Kafka is well suited in scenarios where a message need to be sent to another system in fault tolerant manner. It is useful when the message size could be large and large number of messages could be floating around.
It would be less appropriate or rather an overkill to use Kafka in scenarios where we are sending short messages to offload certain tasks(like invoice generation and sending email) to a worker(like celery). For such use cases, simple queueing solutions like Amazon SQS should suffice.

The versatile Apache Kafka

Rating: 8 out of 10
Incentivized
December 31, 2024
AS
Ankit Singh
Senior Engineering Manager
Pharmeasy (Information Technology & Services, 501-1000 employees)
Vetted Review
Verified User
Apache Kafka
4 years of experience
Use Cases and Deployment Scope
We use Apache Kafka for asynchronous communication.
For any processing that we need to do on background, we use Apache Kafka. We also set the configurations in such a way so that we can use it for retrying messages in a topic.
We also use it for data streaming which powers our data platform.
Pros
  • Its extremely fast. It is able to deliver messages very quickly.
  • It is very reliable, I have not yet seen any cases where messages might have dropped
  • Using different configurations we can model it any way and cater to large number of business use cases.
Cons
  • If there can be some way of scheduling messages to reappear that would be great.
  • There should be functionality of decreasing the partitions on the fly so that we can scale down when needed.
  • Apache Kafka should have better consumer UI view so that we can more details on the consumers attached.
Likelihood to Recommend
It is well suited for any asynchronous programming use case. It is also a good fit for the data streaming use case. It is not suited for use cases where batching of messages is needed.

Apache Kafka: Where messaging meets storage

Rating: 7 out of 10
Incentivized
April 09, 2021
Verified User
Strategist in Information Technology
Package/Freight Delivery Company, 10,001+ employees
Vetted Review
Verified User
Apache Kafka
4 years of experience
Use Cases and Deployment Scope
Apache Kafka is used by our company as the "next generation" of messaging/data-streaming pipeline solutions, to replace our old legacy JMS-based messaging solution and enable the modern streaming API based applications. When it is used for messaging purposes, we shift the responsibility of data replay from the message source (publisher application) to the message destination (consumer application). This flexibility resolved the legacy issue of sources replaying the messages but impacting all subscribers to the same topic. When Kafka is used as the streaming pipeline, it is integrated seamlessly with the Spark/Spring Stream-based analytic solutions, as it is also a kind of distributed storage.
Pros
  • Undoubtedly, Kafka's high throughput and low latency feature are the highlights.
  • Kafka can scale horizontally very well.
Cons
  • The CLI and configuration details need to be worked out more in-depth. The naming convention of configuration is not so good and causing a lot of confusion. Sometimes there are too many configuration parameters to tune--requires the adopter to understand a lot of tricks like NFS entrapment, for example.
  • Lack of a good monitoring solution so far
Likelihood to Recommend
When it is used as messaging, Apache Kafka is majorly preferred when the use case is Pub/Sub typed. It is not suitable to deal with the end-to-end queue use case nor the request/response paradigm. When Apache Kafka is used for streaming purposes, it doesn't have the native implementation of the query language, it is just a pipeline. You still need to put a lot of programming efforts into your streaming client-side to take care of those analytic requirements.

A Deep Dive into the Power and Potential of Apache Kafka

Rating: 10 out of 10
Incentivized
August 20, 2023
AP
Alok Pabalkar
Co-Founder & CTO
GIDE.AI (Internet, 11-50 employees)
Vetted Review
Verified User
Apache Kafka
5 years of experience
View profile
Use Cases and Deployment Scope
We use Apache Kafka as an event bus for all our async activities & Micro Service Communication, like sending emails, SMS, and notifications between services and consumers and for event & data processing.
Pros
  • Event driven architectures
  • Any use case which requires async data processing
  • Any use case with production and consuming the same data to build business-specific processing
Cons
  • Zookeeper services configuration can be simplified
  • Data logging needs to be secured
  • Restarting & overall management needs to be improved
Likelihood to Recommend
- It's Super fast - Has some learning curve but once mastered it brings scale - All logics that need producer & consumer kind of implementation (Bulk Notification, etc) - Event-driven architectures can be implemented with Apache Kafka

Apache Kafka - Default Choice For Large Scale Messaging

Rating: 8 out of 10
August 23, 2023
VT
Victor Tay
Engineer
IronNet CyberSecurity (Computer Software, 51-200 employees)
Vetted Review
Verified User
Apache Kafka
5 years of experience
Use Cases and Deployment Scope
Apache Kafka is really the bedrock of all things streaming and data processing. I cannot imagine if there is any other product that does it better. My last 2 companies used it, and my current one does so as well. If you want your data stream to be organized and sent, Apache Kafka has become the tool of choice. I have dabbled in Azure EventHubs as well, if you are into opensource data streaming, Apache Kafka will take you where you need to be for data lakes and the amount of data that is streamed for the cybersecurity industry that my company is in. Without Apache Kafka, there is no way that my company products can handle the volume of data that we process for our customers.
Pros
  • Data streaming is really second to none.
  • Scaling, done right, Apache Kafka is a workhorse.
  • Ease of administration - Although you cannot really compare to Azure EventHubs, but that is comparing between Apples and Oranges.
Cons
  • The web UI has not really changed in years. UX has been refreshed, but a more streamlined UX instead of many 3rd party webUX tools, will be most welcome.
  • Webhooks can still be tricky to troubleshoot at times.
  • CLI monitoring is a learning curve to get it right.
Likelihood to Recommend
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
Return to navigation