TrustRadius Insights
Apache Kafka is a widely-used platform that has proven to be invaluable in various industries and applications. It is relied upon by …
Overview
What is Apache Kafka?
Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical…
Recent Reviews
Reviewer Pros & Cons
Product Details
- About
- Tech Details
What is Apache Kafka?
Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
Apache Kafka Technical Details
| Operating Systems | Unspecified |
|---|---|
| Mobile Application | No |
Comparisons
Compare with
Reviews and Ratings
(127)
Community Insights
TrustRadius Insights are summaries of user sentiment data from TrustRadius reviews and, when necessary, 3rd-party data sources. Have feedback on this content? Let us know!
- Business Problems Solved
- Pros
- Cons
- Recommendations
Apache Kafka is a widely-used platform that has proven to be invaluable in various industries and applications. It is relied upon by organizations to have real-time communication and keep order information up-to-date. This is particularly useful for organizations that need to process large volumes of data, such as those in the cybersecurity industry. Apache Kafka is also considered the go-to tool for event streaming, generating events and notifying relevant applications for consumption. Additionally, it is used in both first-party and third-party components of applications to address data proliferation and enable efficient notifications.
Another key use case for Apache Kafka is replacing classical messaging software within organizations, becoming the new standard for messaging. This powerful streaming framework plays a crucial role as a queuing mechanism for records in various pipelines, providing a simple yet efficient system for queuing and maintaining records. Moreover, Apache Kafka excels at storing and processing records in dedicated servers, supporting high data loads and offering the ability to replay consumed data. This makes it ideal for buffering incoming records during traffic spikes or in case of data infrastructure failures.
Furthermore, Apache Kafka finds its purpose in driving real-time monitoring by sending log information to feed other applications. Its ability to scale and manage common errors in messaging allows organizations to handle large quantities of messages per second without compromising performance. Another notable use case involves Apache Kafka acting as an efficient stream/message ingestion engine for customer-facing applications, enabling internal analytics and real-time decision-making.
Additionally, Apache Kafka integrates seamlessly with big data technologies like Spark, making it a valuable addition to big data ecosystems. Organizations have successfully replaced legacy messaging solutions with Apache Kafka, thanks to its ability to serve as a messaging and data-streaming pipeline solution. It enables modern streaming API-based applications while ensuring high availability and clustering as a message broker between client-facing applications.
Moreover, Apache Kafka serves as an ingress and egress queue for big data systems, facilitating data storage and retrieval processes. It also acts as a reliable queue for frontend applications to retrieve data and analytics from MapR and HortonWorks. With over five years of being utilized in data pipelines, Apache Kafka has consistently demonstrated excellent performance and reliability.
In summary, Apache Kafka proves to be versatile and essential across various industries and use cases. It facilitates real-time communication, ensures data integrity, enables efficient event streaming, replaces classical messaging software, and supports high scalability and fault tolerance. With its robust capabilities, Apache Kafka continues to be the go-to solution for organizations seeking to streamline their data processing and communication systems.
Fault tolerance and high scalability: Users have consistently praised Apache Kafka for its fault tolerance and high scalability. Many reviewers have stated that Kafka excels in handling large volumes of data and is considered a workhorse in data streaming.
Ease of administration: Reviewers appreciate Kafka's ease of administration, noting that it offers an abundance of options for managing and maintaining queues. Multiple users have mentioned that the platform allows for easy expansion and configuration of cluster growth, making it straightforward to administer.
Real-time streaming capabilities: Kafka's real-time streaming capabilities are seen as a significant advantage by users. Several reviewers have highlighted the platform's ability to handle real-time data pipelines and its resistance to node failure within the cluster. This feature enables users to process asynchronous data efficiently and ensures continuous availability of the system.
Difficulty Monitoring Kafka Deployments: Some users have found it difficult to monitor their Kafka deployments and have expressed a desire for a separate monitoring dashboard that would provide them with better visibility into their topics and messages.
Steep Learning Curve for Creating Brokers and Topics: The process of creating brokers and topics in Kafka has been described as having a steep learning curve by some users, who believe that it could be simplified to make it more accessible.
Outdated Web User Interface: The web user interface of Kafka has not been updated in years, leading some users to feel that it lacks a streamlined user experience. They express the need for a more modern interface instead of relying on third-party tools.
Users have recommended using Apache Kafka for various messaging platform requirements. It integrates easily with multiple programming languages, offers stream processing capabilities, distributed data storage, and the ability to handle multiple requests simultaneously.
Another common recommendation is to consider Apache Kafka as a messaging broker due to its extensive feature set and guaranteed delivery of data to consumers. Users find it highly supported and widely used within the community.
Users also recommend Apache Kafka for streaming large amounts of data. They praise its scalability and ease of use, although they mention that manual rebalancing of partitions may be required when adding or deleting nodes. Additionally, users appreciate that Kafka allows connections between multiple producers and consumers with low resource consumption.
Overall, Apache Kafka is regarded as a practical choice for message processing systems, data streaming, and handling large volumes of data due to its stability, scalability, and diverse features.
Attribute Ratings
Reviews
(1-18 of 18)Companies can't remove reviews or game the system. Here's why
August 23, 2023
Apache Kafka - Default Choice For Large Scale Messaging
Score 8 out of 10
Vetted Review
Verified User
- Data streaming is really second to none.
- Scaling, done right, Apache Kafka is a workhorse.
- Ease of administration - Although you cannot really compare to Azure EventHubs, but that is comparing between Apples and Oranges.
- The web UI has not really changed in years. UX has been refreshed, but a more streamlined UX instead of many 3rd party webUX tools, will be most welcome.
- Webhooks can still be tricky to troubleshoot at times.
- CLI monitoring is a learning curve to get it right.
August 21, 2023
Apache Kafka - FTW
- High availability
- performance
- Admin user interface
- zookeeper logs could be better
- monitoring
August 20, 2023
A Deep Dive into the Power and Potential of Apache Kafka
- Event driven architectures
- Any use case which requires async data processing
- Any use case with production and consuming the same data to build business-specific processing
- Zookeeper services configuration can be simplified
- Data logging needs to be secured
- Restarting & overall management needs to be improved
May 30, 2023
Kafka for tracking changes
Score 8 out of 10
Vetted Review
Verified User
- Receiving messages from publisher and sending to consumer in FIFO manner
- Handling of errors using Dead Letter Queue when message could not be consumed on the consumer end
- Fault tolerance
- Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
- Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
- Learning curve around creation of broker and topics could be simplified
May 22, 2023
Confluent Kafka for messaging.
- Brokering
- Topic definition.
- Private access to a cluster.
- Visualisation solutions.
January 23, 2022
Kafka: Best Streaming Platform on the Market
- Real time streaming
- Performance
- Scalability
- Management tools
January 21, 2022
Apache Kafka is awesome! Tricky sometimes, but we love it!
- The pub/sub model
- Quick data transfer - regardless of volume (if you have enough resources)
- Ability to transfer large amounts of data consistently (non-binary)
- The Kafka Tool is a community-made Java application that looks and feels from the past century.
- Logging can be confusing. This certainly shows when we have to do troubleshooting.
- Hybrid scenarios - pub/sub, but there are services in and outside a Kubernetes cluster. Then there are a ~3 options, but only 2 (the harder ones) are production-safe.
January 06, 2022
Kafka events; helping your company work with data
- Queuing of records
- Easy expansion of Topic parititions
- An abundance of options for managing and maintaining queues
- Easy expansion of cluster for growth
- A management interface would be nice
- Built in logging tools
December 06, 2021
Apache Kafka - a must have tool for distributed toolkit
- Every setting is configurable.
- Work seamlessly during high data load.
- Partition mechanism.
- Easy configurable.
- Zookeeper configuration.
- Front-end can be developed to configure properties.
- UI for administrative configuration.
November 08, 2021
Kafka is an excellent tool for data integration!
- Message queue
- Capture data
- Make data available
- Integration between systems
- More out of the box connectors for various other system integration
August 07, 2021
Apache Kafka for your Data solutions
- Data Pipeline
- Asynchronous processing
- Data retention for reprocessing
- Dashboards to monitor the performance
- ZooKeeper free
- Connectors for more languages
- Scalable
- Fast
- Performance
- Open source
- Performance security
- Monitoring
- Configuration
April 09, 2021
Apache Kafka: Where messaging meets storage
- Undoubtedly, Kafka's high throughput and low latency feature are the highlights.
- Kafka can scale horizontally very well.
- The CLI and configuration details need to be worked out more in-depth. The naming convention of configuration is not so good and causing a lot of confusion. Sometimes there are too many configuration parameters to tune--requires the adopter to understand a lot of tricks like NFS entrapment, for example.
- Lack of a good monitoring solution so far
March 23, 2020
Apache Kafka open source stream processing software
- It handles large amount of data simultaneously. Makes application scalable.
- It is able to handle real time data pipeline.
- Resistant to node failure within the cluster.
- Does not have complete set of monitoring tools.
- It does not support wild card topic selection.
- Brokers and consumer pattern reduces the performance.
November 07, 2019
Apache Kafka for large scale message ingestion
- Apache Kafka is able to handle a large number of I/Os (writes) using 3-4 cheap servers.
- It scales very well over large workloads and can handle extreme-scale deployments (eg. Linkedin with 300 billion user events each day).
- The same Kafka setup can be used as a messaging bus, storage system or a log aggregator making it easy to maintain as one system feeding multiple applications.
- Apache Kafka does take some initial setup and deployment time especially if you haven't bought support from Confluent.
- It is not a full solution so for an analytics use case, you will still need something like Tibco.
- It does not have a SQL based query engine out-of-the-box so building/using analytics on top can be a lot of work. It would be great to have something already baked into Kafka out-of-the-box.
November 06, 2019
Battle-tested, de facto option for message broker
- Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
- Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
- Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
- Doesn't work well with many small topics (on the order of thousands). There is a physical limit due to file handler usage on the number of topics Kafka can have before it grinds to a halt. This is not an issue for most people but it became an issue for us, as we need to have many, many topics and so we weren't able to fully migrate to Kafka except for a few of our big queues.
- Lack of tenant isolation: if a partition on one node starts to lag on consume or publish, then all the partitions on that node will start to lag. That's what we've noticed and it's really frustrating to our customers that another customer's bad data affects them as well.
- I don't have tooo much experience here, but I hear from other engineers on my team that the CLI admin tool is a real pain to use. For example, they say the arguments have no clear naming convention so they are hard to memorize and sometime you have to pass in undocumented properties.
January 30, 2019
Kafka quick queue
- Fast queuing
- Easy to set up and configure
- Easy to add and remove queues
- User interface for configuration could be a little better
- Could be a little more defined when configuring files
- Logging is a little hard to follow
March 01, 2018
Apache Kafka, the F1 of messaging
- High volume/performance throughput environments
- Low latency projects
- Multiple consumers for the same data, reprocessing, long-lasting information
- Still a bit inmature, some clients have required recoding in the last few versions
- New feaures coming very fast, several upgrades a year may be required
- Not many commercial companies provide support