Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
ClickHouse
Score 9.1 out of 10
N/A
ClickHouse is an open-source, column-oriented OLAP database system enabling real-time analytical reports using SQL queries. With linear scalability, it handles trillions of rows and petabytes of data. ClickHouse Cloud offers a scalable serverless solution for real-time analytics.
N/A
Pricing
Apache Kafka
ClickHouse
Editions & Modules
No answers on this topic
No answers on this topic
Offerings
Pricing Offerings
Apache Kafka
ClickHouse
Free Trial
No
Yes
Free/Freemium Version
No
Yes
Premium Consulting/Integration Services
No
Yes
Entry-level Setup Fee
No setup fee
Optional
Additional Details
—
Pay for what is used:
It automatically scales up and down compute resources based on the user's workload
It scales storage and compute separately
It automatically scales unused resources down to zero so that users don’t pay for idle services
More Pricing Information
Community Pulse
Apache Kafka
ClickHouse
Considered Both Products
Apache Kafka
Verified User
Engineer
Chose Apache Kafka
All stack tech helps our app and system. These technologies allow us to have the data available faster between different regions (due to our particular configuration) and thus the data and processing load of each system is lower. This allows the systems to be used more …
ClickHouse was not compared to them as a competitor but as the ideal partner to complete an information analysis system, providing users with the most complete and efficient tools. Therefore, in this case it was considered that it would be the ideal candidate due to its …
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
The most important thing when using ClickHouse is to be clear that the scenarios in which you want to use it really are the right ones. Many users think that when a database is very fast for a specific use case, it can be extrapolated to other contexts (most of the time different) in which a previous analysis has not been carried out.
ClickHouse is an analytical database, as such, it should be used for such purposes, where the information is stored correctly, the data volumes are really large and the queries to be performed are not the typical traditional queries on several columns with multiple aggregations. ClickHouse is not the solution for this.
On the other hand, if your case is not one of the above, it is quite possible that ClickHouse can help you. Where ClickHouse shines is when you are looking for aggregation over a particular column in large volumes of data.
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
Their MergeTree table engine provide impressive performance for data insert in bulk
Not only data insert but also the way MergeTree engine uses Primary Keys to sort the data and perform data skipping based on the granules its also their secret for ridiculous fast queries
Data compression its also great
They provide especial table engines that allow you to read data directly from other sources like S3
Since its written with C++ you have very granular data types and especial ones like enum, LowCardinality and etc, they save you a lot of storage since are stored as integer values
ClickHouse functions besides the ones that respect ANSI Standards are also awesome and useful
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
ClickHouse outperforms, especially in costs, since its compression/indexing engines are so smart, and even with very low computing power, you can already perform huge analyses of the data.
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
