Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
Cloud BigTable
Score 8.6 out of 10
N/A
Google's Cloud Bigtable is a fully managed, scalable NoSQL database service for large analytical and operational workloads with up to 99.999% availability.
$0.03
per month
Pricing
Apache Kafka
Google Cloud BigTable
Editions & Modules
No answers on this topic
Backup Storage
$0.026
per month per GB
HDD storage
$0.026
per month per GB
SSD storage
$0.17
per month per GB
Nodes
$0.65/hour
per month per node (minimum 1 nodes)
Offerings
Pricing Offerings
Apache Kafka
Cloud BigTable
Free Trial
No
No
Free/Freemium Version
No
No
Premium Consulting/Integration Services
No
No
Entry-level Setup Fee
No setup fee
No setup fee
Additional Details
—
—
More Pricing Information
Community Pulse
Apache Kafka
Google Cloud BigTable
Features
Apache Kafka
Google Cloud BigTable
Database-as-a-Service
Comparison of Database-as-a-Service features of Product A and Product B
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
Google Bigtable is ONLY suited for massive data sets which scale PetaBytes and TerraBytes. Anything under this can easily be done via dedicated VMs and open source tools. Google Bigtable is expensive and shall be used wisely. It should be utilised only where it is well suited else you would simply be wasting dollars and not utilizing its full benefits.
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
Analytics: is at Google's heart. No on can beat Google in this space and BigTable is one of its implementation of this. The insights you gain from BigTable are simply usable in your day to day activities and can help you make real difference.
Speed: Processing TBs and PBs of data under minutes needs real efficient platform which is capable of doing much more than just processing data. All this data cannot be processed by a single machine, but rather huge pairs of machines working in conjuction with each other. BigTable's implementation is one of the finest and allows you achieve great speeds!
Interface: is great. Google has segregated required task under logically placed buttons which takes no time by users to understand and get habituated.
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
User interface's responsiveness: I understand so much is going on under the hood, but laggyness is acceptable if a workload is running or being processed. In case their is not workload being process, GUI should work blazing fast. I have faced this at times, and this becomes frustrating as well.
Nothing other than this - BigTable is quite efficient platform and does exactly what it is built for.
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
For big IT firms like us, data is very important and it only holds its value if it can make sense to us. Therefore, Bigtable's usability is priceless when it comes to decision making based on data.
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
