Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
Associate in Information Technology at Cognizant (Entertainment, 10,001+ employees)
Chose Cassandra
Cassandra has its own use case. It performs very well as a data store. Data can be written to it at a high rate. It cannot be compared to traditional RDBMS like Oracle, because they all have their own usage. Even MongoDB, which is somewhat similar, cannot be stacked up against …
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Apache Cassandra is a NoSQL database and well suited where you need highly available, linearly scalable, tunable consistency and high performance across varying workloads. It has worked well for our use cases, and I shared my experiences to use it effectively at the last Cassandra summit! http://bit.ly/1Ok56TK It is a NoSQL database, finally you can tune it to be strongly consistent and successfully use it as such. However those are not usual patterns, as you negotiate on latency. It works well if you require that. If your use case needs strongly consistent environments with semantics of a relational database or if the use case needs a data warehouse, or if you need NoSQL with ACID transactions, Apache Cassandra may not be the optimum choice.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
Continuous availability: as a fully distributed database (no master nodes), we can update nodes with rolling restarts and accommodate minor outages without impacting our customer services.
Linear scalability: for every unit of compute that you add, you get an equivalent unit of capacity. The same application can scale from a single developer's laptop to a web-scale service with billions of rows in a table.
Amazing performance: if you design your data model correctly, bearing in mind the queries you need to answer, you can get answers in milliseconds.
Time-series data: Cassandra excels at recording, processing, and retrieving time-series data. It's a simple matter to version everything and simply record what happens, rather than going back and editing things. Then, you can compute things from the recorded history.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Cassandra runs on the JVM and therefor may require a lot of GC tuning for read/write intensive applications.
Requires manual periodic maintenance - for example it is recommended to run a cleanup on a regular basis.
There are a lot of knobs and buttons to configure the system. For many cases the default configuration will be sufficient, but if its not - you will need significant ramp up on the inner workings of Cassandra in order to effectively tune it.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
I would recommend Cassandra DB to those who know their use case very well, as well as know how they are going to store and retrieve data. If you need a guarantee in data storage and retrieval, and a DB that can be linearly grown by adding nodes across availability zones and regions, then this is the database you should choose.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
We evaluated MongoDB also, but don't like the single point failure possibility. The HBase coupled us too tightly to the Hadoop world while we prefer more technical flexibility. Also HBase is designed for "cold"/old historical data lake use cases and is not typically used for web and mobile applications due to its performance concern. Cassandra, by contrast, offers the availability and performance necessary for developing highly available applications. Furthermore, the Hadoop technology stack is typically deployed in a single location, while in the big international enterprise context, we demand the feasibility for deployment across countries and continents, hence finally we are favor of Cassandra
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
I have no experience with this but from the blogs and news what I believe is that in businesses where there is high demand for scalability, Cassandra is a good choice to go for.
Since it works on CQL, it is quite familiar with SQL in understanding therefore it does not prevent a new employee to start in learning and having the Cassandra experience at an industrial level.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
